A high-precision crack segmentation method based on double-flow visual base model collaboration

Abstract

The present application relates to a kind of high-precision crack segmentation method based on double-flow vision basic model cooperation, belong to computer vision and deep learning technical field, it aims at solving the problem of poor generalization of existing crack segmentation model under complex background, high-resolution under the problem of missing detection of fine crack, large model fine-tuning cost is high.The infrastructure surface image collected by the present application is preprocessed into 1024×1024 pixel standard input, and the features are extracted using a double-flow encoder with parallel SAM3 and DINOv3 double branches;Freeze the training parameters of double backbone, embed the lightweight adapter to complete efficient parameter fine-tuning;Extract double-branch multi-scale features, complete multi-scale splicing and dimensionality reduction fusion after channel alignment;Output the pixel-level crack binary segmentation result through the lightweight decoder based on depth separable convolution.The present application considers segmentation accuracy, cross-domain generalization ability and training deployment cost, and meets the engineering landing needs of infrastructure structure health monitoring.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, specifically relating to a high-precision crack segmentation method based on the collaboration of a dual-stream vision basic model. Background Technology

[0002] For critical infrastructure such as bridges, dams, roads and buildings, high-precision crack detection and quantitative segmentation are core components of structural health monitoring. Timely and accurate identification and quantification of cracks are key early warning methods to prevent catastrophic structural failures.

[0003] In recent years, crack segmentation models driven by deep learning have shown certain performance. However, existing technologies still suffer from the following major drawbacks in practical engineering applications: poor cross-domain generalization ability; existing models are highly dependent on specific datasets, and when faced with complex real-world visual environments (such as fallen leaves, weeds, gravel, oil stains, water stains, or strong light and shadow changes), they cannot effectively distinguish cracks from similar backgrounds, resulting in a sharp degradation in performance, serious false detection and false negative problems, and failing to meet the robustness requirements for engineering implementation; feature loss at high resolution; when processing high-resolution images of 1024×1024 and above, traditional convolutional networks struggle to balance global contextual information with the edge details of tiny cracks, leading to the discontinuity of long and thin cracks and the failure to detect micron-level tiny cracks, and pixel-level segmentation accuracy cannot meet the accuracy requirements of engineering monitoring; and high difficulty in adapting large models; although basic visual models such as SAM and DINO have powerful general visual knowledge, direct full-scale fine-tuning is computationally expensive and prone to "catastrophic forgetting," failing to retain pre-trained general features. At the same time, a single model often struggles to simultaneously consider the geometric continuity and texture discrimination of cracks, resulting in poor adaptability of large models to vertical engineering domains.

[0004] Therefore, there is an urgent need for a crack segmentation method that can utilize general knowledge from large models, adapt to specific engineering scenarios at low cost, and possess high precision and anti-interference capabilities. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems by providing a high-precision crack segmentation method based on a collaborative dual-stream vision model. This method solves the issues of weak generalization ability and severe false positives and false negatives in existing crack segmentation models under complex engineering backgrounds, improving the robustness of the model in unseen scenarios. It also addresses the problems of feature loss and missed detection of minor cracks in high-resolution images, balancing the capture of global crack topology and local texture details to improve pixel-level segmentation accuracy. Furthermore, it addresses the high cost and catastrophic forgetting issues of full-scale fine-tuning of large vision models, enabling low-cost, lossless knowledge transfer from models with billions of parameters to the crack segmentation domain, lowering the training computational barrier. Finally, it provides a crack segmentation method that balances segmentation accuracy, cross-domain generalization ability, and training and deployment costs, meeting the engineering implementation needs of infrastructure structural health monitoring.

[0006] The method includes the following steps:

[0007] S1. Collect surface images of the infrastructure to be inspected, uniformly adjust them to a high resolution size of 1024×1024 pixels, perform normalization and standardization preprocessing to obtain model input data;

[0008] S2. Construct a two-stream encoder that includes a first feature extraction branch and a second feature extraction branch with parallel settings; input the model input data into the two-stream encoder, the first feature extraction branch loads a pre-trained SAM3 backbone network to extract the geometric structure and object features of the image; the second feature extraction branch loads a pre-trained DINOv3 backbone network to extract the local texture and semantic features of the image;

[0009] S3. Freeze all pre-trained parameters of the SAM3 backbone network and the DINOv3 backbone network, embed trainable lightweight adapter modules in the Transformer blocks of the two backbone networks, and map the pre-trained general visual features to the feature representations used in the crack segmentation task by training the adapter modules.

[0010] S4. Extract multi-scale feature maps of the dual-stream encoder at different levels; use an aligned convolutional layer to map the feature channel dimension of the second feature extraction branch to be consistent with that of the first feature extraction branch; at multiple resolution scales, concatenate and reduce the dimensions of the aligned dual-stream features to obtain a multi-scale fused feature map;

[0011] S5. Construct a lightweight decoder based on depthwise separable convolution, receive multi-scale fused feature maps, restore image resolution step by step through cascaded upsampling modules, and output pixel-level crack binary segmentation prediction maps.

[0012] Optionally, in step S2:

[0013] The SAM3 backbone network is configured with img_size=1024, patch_size=14, embed_dim=1024, depth=32, num_heads=16, mlp_ratio=4.625, window_size=24, and global attention blocks at layers 7, 15, 23, and 31.

[0014] The DINOv3 backbone network is configured with patch_size=16, embed_dim=768, depth=12, and the feature output layers are layers 2, 5, 8, and 11.

[0015] Optionally, in step S3:

[0016] The adapter module adopts a serial bottleneck structure with residual connections, including a dimension-reduced linear layer, a GELU nonlinear activation layer, and a dimension-upgrading linear layer connected in sequence. The operation logic is as follows:

[0017] ;

[0018] Where x represents the input features of the backbone network layer, and x' represents the output features after injection into the adapter module. For linear upsampling, For linear downsampling, For activation functions;

[0019] The adapter module is embedded in series between the attention layer and the feedforward neural network layer of the backbone network, or in parallel within the attention layer or the feedforward neural network layer of the backbone network.

[0020] Optionally, the specific steps of step S4 are as follows:

[0021] S41. Extract the feature maps of layers 2, 5, 8, and 11 from the DINOv3 backbone network, and map and align their channel count from 768 dimensions to 1024 dimensions using 1×1 convolution;

[0022] S42. Extract feature maps of the corresponding scale from the SAM3 backbone network, with 1024 channels;

[0023] S43. At four resolution scales of 1 / 4, 1 / 8, 1 / 16 and 1 / 32, the aligned DINOv3 feature map and the SAM3 feature map are concatenated in the channel dimension to obtain a 2048-dimensional hybrid feature.

[0024] S44. A convolutional layer is used to reduce the number of channels of the hybrid features from 2048 dimensions to 128 dimensions, thereby completing feature fusion and obtaining a multi-scale fused feature map.

[0025] Optionally, in step S5, the lightweight decoder consists of four cascaded upsampling modules, each of which sequentially includes a 2x bilinear interpolation upsampling layer, a feature stitching layer at the same resolution, and a lightweight convolutional block (LightBlock).

[0026] Optionally, the structure of the lightweight convolutional block LightBlock is as follows:

[0027] The input features are first reduced in dimensionality by 1×1 convolution and then split into two paths. One path extracts deep features through cascaded 3×3 depth convolution, BN layer, and GELU activation layer, while the other path keeps the original features unchanged.

[0028] After the two features are spliced ​​together, they are fused by 1×1 convolution and restored to the output channel, which is then output in conjunction with the BN layer and the GELU activation layer.

[0029] Optionally, the method further includes a model training step:

[0030] S6. Use a joint loss function combining binary cross-entropy loss and Dice coefficient loss to perform end-to-end training on the network constructed in steps S2-S5.

[0031] Optionally, in step S6, the formula for the joint loss function is:

[0032] L = L BCE + L Dice ;

[0033] in, L BCE The binary cross-entropy loss is used to supervise pixel classification accuracy. L Dice The Dice coefficient loss is used to monitor the overall overlap of the crack region.

[0034] As can be seen from the above technical solutions, the present invention has the following advantages:

[0035] This invention utilizes a parallel SAM3 and DINOv3 dual-stream encoder to extract global geometric features and local texture semantic features of cracks, respectively. This accurately captures the slender and continuous topological structure of cracks, solving the problem of discontinuous and missed detections of subtle cracks at high resolution. It also effectively distinguishes cracks from similar interference backgrounds such as water stains and oil, reducing the false detection rate. Furthermore, it employs an efficient fine-tuning scheme that freezes all pre-trained parameters of the dual-stream backbone network and trains only the embedded lightweight adapter module. Although the total number of parameters in the large dual-stream model is approximately 538.9M, the actual trainable parameters are only 7M, accounting for less than 1.3%, requiring only a consumer-grade display. The card can complete efficient training, reducing the computing power threshold; at the same time, it completely avoids the catastrophic forgetting problem caused by full fine-tuning, and fully retains the general visual features of the large model, realizing low-cost and lossless knowledge transfer from the large model to the vertical domain of crack segmentation; it adopts LightBlock based on depthwise separable convolution design to build a lightweight decoder, which can achieve pixel-level resolution recovery with low memory usage for 1024×1024 high-resolution input, ensuring high-precision segmentation effect while taking into account the model inference speed, and can be adapted to the deployment needs of mid-end and edge in engineering scenarios, with strong practicality for engineering implementation. Attached Figure Description

[0036] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a high-precision crack segmentation method based on a dual-stream vision model in a specific embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the network architecture of the dual-stream vision basic model in a specific embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the structure of the LightBlock, a lightweight convolutional block, in a specific embodiment of the present invention.

[0040] Figure 4 The images shown are the original test set image, the manually annotated mask image, and the segmentation mask image of the present invention in a specific embodiment of the present invention. Figure 4 (a) in the image is the original image of the test set. Figure 4 (b) in the image is a manually annotated mask. Figure 4 (c) in the figure is the segmentation mask diagram of the present invention;

[0041] Figure 5This is an example of the original image of the test set, the manually annotated mask image, and the segmentation mask image of this invention in another specific embodiment of the present invention. Figure 5 (a) in the image is the original image of the test set. Figure 5 (b) in the image is a manually annotated mask. Figure 5 (c) in the figure is the segmentation mask diagram of the present invention;

[0042] Figure 6 The images provided are the original image of the Road420 dataset (not used in training), the manually annotated mask image, and the segmentation mask image of this invention, as shown in this specific embodiment of the invention. Figure 6 Image (a) in the image is the original image from the Road420 dataset (not used in training). Figure 6 (b) in the image is a manually annotated mask. Figure 6 (c) in the figure is the segmentation mask diagram of the present invention;

[0043] Figure 7 In another specific embodiment of the present invention, the original image of the Road420 dataset (not used in training), the manually annotated mask image, and the segmentation mask image of the present invention are shown, wherein, Figure 7 Image (a) in the image is the original image from the Road420 dataset (not used in training). Figure 7 (b) in the image is a manually annotated mask. Figure 7 (c) in the figure is the segmentation mask diagram of the present invention;

[0044] Figure 8 The images shown are the original image, manually annotated mask image, and segmentation mask image of the Concrete3K dataset (not used in training) in a specific embodiment of the present invention. Figure 8 Image (a) in the image is the original image from the Concrete3K dataset (not used in training). Figure 8 (b) in the image is a manually annotated mask. Figure 8 (c) in the figure is the segmentation mask diagram of the present invention;

[0045] Figure 9 In another specific embodiment of the present invention, the original image of the Concrete3K dataset (not used in training), the manually annotated mask image, and the segmentation mask image of the present invention are shown, wherein, Figure 9 Image (a) in the image is the original image from the Concrete3K dataset (not used in training). Figure 9 (b) in the image is a manually annotated mask. Figure 9 (c) in the figure is the segmentation mask diagram of the present invention. Detailed Implementation

[0046] The various embodiments of the invention will be described more fully in the detailed steps of the high-precision crack segmentation method based on a dual-stream vision model, which will be described in detail below. The invention may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the invention to the specific embodiments disclosed herein, but rather the invention should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the invention.

[0047] It should be understood that, when used in this specification, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0048] The terms "one embodiment" or "some embodiments" used in this invention mean that one or more embodiments of the invention include the specific features, structures, or characteristics described in that embodiment. Therefore, the terms "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of the invention do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0049] To make the objectives, features, and advantages of this invention more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solutions protected by this invention. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0050] Please see Figure 1 The diagram shows a flowchart of a high-precision crack segmentation method based on a dual-stream vision model. The method includes:

[0051] S1. Collect surface images of the infrastructure to be inspected, uniformly adjust them to a high resolution size of 1024×1024 pixels, perform normalization and standardization preprocessing to obtain model input data;

[0052] S2. Construct a two-stream encoder that includes a first feature extraction branch and a second feature extraction branch with parallel settings; input the model input data into the two-stream encoder, the first feature extraction branch loads a pre-trained SAM3 backbone network to extract the geometric structure and object features of the image; the second feature extraction branch loads a pre-trained DINOv3 backbone network to extract the local texture and semantic features of the image;

[0053] S3. Freeze all pre-trained parameters of the SAM3 backbone network and the DINOv3 backbone network, embed trainable lightweight adapter modules in the Transformer blocks of the two backbone networks, and map the pre-trained general visual features to the feature representations used in the crack segmentation task by training the adapter modules.

[0054] S4. Extract multi-scale feature maps of the dual-stream encoder at different levels; use an aligned convolutional layer to map the feature channel dimension of the second feature extraction branch to be consistent with that of the first feature extraction branch; at multiple resolution scales, concatenate and reduce the dimensions of the aligned dual-stream features to obtain a multi-scale fused feature map;

[0055] S5. Construct a lightweight decoder based on depthwise separable convolution, receive multi-scale fused feature maps, restore image resolution step by step through cascaded upsampling modules, and output pixel-level crack binary segmentation prediction maps.

[0056] As a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another high-precision crack segmentation method based on dual-stream vision model collaboration is provided. This method includes the following steps:

[0057] S1. Collect surface images of the infrastructure to be inspected, uniformly adjust them to a high resolution size of 1024×1024 pixels, perform normalization and standardization preprocessing to obtain model input data;

[0058] It should be noted that the surface images of the infrastructure to be inspected include road surfaces, water conservancy dams, bridge structures, etc.

[0059] S2. Construct a two-stream encoder that includes a first feature extraction branch and a second feature extraction branch with parallel settings; input the model input data into the two-stream encoder, the first feature extraction branch loads a pre-trained SAM3 backbone network to extract the geometric structure and object features of the image; the second feature extraction branch loads a pre-trained DINOv3 backbone network to extract the local texture and semantic features of the image;

[0060] In step S2:

[0061] The SAM3 backbone network is configured with img_size=1024, patch_size=14, embed_dim=1024, depth=32, num_heads=16, mlp_ratio=4.625, window_size=24, and global attention blocks at layers 7, 15, 23, and 31.

[0062] The DINOv3 backbone network is configured with patch_size=16, embed_dim=768, depth=12, and the feature output layers are layers 2, 5, 8, and 11.

[0063] S3. Freeze all pre-trained parameters of the SAM3 backbone network and the DINOv3 backbone network, embed trainable lightweight adapter modules in the Transformer blocks of the two backbone networks, and map the pre-trained general visual features to the feature representations used in the crack segmentation task by training the adapter modules.

[0064] In step S3:

[0065] The adapter module adopts a serial bottleneck structure with residual connections, including a dimension-reduced linear layer, a GELU nonlinear activation layer, and a dimension-upgrading linear layer connected in sequence. The operation logic is as follows:

[0066] ;

[0067] Where x represents the input features of the backbone network layer, and x' represents the output features after injection into the adapter module. For linear upsampling, For linear downsampling, For activation functions;

[0068] The SAM3 branch adapter reduces 1024-dimensional input features to 32 dimensions and then restores them to 1024 dimensions; the DINOv3 branch adapter reduces 768-dimensional input features to 32 dimensions and then restores them to 768 dimensions.

[0069] The adapter module is embedded in series between the attention layer and the feedforward neural network layer of the backbone network, or in parallel within the attention layer or the feedforward neural network layer of the backbone network.

[0070] S4. Extract multi-scale feature maps of the dual-stream encoder at different levels; use an aligned convolutional layer to map the feature channel dimension of the second feature extraction branch to be consistent with that of the first feature extraction branch; at multiple resolution scales, concatenate and reduce the dimensions of the aligned dual-stream features to obtain a multi-scale fused feature map;

[0071] The specific steps of step S4 are as follows:

[0072] S41. Extract the feature maps of layers 2, 5, 8, and 11 from the DINOv3 backbone network, and map and align their channel count from 768 dimensions to 1024 dimensions using 1×1 convolution;

[0073] S42. Extract feature maps of the corresponding scale from the SAM3 backbone network, with 1024 channels;

[0074] S43. At four resolution scales of 1 / 4, 1 / 8, 1 / 16 and 1 / 32, the aligned DINOv3 feature map and the SAM3 feature map are concatenated in the channel dimension to obtain a 2048-dimensional hybrid feature.

[0075] S44. A 1×1 convolutional layer is used to reduce the number of channels of the hybrid features from 2048 dimensions to 128 dimensions to complete feature fusion and obtain a multi-scale fused feature map.

[0076] S5. Construct a lightweight decoder based on depthwise separable convolution, receive multi-scale fused feature maps, restore image resolution step by step through cascaded upsampling modules, and output pixel-level crack binary segmentation prediction maps.

[0077] In step S5, the lightweight decoder consists of four cascaded upsampling modules. Each upsampling module sequentially includes a 2x bilinear interpolation upsampling layer, a feature stitching layer with the same resolution, and a lightweight convolutional block (LightBlock).

[0078] like Figure 3 As shown, the structure of the lightweight convolutional block LightBlock is as follows:

[0079] The input features are first reduced in dimensionality by 1×1 convolution and then split into two paths. One path extracts deep features through cascaded 3×3 depth convolution, BN layer, and GELU activation layer, while the other path keeps the original features unchanged.

[0080] After the two features are spliced ​​together, they are fused by 1×1 convolution and restored to the output channel, which is then output in conjunction with the BN layer and the GELU activation layer.

[0081] It should be noted that this structure can significantly reduce the amount of computation while maintaining a large receptive field, adapting to the low memory usage requirements of high-resolution input.

[0082] In this embodiment of the invention, the decoding process is as follows: starting from the fused features with the lowest resolution of 1 / 32, after the upsampling module completes 2x upsampling, it is concatenated with the fused features of the same resolution, and then the LightBlock completes feature optimization, and the feature recovery of 1 / 16, 1 / 8 and 1 / 4 resolutions is completed step by step; finally, after upsampling and feature optimization of 1 / 4 to 1 / 2 resolution, the 128-dimensional features are reduced to 1 dimension by a 1×1 convolution head, and then bilinear interpolation is used to restore the input size to 1024×1024, and the pixel-level crack binary segmentation map is output.

[0083] The method also includes a model training step:

[0084] S6. Use a joint loss function combining binary cross-entropy loss and Dice coefficient loss to perform end-to-end training on the network constructed in steps S2-S5.

[0085] In step S6, the formula for the joint loss function is:

[0086] L = L BCE + L Dice ;

[0087] in, L BCE The binary cross-entropy loss is used to supervise pixel classification accuracy. L Dice The Dice coefficient loss is used to monitor the overall overlap of the crack region.

[0088] As a refinement and extension of the specific implementation of the above embodiments, the present invention provides another embodiment, such as... Figure 2 The diagram shown is a schematic representation of the network architecture in this embodiment:

[0089] 1. Dual-stream feature extraction encoder:

[0090] SAM3 Branch: Input a 1024×1024 RGB image, which is then processed through a 14×14 Patch Embedding before entering the SAM3 ViT backbone network. The backbone network has a fixed configuration of img_size=1024, embed_dim=1024, depth=32, num_heads=16, mlp_ratio=4.625, window_size=24, and global attention blocks at layers 7, 15, 23, and 31. All backbone parameters of this branch are frozen during training.

[0091] DINOv3 branch: Input a 1024×1024 RGB image, which is then processed through a 16×16 Patch Embedding and fed into the DINOv3 ViT-B / 16 backbone network; the backbone network is configured with patch_size=16, embed_dim=768, and depth=12, and the outputs of layers 2, 5, 8, and 11 are selected as the multi-scale feature pyramid; all backbone parameters of this branch are frozen during training.

[0092] 2. Adapter injection:

[0093] Lightweight adapter modules are injected into the 32 Transformer Blocks of the SAM3 branch and the 12 Transformer Blocks of the DINOv3 branch. The structure of the adapter module is: Linear (dim→32) dimensionality reduction projection layer → GELU activation function → Linear (32→dim) dimensionality increase projection layer, with residual connections. The dim=1024 of the SAM3 branch and the dim=768 of the DINOv3 branch. During model training, only the parameters of the adapter module participate in gradient updates, while the parameters of the backbone network remain fixed.

[0094] 3. Feature Alignment and Fusion Module:

[0095] (1) Channel alignment: The four sets of 768-dimensional multi-scale features output by the DINOv3 branch are mapped to 1024 dimensions through four 1×1 convolutional layers, thus unifying the channel dimensions with those of the SAM3 branch features.

[0096] (2) Resolution alignment: The DINOv3 features and SAM3 branch features after channel alignment are adjusted to the same spatial resolution by bilinear interpolation to generate feature pairs at four scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32.

[0097] (3) Feature fusion: The DINOv3 features aligned at each scale are concatenated with the SAM3 features along the channel dimension to obtain a 2048-dimensional hybrid feature. Then, the 2048-dimensional feature is reduced to 128-dimensional through a 1×1 convolutional dimensionality reduction layer to obtain 4 sets of fused feature maps.

[0098] 4. Lightweight decoder:

[0099] The decoder consists of four cascaded upsampling modules and LightBlock lightweight convolutional blocks. LightBlock employs grouped convolution and depthwise separable convolution design, which significantly reduces computational load and memory usage. The decoder restores the image resolution step by step through four 2x bilinear interpolation upsampling operations, and finally outputs a 1024×1024 single-channel crack binary segmentation mask.

[0100] Model training: Completed using the PyTorch deep learning framework, with a hardware environment consisting of two NVIDIA RTX 4090 GPUs (24GB VRAM per card).

[0101] 1. Dataset Construction: A mixed training dataset containing 5684 1024×1024 pixel images was constructed. The dataset covers concrete pavement and asphalt pavement scenes, and also includes a large number of samples with strong interference such as water stains, shadows, gravel, and weeds. All samples have pixel-level crack annotations.

[0102] 2. Training hyperparameter settings: The optimizer used is AdamW, the initial learning rate is set to 0.0001, the batch size is set to 4, and the total number of training epochs is set to 30.

[0103] 3. Loss Function: A joint loss function combining binary cross-entropy loss and Dice coefficient loss is used, as shown in the formula below. L = L BCE + L Dice ,in L BCE For binary cross-entropy loss, L Dice This represents the Dice coefficient loss.

[0104] 4. Training process: Freeze all parameters of the SAM3 and DINOv3 backbone networks, and train only the parameters of the adapter module, feature alignment and fusion module, and decoder module; after completing 30 epochs of training, save the model weights for inference and validation.

[0105] Model performance verification experiments:

[0106] 1. Quantitative performance verification:

[0107] UNet and SegFormer were selected as the baselines for comparison with traditional convolutional / transformer models, and CrackSAM was selected as the baseline for comparison with similar large models. Performance tests were conducted on the test set in this paper and the cross-domain test sets Road420 and Concrete3K. The test results are shown in Table 1.

[0108] Table 1 Model test results

[0109]

[0110] Quantitative results show that the method of this invention significantly outperforms traditional models in F1 and IoU values ​​on the test set, while maintaining extremely high performance on unseen cross-domain datasets and exhibiting strong cross-domain generalization ability; moreover, the number of trainable parameters is only 7M, and the training cost is far lower than that of existing models that require full fine-tuning.

[0111] 2. Qualitative performance verification:

[0112] A visual comparison of samples from the target test set, the Road420 cross-domain dataset, and the Concrete3K cross-domain dataset is performed, such as... Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Figure 9 As shown, Figure 4 (a) in the image is the original image of the test set. Figure 4 (b) in the image is a manually annotated mask. Figure 4 (c) in the figure is the segmentation mask diagram of the present invention; Figure 5 (a) in the image is the original image of the test set. Figure 5 (b) in the image is a manually annotated mask. Figure 5 (c) in the figure is the segmentation mask diagram of the present invention; Figure 6 Image (a) in the image is the original image from the Road420 dataset (not used in training). Figure 6 (b) in the image is a manually annotated mask. Figure 6 (c) in the figure is the segmentation mask diagram of the present invention; Figure 7 Image (a) in the image is the original image from the Road420 dataset (not used in training). Figure 7 (b) in the image is a manually annotated mask. Figure 7 (c) in the figure is the segmentation mask diagram of the present invention; Figure 8 Image (a) in the image is the original image from the Concrete3K dataset (not used in training). Figure 8 (b) in the image is a manually annotated mask. Figure 8 (c) in the figure is the segmentation mask diagram of the present invention; Figure 9 Image (a) in the image is the original image from the Concrete3K dataset (not used in training). Figure 9 (b) in the image is a manually annotated mask. Figure 9 (c) in the diagram represents the segmentation mask of this invention. The results show that:

[0113] (1) In the test set samples with complex backgrounds such as gravel and oil, traditional models such as UNet, SegFormer and CrackSAM produced a large number of false detections, while the method of this invention accurately segmented the crack area, and the segmentation accuracy was significantly improved.

[0114] (2) On the Road420 and Concrete3K cross-domain datasets that were not used for training, the cracks segmented by the method of the present invention maintained extremely high boundary integrity and no crack disconnection phenomenon occurred. In contrast, traditional models showed varying degrees of missed detection, disconnection and false detection, which proves the significant improvement effect of the method of the present invention on the generalization ability of the model.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-precision crack segmentation method based on a dual-stream vision fundamental model, characterized in that, The method includes the following steps: S1. Collect surface images of the infrastructure to be inspected, uniformly adjust them to a high resolution size of 1024×1024 pixels, perform normalization and standardization preprocessing to obtain model input data; S2. Construct a two-stream encoder that includes a first feature extraction branch and a second feature extraction branch with parallel settings; input the model input data into the two-stream encoder, the first feature extraction branch loads a pre-trained SAM3 backbone network to extract the geometric structure and object features of the image; the second feature extraction branch loads a pre-trained DINOv3 backbone network to extract the local texture and semantic features of the image; S3. Freeze all pre-trained parameters of the SAM3 backbone network and the DINOv3 backbone network, embed trainable lightweight adapter modules in the Transformer blocks of the two backbone networks, and map the pre-trained general visual features to the feature representations used in the crack segmentation task by training the adapter modules. S4. Extract multi-scale feature maps of the dual-stream encoder at different levels; use an aligned convolutional layer to map the feature channel dimension of the second feature extraction branch to be consistent with that of the first feature extraction branch; at multiple resolution scales, concatenate and reduce the dimensions of the aligned dual-stream features to obtain a multi-scale fused feature map; S5. Construct a lightweight decoder based on depthwise separable convolution, receive multi-scale fused feature maps, restore image resolution step by step through cascaded upsampling modules, and output pixel-level crack binary segmentation prediction maps. In step S3: The adapter module adopts a serial bottleneck structure with residual connections, including a dimension-reduced linear layer, a GELU nonlinear activation layer, and a dimension-upgrading linear layer connected in sequence. The operation logic is as follows: ； Where x represents the input features of the backbone network layer, and x' represents the output features after injection into the adapter module. For linear upsampling, For linear downsampling, For activation functions; The adapter module is embedded in series between the attention layer and the feedforward neural network layer of the backbone network, or in parallel within the attention layer or the feedforward neural network layer of the backbone network.

2. The high-precision crack segmentation method based on dual-stream vision model collaboration according to claim 1, characterized in that, In step S2: The SAM3 backbone network is configured with img_size=1024, patch_size=14, embed_dim=1024, depth=32, num_heads=16, mlp_ratio=4.625, window_size=24, and global attention blocks at layers 7, 15, 23, and 31. The DINOv3 backbone network is configured with patch_size=16, embed_dim=768, depth=12, and the feature output layers are layers 2, 5, 8, and 11.

3. The high-precision crack segmentation method based on dual-stream vision model collaboration according to claim 1, characterized in that, The specific steps of step S4 are as follows: S41. Extract the feature maps of layers 2, 5, 8, and 11 from the DINOv3 backbone network, and map and align their channel count from 768 dimensions to 1024 dimensions using 1×1 convolution; S42. Extract feature maps of the corresponding scale from the SAM3 backbone network, with 1024 channels; S43. At four resolution scales of 1 / 4, 1 / 8, 1 / 16 and 1 / 32, the aligned DINOv3 feature map and the SAM3 feature map are concatenated in the channel dimension to obtain a 2048-dimensional hybrid feature. S44. A convolutional layer is used to reduce the number of channels of the hybrid features from 2048 dimensions to 128 dimensions, thereby completing feature fusion and obtaining a multi-scale fused feature map.

4. The high-precision crack segmentation method based on dual-stream vision model collaboration according to claim 1, characterized in that, In step S5, the lightweight decoder consists of four cascaded upsampling modules. Each upsampling module sequentially includes a 2x bilinear interpolation upsampling layer, a feature stitching layer with the same resolution, and a lightweight convolutional block (LightBlock).

5. The high-precision crack segmentation method based on dual-stream vision model collaboration according to claim 4, characterized in that, The structure of the lightweight convolutional block LightBlock is as follows: The input features are first reduced in dimensionality by 1×1 convolution and then split into two paths. One path extracts deep features through cascaded 3×3 depth convolution, BN layer, and GELU activation layer, while the other path keeps the original features unchanged. After the two features are spliced ​​together, they are fused by 1×1 convolution and restored to the output channel, which is then output in conjunction with the BN layer and the GELU activation layer.

6. The high-precision crack segmentation method based on dual-stream vision fundamental model collaboration according to claim 1, characterized in that, The method also includes a model training step: S6. Use a joint loss function combining binary cross-entropy loss and Dice coefficient loss to perform end-to-end training on the network constructed in steps S2-S5.

7. The high-precision crack segmentation method based on dual-stream vision model collaboration according to claim 6, characterized in that, In step S6, the formula for the joint loss function is: L = L BCE + L Dice ； in, L BCE The binary cross-entropy loss is used to supervise pixel classification accuracy. L Dice The Dice coefficient loss is used to monitor the overall overlap of the crack region.

Images