Pavement crack detection method and system based on multi-scale cavity convolution and domain prompt generation

Abstract

The invention provides a pavement crack detection method and system based on multi-scale cavity convolution and domain prompt generation, and relates to the technical field of pavement crack detection, and the method comprises the steps: obtaining original input features extracted by an image encoder; the method comprises the following steps of: inserting a multi-scale cavity convolution adapter in front of each Transform layer of an image encoder; processing the original input features through a multi-scale cavity convolution adapter to generate enhanced image features; the enhanced image features are input into a domain prompt encoder, and based on the enhanced image features, the domain prompt encoder dynamically generates a domain prompt vector in semantic alignment with the current image through an internally constructed feature memory bank and a trans-attention mechanism; and performing sequence processing on the domain prompt vector, namely, reconstructing corresponding reconstructed image embedding according to the domain prompt decoder, and expanding the domain prompt vector to the same size as the original image embedding. According to the invention, the precision and generalization ability of pavement crack detection can be improved.

Description

Technical Field

[0001] This invention relates to the field of pavement crack detection technology, and in particular to a pavement crack detection method and system based on multi-scale void convolution and domain cue generation. Background Technology

[0002] With the acceleration of urbanization, the health of roads, as core transportation infrastructure, directly affects traffic safety and operational efficiency. Road surface cracks are the most common road surface defects. If they are not detected in a timely and accurate manner, they can easily lead to further road damage and traffic accidents. Therefore, automated and high-precision detection of road surface cracks has become a key task in urban infrastructure management.

[0003] In recent years, deep learning techniques such as Convolutional Neural Networks (CNNs) and Segmentation Models (SAMs) have been widely applied to pavement crack detection. However, most existing methods suffer from the following technical shortcomings: insufficient cross-domain generalization ability. This deficiency stems from the complexity and diversity of pavement crack sampling scenarios. For example, training datasets often only cover samples under specific lighting, material, or climatic conditions, while the prompting input mechanisms of existing models lack domain feature adaptation capabilities, resulting in limited perceptual adaptation to multi-scale crack features. Consequently, the detection performance of the model significantly decreases in non-training scenarios such as rainy days, different pavement materials, and drastic changes in lighting.

[0004] This defect leads to the following problems: the rate of missed and false detections of cracks increases in real-world complex road scenarios, making it impossible to provide a reliable basis for road surface repair and indirectly increasing traffic safety risks; because data needs to be collected repeatedly and models need to be retrained to adapt to different scenarios, it increases detection costs and reduces the efficiency of large-scale road monitoring, making it difficult to meet the needs of routine urban road management. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a road surface crack detection method and system based on multi-scale void convolution and domain cue generation, which can improve the model's ability to perceive multi-scale cracks and adapt to different scenarios, while improving the accuracy of road surface crack detection.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a method for detecting pavement cracks based on multi-scale void convolution and domain cue generation, the method comprising: Obtain the original input features extracted by the image encoder; insert a multi-scale dilated convolution adapter before each Transformer layer of the image encoder; process the original input features through the multi-scale dilated convolution adapter to generate enhanced image features; The enhanced image features are input into the domain cue encoder. Based on the enhanced image features, the domain cue encoder dynamically generates domain cue vectors that are semantically aligned with the current image through an internally built feature memory and cross-attention mechanism. The domain cue vectors are then processed sequentially, i.e., the corresponding reconstructed image embedding is reconstructed according to the domain cue decoder. The domain cue vectors are expanded to the same size as the original image embedding to calculate the similarity between the expanded vectors and the original image embedding. Finally, the vectors are fused to generate semantic cue vectors. A joint loss function for model training is constructed based on semantic cue vectors and original image embeddings. The joint loss function consists of two parts: supervised segmentation loss based on semantic cue vectors and self-supervised reconstruction loss based on reconstructed image embeddings and original image embeddings. Using a dataset containing road surface crack images and labels, the model is trained based on the constructed joint loss function to obtain a trained crack detection model; In actual detection, the image of the road surface to be detected is resized and then input into the trained crack detection model for inference to generate a mask image of the crack area. By restoring the mask image of the crack area to the original image size, the location and visualization of road cracks can be achieved.

[0007] Furthermore, the original input features extracted by the image encoder are obtained; a multi-scale dilated convolutional adapter is inserted before each Transformer layer of the image encoder; the original input features are processed by the multi-scale dilated convolutional adapter to generate enhanced image features, including: Obtain the raw input features extracted by the image encoder and input the raw input features into the multi-scale dilated convolution adapter; The original input features are processed by multiple parallel dilated convolution branches in the multi-scale dilated convolution adapter, generating multiple intermediate branch features corresponding to different scales. A channel attention mechanism is used to enhance each intermediate branch feature corresponding to different scales, generating enhanced intermediate branch features. All enhanced intermediate branch features are fused to obtain the fused feature; The fused features are residually concatenated with the original input features to obtain enhanced image features.

[0008] Furthermore, the enhanced image features are input into the domain cue encoder. Based on these features, the domain cue encoder dynamically generates domain cue vectors semantically aligned with the current image through its internally built feature memory and cross-attention mechanism. The domain cue vectors undergo sequential processing, namely, reconstructing the corresponding reconstructed image embedding based on the domain cue decoder. The domain cue vectors are then expanded to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding. Finally, these are fused to generate a semantic cue vector, including: The enhanced image features are input into the domain cue encoder; Based on enhanced image features, the domain cue encoder dynamically generates domain cue vectors that are semantically aligned with the current image through an internally built feature memory and cross-attention mechanism. The domain cue vector is first processed by inputting the domain cue vector into the domain cue decoder to reconstruct the corresponding reconstructed image embedding. The domain cue vector is then subjected to a second processing step, namely, expansion of the domain cue vector to generate an expanded domain cue vector with the same embedding size as the original image. Calculate the similarity between the extended domain cue vector and the original image embedding to obtain a similarity feature map; The semantic cue vector is generated by fusing extended domain cue vectors, similarity feature maps, and original image embeddings.

[0009] Furthermore, a joint loss function for model training is constructed based on the semantic cue vector and the original image embedding. This joint loss function consists of two parts: a supervised segmentation loss based on the semantic cue vector and a self-supervised reconstruction loss based on both the reconstructed and original image embeddings. Obtain the semantic cue vector and the original image embedding; Based on the semantic cue vector and the original image embedding, the supervised segmentation loss is calculated as a weighted combination of cross-entropy loss and Dice coefficient loss; The self-supervised reconstruction loss is calculated using the mean square error function based on the reconstructed image embedding and the original image embedding. The calculated supervised segmentation loss and the calculated self-supervised reconstruction loss are weighted and summed to generate a joint loss function for model training.

[0010] Furthermore, using a dataset containing road surface crack images and labels, the model is trained based on the constructed joint loss function to obtain a trained crack detection model, including: Prepare a dataset containing images of road surface cracks and their corresponding labels, and divide the dataset into a training set and a validation set; Based on the dataset, a training model containing a multi-scale dilated convolutional adapter, a domain cue encoder, a domain cue decoder, and a mask decoder is loaded and initialized, while the backbone weights of the image encoder are frozen to obtain the initialized training model. The images in the training set are input into the initialized model to be trained, and the trainable parameters in the initialized model are iteratively optimized based on the joint loss function and the backpropagation algorithm to obtain the updated model. The performance of the model after updating parameters is evaluated using a validation set. When the evaluation metrics stabilize or reach the preset number of training rounds, training is stopped and the final model parameters are saved to obtain the trained crack detection model.

[0011] Furthermore, during actual detection, the image of the road surface to be detected is resized and input into the trained crack detection model for inference, generating a mask image of the crack region. By restoring the mask image of the crack region to the original image size, the localization and visualization of road cracks are achieved, including: The road surface image to be detected is adjusted to the input size specified by the trained crack detection model to obtain a size-fitted image to be detected; The size-adapted image to be detected is input into the trained crack detection model for forward inference to generate an initial crack mask map corresponding to the input size. The initial crack mask image is restored to the original size of the road surface image to be detected through bilinear interpolation to obtain the final crack mask image; Based on the final crack mask image, the crack area is located and visualized on the road surface image to be detected.

[0012] Furthermore, the initial crack mask image is restored to the original size of the road surface image to be detected through bilinear interpolation to obtain the final crack mask image, including: Obtain the initial crack mask map corresponding to the input size, and obtain the original size of the road surface image to be detected; Based on the original size of the road surface image to be detected and the current size of the initial crack mask image corresponding to the input size, calculate the spatial scale transformation parameters; Based on the spatial scale transformation parameters, a bilinear interpolation algorithm is used to perform a size restoration operation on the initial crack mask image to obtain the restored crack mask image. The restored crack mask image is output as the final crack mask image with the same dimensions as the original.

[0013] Secondly, a pavement crack detection system based on multi-scale void convolution and domain cueing includes: The acquisition and insertion module is used to acquire the original input features extracted by the image encoder; insert a multi-scale dilated convolution adapter before each Transformer layer of the image encoder; and process the original input features through the multi-scale dilated convolution adapter to generate enhanced image features. The fusion generation module is used to input the enhanced image features into the domain cue encoder. Based on the enhanced image features, the domain cue encoder dynamically generates domain cue vectors that are semantically aligned with the current image through an internally built feature memory and cross-attention mechanism. The domain cue vectors are then subjected to sequence processing, i.e., the corresponding reconstructed image embedding is reconstructed according to the domain cue decoder. The domain cue vectors are expanded to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding. Finally, the semantic cue vectors are fused to generate the semantic cue vector. The function building module is used to construct a joint loss function for model training based on semantic cue vectors and original image embeddings. The joint loss function consists of two parts: supervised segmentation loss based on semantic cue vectors and self-supervised reconstruction loss based on reconstructed image embeddings and original image embeddings. The model training module is used to train the model based on the constructed joint loss function using a dataset containing road crack images and labels, to obtain a trained crack detection model. The final inference and restoration module is used to resize the road surface image to be detected during actual detection and input it into the trained crack detection model for inference to generate a mask image of the crack area. By restoring the mask image of the crack area to the original image size, the localization and visualization of road cracks can be achieved.

[0014] Thirdly, a computing device includes: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0015] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0016] The above-described solution of the present invention has at least the following beneficial effects: By employing a multi-scale dilated convolutional adapter (MACA), the model's ability to perceive multi-scale crack features is enhanced, overcoming the technical challenge of traditional methods in extracting subtle and wide-scale crack features in complex scenes. This improves the accuracy and boundary clarity of crack segmentation. Simultaneously, the introduction of a domain cue encoder (DPE) and its memory mechanism dynamically generates domain cue vectors semantically aligned with the input image, addressing the model's weak generalization ability under different lighting, material, and weather conditions. This effectively enhances the adaptability and robustness of cross-domain detection. Furthermore, the construction of a joint loss function combining supervised segmentation and self-supervised reconstruction achieves synergistic optimization of segmentation accuracy and feature representation robustness, enabling the model to maintain high accuracy and stable crack detection performance across various real-world road scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a road surface crack detection method based on multi-scale void convolution and domain hint generation provided by an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of a road surface crack detection system based on multi-scale void convolution and domain hint generation, provided by an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0020] like Figure 1 As shown, an embodiment of the present invention proposes a method for detecting pavement cracks based on multi-scale void convolution and domain cueing, the method comprising the following steps: Step 1: Obtain the original input features extracted by the image encoder; insert a multi-scale dilated convolution adapter before each Transformer layer of the image encoder; process the original input features through the multi-scale dilated convolution adapter to generate enhanced image features; Step 2: The enhanced image features are input into the domain cue encoder. Based on the enhanced image features, the domain cue encoder dynamically generates a domain cue vector that is semantically aligned with the current image through its internally built feature memory and cross-attention mechanism. The domain cue vector is then processed sequentially, i.e., the corresponding reconstructed image embedding is reconstructed according to the domain cue decoder. The domain cue vector is expanded to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding. Finally, the vectors are fused to generate a semantic cue vector. Step 3: Based on the semantic cue vector and the original image embedding, construct a joint loss function for model training; the joint loss function consists of two parts: supervised segmentation loss based on the semantic cue vector and self-supervised reconstruction loss based on the reconstructed image embedding and the original image embedding. Step 4: Using a dataset containing road surface crack images and labels, train the model based on the constructed joint loss function to obtain the trained crack detection model. Step 5: During actual detection, the image of the road surface to be detected is resized and then input into the trained crack detection model for inference to generate a mask image of the crack area. By restoring the mask image of the crack area to the original image size, the location and visualization of road cracks can be achieved.

[0021] In this embodiment of the invention, by introducing a multi-scale dilated convolutional adapter, the model's ability to extract crack features at different scales is enhanced, effectively improving the accuracy of crack segmentation and the preservation of boundary details. Simultaneously, by dynamically generating cue vectors aligned with image semantics using a domain cue encoder, the model's cross-domain adaptability and generalization performance under varying lighting, road surface materials, and weather conditions are effectively improved. Furthermore, by constructing a joint loss function that integrates supervised and self-supervised learning, the robustness of feature representation and segmentation accuracy are further optimized, enabling the overall method to achieve highly reliable and accurate automated detection of road surface cracks in various real-world road scenarios, demonstrating significant practical application value.

[0022] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1: Obtain the original input features extracted by the image encoder and input them into the multi-scale dilated convolution adapter. Specifically, this includes: first, acquiring the original image of the road surface crack to be detected and adjusting its size to 448×448×3 according to a unified standard to ensure consistent input specifications for all subsequent feature processing steps; then, inputting the resized image into the image encoder of the crack detection model. Through the built-in patch embedding operation of the image encoder, the entire 448×448×3 image is first uniformly divided into 28×28 fixed-size 16×16 pixel image patches, each patch having a dimension of 16×16×3; then, feature transformation is performed, transforming each 16×16×3 image patch... The mapping is transformed into a 256-dimensional high-dimensional vector, which is then integrated to form a 28×28×256 embedding vector with semantic expressive capabilities. After the above processing, the original input features, denoted as feature x, are finally obtained and input to the image encoder. This feature can be either the output feature of the previous Transformer layer in the image encoder or the 28×28×256 basic feature vector generated after the patch embedding operation. To fix the stability of the basic feature extraction and reduce the computational complexity of subsequent training steps, the weights of the image encoder are kept frozen throughout the entire processing flow, and the features are only optimized and enhanced by subsequent added modules. Finally, the obtained original input feature x is completely input into the multi-scale dilated convolution adapter (MACA).

[0023] Step 1.2 involves processing the original input features through multiple parallel dilated convolution branches in the multi-scale dilated convolution adapter, generating multiple intermediate branch features corresponding to different scales. Specifically, this includes: firstly, reducing the channel dimension of the original input feature x input to MACA by using a linear layer with a shape of 256×192 to perform projection transformation, reducing the channel dimension of feature x from 256 to 192, thus obtaining low-dimensional features. This operation can effectively reduce computational cost while preserving the core feature information of the crack; then, three parallel 1×1 convolutional layers with identical structures are configured, and the number of convolutional kernels in each 1×1 convolutional layer is set to 64, which is beneficial for low-dimensional features. Channel adjustment and feature transformation are performed, followed by GeLU activation function. After introducing nonlinear representation, three consistent intermediate feature maps are generated respectively. , and The calculation formula is as follows: ;in This represents a 1×1 convolution operation. i =1, 2, 3, This represents the GeLU activation function, used to improve the model's ability to fit complex crack features.

[0024] The three intermediate feature maps are then fed into three parallel 3×3 dilated convolutional layers. To achieve comprehensive capture of cracks at different scales (from fine lines to wider cracks), the dilation rate of the three dilated convolutional layers is adjusted. , , The parameters are set to 1, 3, and 5 respectively. This configuration can gradually expand the receptive field of the model without significantly increasing the number of model parameters. Simultaneously, to optimize feature learning efficiency, a parameter locking mechanism is adopted, with the parameters of the three dilated convolution branches defined as follows: ;in All are trainable parameters, and at the beginning of training... and The parameter is set to 0; this design achieves partial parameter sharing across branches, reducing redundant computation, while retaining the flexibility to independently learn crack features at specific scales for each branch, ultimately generating three intermediate branch features corresponding to different scales. The calculation formula is as follows: Where i = 1, 2, 3, Indicates the expansion rate 3×3 dilated convolution operation.

[0025] Step 1.3 employs a channel attention mechanism to enhance the features of each intermediate branch corresponding to different scales, generating enhanced intermediate branch features. Specifically, for the three intermediate branch features A1, A2, and A3 at different scales, a separate channel attention (CA) module is configured for each branch to enhance the features and improve their discriminative power. The CA module adopts a squeeze-excitation core mechanism, and the specific implementation process is as follows: First, a global average pooling operation is performed on each branch feature to compress the feature information in the spatial dimension into a single-channel global feature vector, thereby comprehensively capturing the global dependencies of each channel feature. Then, this global feature vector is input into a transformation network consisting of two fully connected layers. The network consists of three layers. The first fully connected layer reduces the dimensionality of the global features and introduces non-linearity. The second fully connected layer restores the feature dimension to the original number of channels. Then, the feature values ​​of the restored features are mapped to the range of 0 to 1 to generate attention weights for each channel. The closer the value is to 1, the more important the channel is for crack detection; the closer it is to 0, the more redundant it is. Finally, the generated attention weights are multiplied with the original branch features channel by channel to strengthen the channels related to crack features while suppressing interference from irrelevant or redundant channels. Through this series of operations, the three intermediate branch features are assigned targeted channel weights to generate enhanced intermediate branch features. The crack feature representation ability and discriminative power of each feature are effectively improved.

[0026] Step 1.4: Fuse all enhanced intermediate branch features to obtain fused features. Specifically, this includes: fusing the three enhanced intermediate branch features obtained. A 1. A 2. A 3. Perform a concatenation operation based on channel dimensions, i.e., the Concat operation, to integrate the channel dimensions of the three features into a single multi-channel feature matrix. This ensures the complete preservation of feature information at different scales, such as fine cracks, medium cracks, and wide cracks, captured by each branch, without missing any key feature details. Next, input this concatenated multi-channel feature into a 1×1 convolutional layer with 192 kernels. Its core function is to fuse information and regularize the dimensions of the multi-channel features: on one hand, weighted fusion of features through convolution operations eliminates information redundancy between different branches, strengthening the correlation between crack features across scales; on the other hand, it uniformly adjusts the channel dimensions of the fused feature to 192, laying the foundation for the channel dimension restoration operation in subsequent steps. After processing by the 1×1 convolutional layer, a fully integrated and dimensionally regular fused feature is finally obtained. This feature has comprehensively integrated key information from cracks at different scales, possessing a more powerful crack feature expression capability.

[0027] Step 1.5 involves performing a residual connection between the fused features and the original input features to obtain enhanced image features. Specifically, to avoid the gradient vanishing problem during feature processing and to ensure that the basic information of the original input features is not lost, the fused features are first input into a linear layer of shape 192×256. This linear layer restores the channel dimension of the fused features from 192 dimensions to the same level as the original input features. x Using the same 256 dimensions, we ensure that both satisfy the dimension matching requirement of residual connections; then, we use element-wise addition to combine the fused features after restoring the dimensions with the original input features. x Perform a residual connection operation, that is, fuse each element of the feature with the original input feature. x The elements at corresponding positions are added one by one to obtain the enhanced image features. The calculation formula is as follows: Concat represents a channel cascading operation. The 1×1 convolution process represents the fusion of features; the enhanced image features retain the basic structural information of the original input features and fuse multi-scale crack features and key detail information after channel enhancement, effectively improving the comprehensive expressive power of the features. Then, they are fed into the next Transformer layer of the image encoder for further feature deepening and semantic extraction.

[0028] In a preferred embodiment of the present invention, step 2 above may include: Step 2.1 involves inputting the enhanced image features into the domain cue encoder, specifically including: the final generated enhanced image features. With dimensions of 28×28×256, the enhanced image features retain the basic structural information of the original input features while incorporating multi-scale crack texture features and key details enhanced by channel attention, thus possessing a more comprehensive crack feature representation capability; by using the enhanced image features... It is directly used as input data to the Domain Prompt Encoder (DPE) to initiate the generation process of domain-adaptive semantic prompts, providing a foundation for cross-domain scene adaptation and accurate segmentation.

[0029] Step 2.2, based on enhanced image features, the domain cue encoder dynamically generates domain cue vectors semantically aligned with the current image through its internally constructed feature memory and cross-attention mechanism. Specifically, the domain cue encoder pre-constructs a learnable feature memory (MemoryBank) with a shape of n×256, where n represents the number of feature vectors in the memory bank, specifically set to 256. All feature vectors in the memory bank are represented by a set. This indicates that the feature memory is continuously updated during model training to capture domain feature patterns under different training scenarios; firstly, the enhanced image features of the input domain cue encoder are defined as the original image embedding. I (Maintaining a dimension of 28×28×256), perform global pooling: first, extract the most representative key features from the image embedding using global max pooling; then, retain the overall global information of the image embedding using global average pooling; finally, sum the two pooling results element-wise to obtain a semantic vector S with a dimension of 1×256, calculated as follows: ,in This represents a global max pooling operation, used to filter the maximum value feature for each channel; This represents a global average pooling operation, used to calculate the average value feature for each channel; I Embed the original image to 28×28×256.

[0030] Next, configure a cross-attention Transformer module with 2 heads and 1 layer, using the semantic vector S as the query vector, and the feature memory... Simultaneously serving as both the key and value vectors, attention weights are obtained by calculating the similarity between the query and the key. These weights are then used to perform a weighted summation of the values, achieving precise alignment between the semantic vector and the features of the training data domain. The final output is a domain cue vector with a dimension of 1×256 that highly matches the semantics of the current input image. D .

[0031] Step 2.3 involves the first processing of the domain cue vector, which is to input the domain cue vector into the domain cue decoder to reconstruct the corresponding reconstructed image embedding. Specifically, the domain cue decoder is designed specifically for self-supervised image embedding and reconstruction tasks, and is constructed using three fully connected layers in series. Each layer has a clearly defined configuration and matched dimensions: the feature dimensions of both the input and output layers are set to 256, which is consistent with the domain cue vector. D The 1×256 dimension is maintained, while the dimension of the intermediate hidden layer is set to 128 to mine deep feature correlations and enhance non-linear expressive power. Each fully connected layer is followed by a GeLU activation function to introduce non-linear feature transformation. The 1×256 domain cue vector generated in step 2.2 is then used. D After being input into the decoder, the first fully connected layer performs a dimensionality mapping from 256 to 128 dimensions, followed by GeLU activation to filter effective features and suppress redundant information. Next, an intermediate hidden layer further optimizes and integrates the 128-dimensional features, uncovering the potential correlation between domain features and image embedding. Then, a third fully connected layer restores the feature dimension from 128 to 256 dimensions, preserving core semantic information. Finally, a Reshape operation transforms the 1×256 one-dimensional feature vector into a 28×28×256 three-dimensional structure, ultimately yielding the image embedding. I Reconstructed image embedding with completely consistent size and dimensions I' The reconstruction process is essentially a self-supervised learning task, with the core objective of embedding the reconstructed image into... I' Embedding as close as possible to the original image I This process involves reverse-constraining the feature memory to continuously optimize it, enabling it to learn more discriminative and generalizable domain features.

[0032] Step 2.4 involves a second processing step on the domain cue vector, which expands the domain cue vector to generate an expanded domain cue vector with the same embedding size as the original image. Specifically, this includes: the domain cue vector... D The dimension is 1×256, while the original image embedding I The dimensions of the two are 28×28×256, and there is a difference in spatial dimensions, making it impossible to directly perform pixel-by-pixel feature interaction and similarity calculation. To solve this dimensional matching problem, the domain hint vector needs to be modified. D Perform feature dimension expansion operation: First, clarify that the target dimension to be expanded must be consistent with the original image embedding. I Completely identical, i.e., height 28, width 28, channel 256; followed by a field-hinted vector. DBased on the 256-dimensional channel features, directional repeat padding is performed in the spatial dimension: repeating 28 times row-wise in the height direction to expand the one-dimensional feature to 28 in the height dimension; repeating 28 times column-wise in the width direction to expand the feature to 28 in the width dimension as well; the channel dimension remains unchanged at 256 dimensions without any additional adjustments; after expansion, dimension verification is performed to ensure the generated extended domain hint vector is accurate. D The dimensions are 28×28×256, and the original image is embedded... I The size and dimensions are a perfect match.

[0033] Step 2.5: Calculate the similarity between the extended domain cue vector and the original image embedding to obtain a similarity feature map. Specifically, this includes using the generated 28×28×256 extended domain cue vector... D' and original image embedding I For input, calculate the similarity between the two according to the following process: First, for... D' and I The 256-dimensional feature vectors corresponding to each spatial location (28×28 in total) are respectively processed... L 2. Normalization is performed to eliminate the interference of vector magnitude differences on similarity calculation. Then, the dot product of the two normalized vectors is calculated. The dot product result is the cosine similarity at that position. Since the magnitude of the normalized vectors is all 1, the dot product is equivalent to the cosine value. The calculation formula is as follows: ,in Cosine Similarity This indicates an operation that first normalizes the data and then calculates the cosine similarity of the dot product. D' This is the normalized 28×28×256 extended field cue vector. I The normalized 28×28×256 original image is embedded; after all spatial locations are calculated, a similarity feature map M with a dimension of 28×28×1 is formed. Each pixel value of this feature map is in the range of [-1, 1]: the closer the value is to 1, the higher the semantic fit between the domain cue at the corresponding location and the image feature; the closer the value is to -1, the lower the fit. This feature map will be used as the quantitative basis for domain fit in subsequent fusion.

[0034] Step 2.6: Fuse the extended domain cue vector, similarity feature map, and original image embedding to generate a semantic cue vector. This specifically includes: first, determining the fixed order of feature concatenation: according to the extended domain cue vector... D' Similarity feature map M, original image embedding I The splicing operation is performed along the channel dimension in the following order; the spatial dimension (28×28) remains unchanged during splicing, and only the channel dimension is superimposed: D' Contribute 256 dimensions M Contribute 1D, IThe input consists of 256 dimensions, which are concatenated to obtain a high-dimensional feature matrix of 28×28×513. This order ensures that domain adaptive information, matching degree information, and original feature information are arranged sequentially, facilitating accurate capture of core correlations during subsequent convolutional fusion. This high-dimensional feature matrix is ​​then input into a pre-defined 1×1 convolutional layer with a kernel size of 1×1 and a number of 256, consistent with the number of channels embedded in the original image, ensuring that the output feature dimension adapts to the downstream module. During convolution, the weight coefficients of different channels are learned to weight and integrate multi-source features: strengthening the information of regions with high matching degree between domain cues and image features, and suppressing redundant and poorly adaptable feature interference. After convolution, the channel dimension is compressed from 513 to 256, ultimately generating a semantic cue vector Prompt of 28×28×256. This vector retains the crack features of the original image while incorporating domain adaptive cues and feature matching information, possessing strong cross-domain adaptability and high discriminative power, and will be directly fed into the downstream mask decoder for accurate segmentation of road cracks.

[0035] In this embodiment of the invention, enhanced image features, which integrate multi-scale crack features and channel enhancement information, are input into the domain cue encoder, providing a high-quality foundation for domain-adaptive cue generation. Semantically aligned domain cue vectors are dynamically generated through a feature memory and a cross-attention mechanism, effectively improving the model's adaptability to domain features in different scenarios and overcoming the shortcomings of existing methods in cross-domain generalization. During the process of reconstructing the image embedding from the domain cue vector using the decoder, the feature memory is optimized through self-supervised learning, enabling it to learn more discriminative domain features. The domain cue vector is expanded to match the original image embedding size, ensuring the feasibility of subsequent similarity calculations. The similarity feature map accurately quantifies the degree of matching between the domain cue and image features, providing a targeted basis for feature fusion. Finally, the semantic cue vector generated by fusing the expanded domain cue vector, the similarity feature map, and the original image embedding integrates domain-adaptive information, feature matching information, and the original crack features. This strengthens the model's adaptability to complex scenarios and enhances the discriminative power of crack features, providing high-quality feature input for the downstream mask decoder to achieve accurate road surface crack segmentation.

[0036] In a preferred embodiment of the present invention, step 3 above may include: Step 3.1 involves obtaining the semantic prompt vector and the original image embedding. Specifically, this includes: accurately extracting the semantic prompt vector Prompt from the final output of Step 2.6. This vector, after being fused by a 1×1 convolution, has a dimension of 28×28×256, integrating domain adaptation information, feature matching information, and the original crack features; and simultaneously extracting the original image embedding from the feature processing chain. IIt is the 28×28×256 enhanced image feature mapping result defined in step 2.2. The two dimensions are completely consistent and both meet the input specification requirements for subsequent loss calculation, together providing core feature input for the calculation of supervised segmentation loss.

[0037] Step 3.2: Based on the semantic cue vector and the original image embedding, calculate the supervised segmentation loss, which is a weighted combination of cross-entropy loss and Dice coefficient loss. Specifically, this includes: first, inputting the semantic cue vector Prompt into the downstream mask decoder. The mask decoder extracts high-level semantic features step-by-step through multiple convolutional layers and non-linear activation functions. After processing by the final Sigmoid activation function, a crack prediction probability map with dimensions 28×28×1 is generated. Each pixel value in the map is mapped to the [0, 1] interval, directly representing the probability that the location belongs to the crack region; then, the cross-entropy loss is calculated. With Dice coefficient loss Supervised segmentation loss of weighted portfolio Weighting coefficients Set to 0.8, the combined formula is: ,in To monitor the division of losses, The weights used to balance single-pixel classification accuracy and region overlap optimization); the formula for calculating cross-entropy loss is: in These are the ground truth labels in the training dataset corresponding to the input image, with dimensions of 28×28×1. Cracked areas are labeled as 1, and non-cracked areas are labeled as 0. This is the predicted probability of the corresponding pixel in the predicted probability map. This loss focuses on optimizing the classification accuracy of a single pixel; the formula for calculating the Dice coefficient loss is: ; in The minimum constant is set, ranging from 1e to 6, to avoid calculation anomalies caused by a denominator of 0. This loss focuses on the overall overlap of the crack region, effectively compensating for the sensitivity of cross-entropy loss to the scenario of unbalanced crack samples.

[0038] Step 3.3: Based on the reconstructed image embedding and the original image embedding, calculate the self-supervised reconstruction loss using the mean squared error function. Specifically, this includes: using the generated reconstructed image embedding... I' (Dimensions 28×28×256) and extracted original image embedding I The input is (dimensions 28×28×256), using mean squared error. MSE Function to calculate self-supervised reconstruction loss This is used to quantitatively measure the semantic similarity between two things; its calculation formula is: in To compensate for losses in self-monitoring and reconstruction, N The total number of pixels embedded in the image is calculated as height × width × number of channels = 28 × 28 × 256 = 200704. I and I' The images are embedded separately for the original and the reconstructed images, with corresponding pixels at each location. The specific calculation process is as follows: first, calculate pixel by pixel. I and I' The difference is calculated by squaring each difference to amplify significant differences, then summing all the squared differences, and finally dividing by the total number of pixels. N The average loss value is obtained; through the constraints of this loss function, the feature memory of the domain cue encoder can be continuously optimized, enabling it to learn more discriminative and generalizable domain features, thereby improving the robustness of the model to changes in different scenarios.

[0039] Step 3.4 involves weighted summation of the calculated supervised segmentation loss and the calculated self-supervised reconstruction loss to generate a joint loss function for model training. Specifically, this includes: weighting the calculated supervised segmentation loss... With the obtained self-supervised reconstruction loss The loss function is generated by weighted summation according to preset weights and used for model training; the formula for calculating the joint loss function is as follows: Where L is the joint loss function, and To control the weighting coefficients for the importance of the two loss terms separately, experimental verification and task priority settings were used. , The aforementioned setup ensures the accuracy optimization of the core task (crack segmentation) while enhancing the model's cross-domain generalization ability through an auxiliary self-supervised reconstruction task, thus avoiding overfitting of the main task. During model training, the joint loss function serves as the core basis for parameter optimization. The gradient of the loss with respect to all trainable parameters (multi-scale dilated convolutional adapter, domain cue encoder, etc.) is calculated through backpropagation algorithm. Then, the parameters are iteratively updated based on gradient descent strategy to continuously reduce the joint loss value, ultimately guiding the model to learn a more accurate and scene-adaptive crack feature representation.

[0040] In this embodiment of the invention, by accurately acquiring semantic cue vectors and original image embeddings, a reliable data foundation is laid for loss calculation; the supervised segmentation loss adopts cross-entropy loss and... Dice The design of the coefficient loss weighted combination can optimize the accuracy of single-pixel classification and also... DiceThe coefficient loss mitigates the training bias caused by the imbalance of crack samples and improves the overall overlap of crack region segmentation. The self-supervised reconstruction loss quantifies the difference between the original image embedding and the reconstructed image embedding by using the mean square error function, and inversely drives the domain prompts to learn more discriminative and generalizable domain features from the encoder's feature memory, thereby enhancing the model's robustness to changes in different scene domains. The joint loss function constructed by weighted summation reasonably balances the priority of the main crack segmentation task and the auxiliary task of domain feature learning, ensuring both the accuracy optimization of the core segmentation task and the improvement of the model's cross-domain adaptability, providing reasonable and effective guidance for iterative optimization of model parameters.

[0041] In a preferred embodiment of the present invention, step 4 above may include: Step 4.1: Prepare a dataset containing road surface crack images and their corresponding labels. Divide the dataset into training and validation sets. Specifically, this includes: First, selecting publicly available road surface crack datasets covering diverse scenarios (such as Khanhha and Crack500 datasets) as basic training data. The dataset must include road surface crack images under sunny, rainy, and varying light intensities and road surface materials (asphalt, cement), with each image accompanied by a pixel-level ground truth label to ensure the data can support the training of the model's cross-domain generalization ability. Then, conduct systematic data preprocessing: First, uniformly crop and adjust all original images to a standard size of 448×448×3 to avoid model training instability due to image size differences; Second, perform normalization processing on the image pixel values ​​using the formula... ,in These are the original pixel values, ranging from [0, 255]. The normalized pixel values ​​are set to a range of [0, 1] to eliminate the interference of pixel scale differences on training. The third step is to binarize the ground truth labels, setting the threshold to 0. Crack areas with gray values ​​greater than 0 in the label image are marked as 1, and non-crack areas with gray values ​​equal to 0 are marked as 0, ensuring that the label dimension strictly matches the image size, i.e., 448×448×1. Optional data augmentation operations are performed, generating expanded samples by randomly flipping, rotating, and slightly scaling by 0.8 to 1.2 times to alleviate the problem of insufficient data. Finally, a random partitioning method with a fixed random seed of 42 is used to divide the preprocessed dataset into training and validation sets in an 8:2 ratio. During the partitioning process, stratified sampling is used to ensure that the proportion of samples from different scenarios in the two classes of data is consistent, avoiding the impact of data distribution deviation on the accuracy of the validation set's evaluation of the model's generalization performance. The training set is used for iterative learning of model parameters, and the validation set is used for real-time monitoring of overfitting risks during the training process.

[0042] Step 4.2: Based on the dataset, load and initialize the trainable model containing a multi-scale dilated convolutional adapter, a domain cue encoder, a domain cue decoder, and a mask decoder. Simultaneously, freeze the backbone weights of the image encoder to obtain the initialized trainable model. Specifically, this includes: based on the determined dataset specifications and model task requirements, loading a pre-defined complete model architecture. This architecture consists of a ViT-B image encoder, a multi-scale dilated convolutional adapter (MACA), a domain cue encoder (DPE), a domain cue decoder, and a mask decoder connected in series. The core parameters of each module are pre-configured; for example, the MACA contains three dilated convolutional branches with dilation rates of 1, 3, and 5, and the DPE has a built-in 256×256-dimensional feature memory. Then, initialize the parameters module by module: for newly added trainable modules such as the MACA, DPE, domain cue decoder, and mask decoder, use the He normal distribution initialization method, i.e., the initialization parameters follow a mean of 0 and a variance of 0. The normal distribution, where The module inputs feature dimensions to ensure stable gradient propagation in the early stages of training. For the ViT-B backbone of the image encoder, pre-trained weights on the ImageNet dataset are loaded. After loading, the weights are completely frozen by setting the parameter gradient to False. No parameters of this part are updated during training, thus fixing the basic feature extraction capability and significantly reducing the training computation and overfitting risk. After parameter initialization, inter-module dimension adaptation verification is performed: the feature dimensions of the image encoder output and MACA input, MACA output and DPE input, and DPE output and domain cue decoder / mask decoder input are checked sequentially to ensure that there are no dimension conflicts at the connection interfaces of each module. Finally, the initialized model to be trained is output.

[0043] Step 4.3: Input the images from the training set into the initialized model to be trained, and based on the joint loss function, use the backpropagation algorithm to iteratively optimize the trainable parameters in the initialized model to obtain the updated model. Specifically, this includes: First, configuring the core training parameters: setting the batch size to 8 to balance training efficiency and GPU memory usage; setting the maximum training epochs to 120, maintaining the initial learning rate at 0.0004, selecting the AdamW optimizer, and configuring the optimizer parameters β1=0.9, β2=0.999, and weight decay coefficient=0.01 to suppress overfitting through weight decay; to adapt to the initial learning rate of 0.0004, strengthening gradient stability control by selecting a gradient clipping strategy with a clipping threshold set to 0.5. The learning rate is lower than the usual 1.0, precisely preventing gradient explosion caused by excessively high learning rates in the early stages of training. Then, an iterative training process is initiated: before each training round, the training set is randomly shuffled to avoid data sequence dependencies in model learning; the training set is divided into batches, and in each iteration, a batch of training images and corresponding ground truth labels are extracted and input into the initialized model to be trained; during the model's forward propagation, after the encoder extracts basic features from the images, enhanced image features are generated through MACA, and then domain cue vectors are generated through DPE. On one hand, the domain cue vectors are input into the domain cue decoder to reconstruct the image embedding; on the other hand, semantic cue vectors are generated through expansion and fusion and input into the mask decoder to obtain a 28×28×1 crack prediction probability map; combining the ground truth labels and the original image embedding, the joint loss function value is calculated according to the logic of step 3. L .

[0044] Then, backpropagation is performed: based on the chain rule, from the joint loss value... L Starting from this point, the gradient of the loss with respect to all trainable parameters (MACA, DPE, domain cue decoder, and mask decoder parameters) is calculated layer by layer. After calculation, gradient pruning is used to limit the maximum gradient value to no more than 0.5, effectively alleviating the gradient oscillation problem caused by the high learning rate. The AdamW optimizer updates the trainable parameters based on the pruned gradient, with the update formula as follows: ; in For the current parameter, For the updated parameters, The initial learning rate is 0.0004. , For first-order and second-order momentum estimation, It is the minimum value. This represents the weight decay coefficient; it is combined with a linear learning rate decay strategy, starting from round 60, the learning rate is linearly decayed from 0.0004 to... In the early stage of training, a learning rate of 0.0004 is used to quickly reduce the loss. In the later stage, the parameters are finely adjusted by a decayed low learning rate to adapt to the training rhythm of a high initial learning rate. After each batch of parameter updates is completed, the gradient cache is cleared to avoid gradient accumulation interfering with the next batch of training. After traversing all batches of data in the training set, a training cycle is completed, and the average training loss value of the current cycle is recorded.

[0045] Step 4.4: Evaluate the performance of the updated model using a validation set. When the evaluation metrics stabilize or reach the preset training epochs, stop training and save the final model parameters to obtain the trained crack detection model. Specifically, this includes: immediately initiating the model performance validation process after each training epoch; during validation, the model only performs forward inference and does not update parameters; inputting the validation set into the updated model in batches (batch size=8) to obtain a crack prediction probability map for each validation image; converting the probability map into a binary prediction map using a threshold of 0.5; marking a probability ≥ 0.5 as 1, otherwise marking it as 0; selecting the intersection-union ratio (IUU). IoU And F1-Score as the core evaluation metric, IoU The calculation formula is: ; in TP This represents the number of true positive pixels, i.e., the number of pixels predicted as cracks but actually being cracks. FP This represents the number of false positive pixels, i.e., the number of pixels predicted as cracks but actually not cracks. FN This represents the number of false negative pixels, i.e., the number of pixels predicted as non-cracks but actually representing cracks; it measures the degree of overlap between the predicted and ground truth regions; the F1-Score calculation formula is: ; in Precision For accuracy = , Recall Recall rate = This is used to balance the accuracy of the model with the false negative rate.

[0046] Then calculate the validation set for all images. IoU The average F1-Score is used as the performance metric for the current training round; a sliding window strategy is employed to monitor performance stability. If the performance stability is low for five consecutive training rounds... IoUThe fluctuation range of both the average F1-Score and the average F1-Score is less than 0.001, and there is no obvious downward trend, indicating that the model performance is stabilizing. If it has not reached a stable state, continue to repeat the training process in step 4.3 until the preset maximum number of training epochs is reached, i.e., 120 epochs. When either the performance is stable or the maximum number of epochs is reached, stop training. Save the final model parameters in the preset format .pth, including the optimized parameters of MACA, DPE, domain hint decoder, and mask decoder, and save the training configuration file, including the learning rate, batch size, loss function weights, etc. Finally, load the saved model parameters, input a small number of validation set images for inference verification, confirm that the model can output prediction results normally and that the performance indicators are consistent with the end of training, and finally obtain a crack detection model that has been trained and has reliable performance.

[0047] In this embodiment of the invention, by preparing diverse pavement crack datasets and reasonably dividing them into training and validation sets, sufficient and hierarchical data support is provided for model training, ensuring the effectiveness of training and the accuracy of generalization evaluation. Loading and initializing the model and freezing the backbone weights of the image encoder not only fixes the basic feature extraction capability but also reduces the computational complexity of training and the risk of overfitting. Iteratively optimizing trainable parameters based on the joint loss function and accurately guiding the model to learn crack features through backpropagation continuously improves detection accuracy. Real-time evaluation of model performance using the validation set and stopping training and saving parameters according to preset conditions can effectively avoid overfitting or underfitting, ultimately obtaining a crack detection model with stable performance and reliable detection results.

[0048] In a preferred embodiment of the present invention, step 5 above may include: Step 5.1: Adjust the road surface image to be detected to the input size specified by the trained crack detection model to obtain a size-fitted image. This includes: first, acquiring the original road surface image to be detected; then, extracting the basic information of the image, explicitly recording its original size as H×W×3, where H is the original height of the image, W is the original width, and 3 represents the RGB three channels, and storing the size information in a temporary variable to provide a precise basis for the size restoration of the initial crack mask image; subsequently, strictly following the preset input specifications of the trained crack detection model, performing standardized size adjustment on the original image to be detected: using a bilinear interpolation algorithm for image scaling, specifically: first calculating the scaling ratio between the original image and the target size (448×448), with a horizontal scaling ratio of 448 / W and a vertical scaling ratio of 448 / H; then traversing each pixel position of the target image ( x , y The coordinates of that location on the original image are obtained by reverse mapping through scaling. x' , y' Its coordinate values ​​may be floating-point numbers; then determine the coordinates (x' , y' The four nearest integer coordinate pixels around () x 1, y 1), ( x 1, y 2), ( x 2, y 1), ( x 2, y 2).

[0049] Then calculate the weights: first split into horizontal and vertical dimensions, and calculate them separately. x' , y' )exist x Direction relative to x 1. x The distance weight of 2 is denoted as . wx 1. wx 2, and in y Direction relative to y 1. y The distance weight of 2 is denoted as . wy 1. wy 2. The closer the distance, the greater the weight value, and the sum of the weights in the same dimension is 1; then multiply the horizontal and vertical weights to obtain the final weights of each of the four pixels, such as ( x 1, y The weight of 1) is wx 1× wy 1. The sum of the four final weights is still 1; finally, the pixel values ​​of the four pixels are multiplied by their corresponding weights, and the sum is obtained to get the target pixel. x , y The process of scaling up the image preserves the edge texture and fine structure of road surface cracks to the greatest extent possible, avoiding feature loss during scaling. The algorithm then uniformly scales the original image to a standard input size of 448×448×3. After scaling, pixel normalization is performed according to the data preprocessing standards used in the model training phase, specifically using the formula... Finish, The original pixel values ​​of the image to be detected range from [0, 255]. The normalized pixel values ​​are mapped to the range of [0, 1] to eliminate pixel scale differences between different images. Finally, the processed image is format-checked to ensure that it is an RGB three-channel format, the pixel value range meets the requirement of [0, 1], and the size is accurately 448×448, thus obtaining a size-adapted, format-compliant, and numerically standardized image to be detected.

[0050] Step 5.2: Input the size-adapted image to be detected into the trained crack detection model for forward inference to generate an initial crack mask map corresponding to the input size. Specifically, this includes: inputting the obtained 448×448×3 normalized image to be detected into the trained crack detection model to start the forward inference process. During the inference phase, the model only performs feature forward propagation, does not enable the backpropagation mechanism, and does not update any parameters. The specific process is as follows: First, the image enters the model's image encoder, which adopts the ViT-B architecture, with the backbone weights kept frozen. First, the image is embedded through the built-in patching operation. A 448×448×3 image is segmented into 28×28 16×16×3 image patches, then linearly projected to transform into 28×28×256 embedding vectors. Subsequently, a self-attention mechanism in the Transformer layer extracts globally correlated basic features. These basic features are then input into a multi-scale dilated convolution adapter (MACA), sequentially passing through three parallel dilated convolution branches with dilation rates of 1, 3, and 5 to capture crack features at different scales. The features output from each branch are enhanced for discriminative power by a channel attention module, and then fused with a 1×1 convolution through channel concatenation to form a 28×28 image. The enhanced image features are then fed into the Domain Cueing Encoder (DPE). Combined with an internally optimized 256×256 dimensional feature memory, the semantic vector obtained through global pooling of the enhanced image features is used as the Query, and the features from the memory are used as the Key and Value. Semantic alignment is achieved through cross-attention calculation, generating a 1×256 domain cueing vector. This domain cueing vector is then spatially expanded to a 28×28×256 extended domain cueing vector, concatenated channel-wise with the original image embedding and similarity feature map, and finally fused using a 1×1 convolution to obtain a 28×28 domain cueing vector. The semantic cue vector is 256×28×1. Finally, the semantic cue vector is input into the mask decoder, and high-level semantic features are extracted through two layers of 3×3 convolution. Then, the channel dimension is compressed to 1 through 1×1 convolution. Finally, the output value is mapped to the [0, 1] interval through the Sigmoid activation function to obtain a 28×28×1 crack prediction probability map. The binarization threshold is set to 0.5. Pixels with a probability ≥ 0.5 are marked as 1 (representing crack areas), and pixels with a probability < 0.5 are marked as 0 (representing non-crack areas). Finally, an initial crack mask map matching the model input size of 448×448 is generated.

[0051] Step 5.3: Restore the initial crack mask image to the original size of the road surface image to be detected using bilinear interpolation to obtain the final crack mask image; Step 5.4: Based on the final crack mask image, locate and visualize the crack area on the road surface image to be detected. Specifically, this includes: based on the final crack mask image restored to its original size H×W×1, crack location and visualization are performed in two steps: The first step is to accurately locate the crack area. First, a connected component analysis is performed on the final crack mask image. Continuous regions with a pixel value of 1 are selected using the 8-neighborhood connectivity criterion to exclude isolated noise pixels and avoid mislocation. A unique identifier (e.g., ID=1, ID=2) is assigned to each independent connected region to distinguish different cracks. Then, for each identified connected region, all pixels within it are traversed, and the coordinates of all pixels within that region are collected and recorded. i , j ),in i ∈[1, H], corresponding to the image height direction, j ∈[1, W], corresponding to the image width direction, forming the complete coordinate set of the connected region; then, the minimum bounding rectangle is calculated based on the coordinate set: extract the coordinates of all pixels from the set. i Values ​​(vertical axis), filter out i The minimum value is denoted as `top`, corresponding to the top vertical position of the rectangle, and the maximum value is denoted as `bottom`, corresponding to the bottom vertical position of the rectangle; then, extract the values ​​of all pixels. j Values ​​(horizontal coordinates) are used to filter out... j The minimum value is denoted as left, corresponding to the left horizontal position of the rectangle, and the maximum value is denoted as right, corresponding to the right horizontal position of the rectangle. Based on these four extreme values, the coordinates of the upper left corner of the minimum bounding rectangle are determined as (left, top) and the coordinates of the lower right corner are determined as (right, bottom). This rectangle can tightly select the corresponding crack area and completely cover the extension range of the crack, thereby achieving accurate positioning and clear distinction of one or more cracks.

[0052] The second step is visualization, which uses a semi-transparent overlay method to fuse the crack location results with the original road surface image to be detected: First, each located crack connected region is assigned a red label color with an RGB value of (255, 0, 0), and the label transparency is set to 0.6, which ensures that the crack area is clearly visible without obscuring the background details of the original road surface; at the same time, a serial number corresponding to the region identifier is added to the upper left corner of the smallest bounding rectangle of each crack connected region, such as crack 1, crack 2; the serial number uses a label style with white font and black background to improve readability; in addition, a 1-pixel wide red outline can be drawn on the edge of the crack area to further highlight the shape and outline of the crack; finally, a visualization image that integrates the crack labels is output. This image completely preserves the scene information of the original road surface and clearly presents the location, range, shape and serial number of each crack.

[0053] In this embodiment of the invention, the road surface image to be detected is first adjusted to the input size specified by the model to ensure that the input format is consistent with the training standard, laying the foundation for accurate forward inference of the model. An initial crack mask image is generated through forward inference of the trained model, which can effectively utilize the crack features learned by the model to achieve preliminary accurate identification. Bilinear interpolation is used to restore the initial mask image to its original size, which can preserve the crack edge contour and spatial distribution features to a certain extent, avoid morphological distortion caused by size transformation, and ensure that the mask image and the original image are accurately aligned pixel by pixel. Finally, crack location and visualization are realized based on the restored mask image, which not only makes the location and shape of the crack clearly distinguishable, but also effectively preserves the original road surface scene information, making it easy for users to intuitively obtain the detection results, effectively improving the practicality, accuracy and intuitiveness of road crack detection, and ensuring the efficiency and reliability of the detection process.

[0054] In a preferred embodiment of the present invention, step 5.3 above may include: Step 5.31: Obtain the initial crack mask image corresponding to the input size, and obtain the original size of the road surface image to be detected. Specifically, this includes: accurately extracting the initial crack mask image from the forward inference output. This mask image is a binary image with a fixed size of 28×28×1. The area with a pixel value of 1 corresponds to the predicted crack area, and the area with a pixel value of 0 corresponds to the non-crack area. At the same time, it is determined that its channel number is 1, a single-channel binary image. Then, the original size information of the road surface image to be detected is extracted from the pre-stored temporary variables, and the original size is confirmed to be H×W×3. H represents the height of the original image in pixels, W represents the width of the original image in pixels, and 3 represents the RGB three channels. The size of the initial crack mask image 28×28×1 and the size of the original image H×W×3 are synchronously stored in the cache space, and the dimensional meanings of the sizes are labeled, including height, width, and number of channels.

[0055] Step 5.32: Based on the original size of the road surface image to be detected and the current size of the initial crack mask corresponding to the input size, calculate the spatial scale transformation parameters. Specifically, this includes: First, clarifying the core objective of size restoration: accurately restoring the initial 28×28×1 crack mask to an H×W×1 size consistent with the original H×W×3 spatial dimension of the road surface image to be detected, ensuring that the restored mask can be aligned pixel-by-pixel with the original image, thus guaranteeing the accuracy of subsequent localization; Based on the two sets of size information obtained, calculate the spatial scale transformation parameters, with the core parameter being the horizontal scaling ratio. and vertical scaling ratio ; where the horizontal scaling ratio The calculation formula is , representing the pixel span in the width direction of each pixel in the initial crack mask map that needs to be mapped to the original image's width direction; vertical scaling ratio. The calculation formula is , representing the pixel span in the height direction of each pixel in the initial crack mask image that needs to be mapped to the height direction of the original image; after the calculation is completed, the validity of the two scaling ratios is checked to ensure that the ratio values ​​are positive numbers. Since H and W are both positive integers and 28 is a fixed positive integer, the ratio is always positive to avoid invalid scaling ratios that would cause restoration failure. Finally, the valid horizontal and vertical scaling ratios are determined and stored as the core basis for subsequent size restoration.

[0056] Step 5.33: Based on the spatial scale transformation parameters, a bilinear interpolation algorithm is used to perform a size restoration operation on the initial crack mask image to obtain the restored crack mask image. Specifically, this includes: determining a lateral scaling ratio... Vertical scaling ratio Based on this, a bilinear interpolation algorithm is used to perform a size restoration operation on the initial 28×28×1 crack mask image. The specific process is as follows: First, the size of the restored target mask image is determined to be H×W×1. Then, each pixel position of the target mask image is traversed ( x , y ),in x The value range is [1, H], corresponding to the image height direction; y The value range is [1, W], corresponding to the image width direction; for each target pixel ( x , y By performing a reverse mapping calculation through scaling, the corresponding mapped coordinates on the initial crack mask map are obtained. x' , y' The specific calculation formula is as follows: Since H and W are not necessarily multiples of 28, the resulting ( x' , y' ) is usually a floating-point number; then determine the mapping coordinates ( x' , y' The four nearest integer pixel coordinates of the surrounding area are taken as the reference. x' The integer part is x 0, rounded up to the nearest integer. x 0+1, resulting in x Two integer coordinates of direction x 1= x 0、 x 2= x 0+1, similarly take y' The integer part is y 0, rounded up to the nearest integer. y 0+1, resulting in y Two integer coordinates of direction y 1= y 0、 y 2=y 0+1, ultimately determining the 4 neighboring pixels as ( x 1, y 1), ( x 1, y 2), ( x 2, y 1), ( x 2, y 2).

[0057] Then, the weights are calculated by splitting the horizontal and vertical dimensions. x Directional weights are calculated as follows wx 1= x 2- x' , wx 2= x' - x 1, y Directional weights are calculated as follows wy 1= y 2- y' , wy 2= y' - y 1. The sum of the weights of the same dimension is 1, such as... wx 1+ wx 2=1, and the closer the pixel is to the mapped coordinate, the greater its weight; then multiply the weights of the horizontal and vertical dimensions to obtain the final weights of the four neighboring pixels, such as ( x 1, y The weight of 1) is wx 1× wy 1, ( x 1, y 2) The weight is wx 1× wy 2. And so on; the sum of the four final weights is still 1; finally, extract the pixel values ​​(0 or 1) of the four neighboring pixels, multiply them by their respective final weights, and sum them to obtain the target pixel. x , y The initial pixel value is obtained, and then a threshold of 0.5 is set to binarize the initial pixel value. If it is ≥0.5, it is 1, and if it is <0.5, it is 0, to ensure that the restored mask image still retains the binary characteristics. The above process is repeated until all pixels of the target mask image are traversed, and finally a restored crack mask image with a size of H×W×1 is obtained, and the crack region contour and spatial distribution of the mask image are not distorted.

[0058] Step 5.34 outputs the restored crack mask as the final crack mask with the same dimensions as the original. This includes: performing a comprehensive quality check on the restored crack mask, which includes three aspects: First, size check, confirming its dimensions are accurately H×W×1 using an image dimension detection tool, perfectly matching the original height and width of the road surface image to be detected; second, pixel value check, traversing all pixels in the mask to confirm that all pixel values ​​are 0 or 1, with no intermediate transition values, maintaining the core characteristics of a binary mask; third, spatial alignment check, randomly selecting 3 to 5 pixels to verify the spatial correlation between their positions in the restored mask and their corresponding positions in the original image, ensuring no offset. If the check fails, return to step 5.32 to recalculate the scaling ratio and perform the restoration operation; if the check passes, the restored crack mask is directly output as the final crack mask with the same dimensions as the original road surface image to be detected. Simultaneously, the final mask is associated and stored in the file directory of the corresponding original image, providing direct and reliable input data for the accurate location and visualization of crack areas.

[0059] In this embodiment of the invention, by accurately acquiring the original dimensions of the initial crack mask image and the road surface image to be detected, a clear and reliable data foundation is laid for the dimension restoration operation. Based on two sets of dimensions, targeted spatial scale transformation parameters are calculated to ensure the proportional accuracy of the dimension restoration and avoid crack position shifts caused by improper scaling. The dimension restoration is performed using a bilinear interpolation algorithm, which can preserve the edge contour and spatial distribution characteristics of the crack to the greatest extent and prevent crack morphology distortion during the restoration process. Finally, by verifying and outputting the final crack mask image consistent with the original dimensions, it is ensured that the mask image and the original image are spatially aligned pixel by pixel and the core characteristic of binarization is maintained. This provides high-quality basic data for the accurate positioning and intuitive visualization of the crack area, effectively improving the accuracy and practicality of the entire detection process.

[0060] like Figure 2 As shown, embodiments of the present invention also provide a pavement crack detection system based on multi-scale void convolution and domain cueing generation, comprising: The acquisition and insertion module is used to acquire the original input features extracted by the image encoder; insert a multi-scale dilated convolution adapter before each Transformer layer of the image encoder; and process the original input features through the multi-scale dilated convolution adapter to generate enhanced image features. The fusion generation module is used to input the enhanced image features into the domain cue encoder. Based on the enhanced image features, the domain cue encoder dynamically generates domain cue vectors that are semantically aligned with the current image through an internally built feature memory and cross-attention mechanism. The domain cue vectors are then subjected to sequence processing, i.e., the corresponding reconstructed image embedding is reconstructed according to the domain cue decoder. The domain cue vectors are expanded to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding. Finally, the semantic cue vectors are fused to generate the semantic cue vector. The function building module is used to construct a joint loss function for model training based on semantic cue vectors and original image embeddings. The joint loss function consists of two parts: supervised segmentation loss based on semantic cue vectors and self-supervised reconstruction loss based on reconstructed image embeddings and original image embeddings. The model training module is used to train the model based on the constructed joint loss function using a dataset containing road crack images and labels, to obtain a trained crack detection model. The final inference and restoration module is used to resize the road surface image to be detected during actual detection and input it into the trained crack detection model for inference to generate a mask image of the crack area. By restoring the mask image of the crack area to the original image size, the localization and visualization of road cracks can be achieved.

[0061] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for pavement crack detection based on multi-scale dilated convolution and domain prompt generation, characterized in that, The method comprises: obtaining original input features extracted by an image encoder; inserting a multi-scale hollow convolution adapter before each Transformer layer of the image encoder; processing the original input features through the multi-scale hollow convolution adapter to generate enhanced image features; inputting the enhanced image features into a domain prompt encoder, and based on the enhanced image features, the domain prompt encoder dynamically generates a domain prompt vector aligned with the semantics of the current image through an internally constructed feature memory bank and a cross-attention mechanism; performing sequence processing on the domain prompt vector, i.e., reconstructing a corresponding reconstructed image embedding according to a domain prompt decoder, expanding the domain prompt vector to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding, and finally fusing to generate a semantic prompt vector; based on the semantic prompt vector and the original image embedding, constructing a joint loss function for model training; wherein the joint loss function is composed of a supervised segmentation loss based on the semantic prompt vector and a self-supervised reconstruction loss based on the reconstructed image embedding and the original image embedding; using a data set containing road crack images and labels, training the model based on the constructed joint loss function to obtain a trained crack detection model; in actual detection, after adjusting the size of the road image to be detected, inputting it into the trained crack detection model for inference to generate a mask image of the crack region; by restoring the mask image of the crack region to the original image size, the positioning and visualization of the road crack are realized.

2. The method of claim 1, wherein the method is a method of road crack detection based on multi-scale dilated convolution and domain prompt generation. obtaining original input features extracted by an image encoder; inserting a multi-scale hollow convolution adapter before each Transformer layer of the image encoder; processing the original input features through the multi-scale hollow convolution adapter to generate enhanced image features, comprising: obtaining original input features extracted by an image encoder; inputting the original input features into a multi-scale hollow convolution adapter; processing the original input features through a plurality of parallel hollow convolution branches in the multi-scale hollow convolution adapter to generate a plurality of intermediate branch features corresponding to different scales; enhancing each intermediate branch feature corresponding to a different scale by using a channel attention mechanism to generate enhanced intermediate branch features; fusing all enhanced intermediate branch features to obtain fused features; performing residual connection on the fused features and the original input features to obtain enhanced image features.

3. The method of claim 2, wherein the method is a method of road crack detection based on multi-scale dilated convolution and domain prompt generation. inputting the enhanced image features into a domain prompt encoder, and based on the enhanced image features, the domain prompt encoder dynamically generates a domain prompt vector aligned with the semantics of the current image through an internally constructed feature memory bank and a cross-attention mechanism; performing sequence processing on the domain prompt vector, i.e., reconstructing a corresponding reconstructed image embedding according to a domain prompt decoder, expanding the domain prompt vector to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding, and finally fusing to generate a semantic prompt vector, comprising: inputting the enhanced image features into a domain prompt encoder; Based on the enhanced image features, the domain prompt encoder dynamically generates a domain prompt vector aligned with the semantics of the current image through an internally constructed feature memory bank and a cross-attention mechanism; The domain prompt vector is subjected to a first processing, i.e., the domain prompt vector is input into the domain prompt decoder to reconstruct a corresponding reconstructed image embedding; The domain prompt vector is subjected to a second processing, i.e., the domain prompt vector is expanded to generate an expanded domain prompt vector with the same size as the original image embedding; The similarity between the expanded domain prompt vector and the original image embedding is calculated to obtain a similarity feature map; The expanded domain prompt vector, the similarity feature map, and the original image embedding are fused to generate a semantic prompt vector.

4. The method of claim 3, wherein the method is characterized by, Based on the semantic prompt vector and the original image embedding, a joint loss function for model training is constructed; wherein the joint loss function is composed of a supervised segmentation loss based on the semantic prompt vector and a self-supervised reconstruction loss based on the reconstructed image embedding and the original image embedding, including: Obtaining the semantic prompt vector and the original image embedding; Based on the semantic prompt vector and the original image embedding, a supervised segmentation loss composed of a cross-entropy loss and a Dice coefficient loss is calculated; Based on the reconstructed image embedding and the original image embedding, a self-supervised reconstruction loss is calculated using a mean square error function; The calculated supervised segmentation loss and the calculated self-supervised reconstruction loss are weighted and summed to generate a joint loss function for model training.

5. The method of claim 4, wherein the method is characterized by, A data set containing road crack images and labels is used to train the model based on the constructed joint loss function to obtain a trained crack detection model, including: Prepare a data set containing road crack images and their corresponding labels, and divide the data set into a training set and a validation set; Based on the data set, load and initialize the to-be-trained model containing a multi-scale dilated convolution adapter, a domain prompt encoder, a domain prompt decoder, and a mask decoder, while freezing the backbone weights of the image encoder to obtain an initialized to-be-trained model; Input the images in the training set into the initialized to-be-trained model, and based on the joint loss function, iteratively optimize the trainable parameters in the initialized to-be-trained model through a backpropagation algorithm to obtain a model with updated parameters; Evaluate the performance of the model with updated parameters through the validation set. When the evaluation index tends to be stable or reaches a preset training round, stop training and save the final model parameters to obtain a trained crack detection model.

6. The method of claim 5, wherein the method is a method of road crack detection based on multi-scale dilated convolution and domain prompt generation. In actual detection, adjust the size of the road image to be detected and input it into the trained crack detection model for inference to generate a mask map of the crack region. By restoring the mask map of the crack region to the original image size, the positioning and visualization of the road crack are realized, including: Adjust the road image to be detected to the input size specified by the trained crack detection model to obtain a size-adapted to-be-detected image; Input the size-adapted to-be-detected image into the trained crack detection model for forward inference to generate an initial crack mask map corresponding to the input size; The initial crack mask image is reduced to the original size of the road surface image to be detected by bilinear interpolation to obtain a final crack mask image. Based on the final crack mask image, the crack area is located and visually displayed on the road surface image to be detected.

7. The method of claim 6, wherein the method is a method of road crack detection based on multi-scale dilated convolution and domain prompt generation. The initial crack mask image is reduced to the original size of the road surface image to be detected by bilinear interpolation to obtain a final crack mask image, comprising: An initial crack mask image corresponding to an input size is obtained, and the original size of the road surface image to be detected is obtained. Based on the original size of the road surface image to be detected and the current size of the initial crack mask image corresponding to the input size, a spatial scale transformation parameter is calculated. According to the spatial scale transformation parameter, a size reduction operation is performed on the initial crack mask image by using a bilinear interpolation algorithm to obtain a reduced crack mask image. The reduced crack mask image is output as a final crack mask image consistent with the original size.

8. A road crack detection system based on multi-scale dilated convolution and domain prompt generation, the system implements the method of any one of claims 1 to 7, characterized in that, Comprise: An acquisition and insertion module is used to acquire original input features extracted by an image encoder; A multi-scale dilated convolution adapter is inserted before each Transformer layer of the image encoder; the original input features are processed by the multi-scale dilated convolution adapter to generate enhanced image features; A fusion generation module is used to input the enhanced image features into a domain prompt encoder; based on the enhanced image features, the domain prompt encoder dynamically generates a domain prompt vector aligned with the current image semantics through an internally constructed feature memory bank and a cross-attention mechanism; the domain prompt vector is sequentially processed, i.e., a corresponding reconstructed image embedding is reconstructed according to a domain prompt decoder; the domain prompt vector is expanded to the same size as the original image embedding to calculate the similarity between the expanded vector and the original image embedding, and finally a semantic prompt vector is fused and generated; A function construction module is used to construct a joint loss function for model training based on the semantic prompt vector and the original image embedding; wherein the joint loss function is composed of a supervised segmentation loss based on the semantic prompt vector and a self-supervised reconstruction loss based on the reconstructed image embedding and the original image embedding; A model training module is used to train the model based on the constructed joint loss function using a data set containing road crack image and label to obtain a trained crack detection model; A final inference and reduction module is used to input the road surface image to be detected after adjusting the size into the trained crack detection model for inference to generate a mask image of the crack area; the mask image of the crack area is reduced to the original image size to realize the positioning and visualization of the road crack.

Images