Multispectral image road crack detection method and system based on dynamic threshold and CNN (Convolutional Neural Network)

Abstract

The invention provides a multispectral image road crack detection method and system based on a dynamic threshold value and a CNN. The method comprises the steps that multispectral image data are acquired and preprocessed; extracting local statistical characteristics of the image from the preprocessed multispectral image data, and calculating a dynamic threshold value for each spectral band according to the local statistical characteristics of the image; performing binarization processing on the multispectral image based on the calculated dynamic threshold value to obtain a multispectral image after binarization processing; and establishing a CNN network architecture model, and processing the binarized multispectral image to obtain multispectral image features.

Description

Technical Field

[0001] This invention belongs to the field of multispectral image processing technology, and particularly relates to a method and system for detecting road cracks in multispectral images based on dynamic thresholding and CNN. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of intelligent transportation, road crack detection is a crucial component of infrastructure health monitoring. Traditional detection methods rely on manual inspections or visible light cameras, which are susceptible to uneven lighting, shadows, and complex background interference, leading to high false negative rates. For example, visible light images struggle to capture minute cracks in strong light or at night, and road surface imperfections such as oil stains and fallen leaves are easily misidentified as cracks. Furthermore, minute cracks exhibit low contrast in single-spectral images, making them difficult to extract effectively using traditional thresholding methods. While multispectral images can enhance the difference between cracks and the background through infrared and thermal imaging, existing methods (such as fixed thresholds or linear dimensionality reduction) struggle to adapt to the diversity of road materials and environmental noise, resulting in insufficient feature extraction accuracy. Therefore, a highly adaptive and robust feature extraction technique is urgently needed to improve the automation level of crack detection.

[0004] In road scenarios, the morphology (e.g., linear, mesh-like) and size (width, depth) of cracks directly affect structural safety assessments. Multispectral technology's infrared band can penetrate surface contaminants, while its thermal band reflects internal damage through material stress differences, providing multidimensional data support for crack detection. However, due to the varying road materials, asphalt and concrete pavements exhibit significant differences in spectral response, making it difficult to unify the modeling of linear methods like PCA. Complex environmental noise, such as vehicle shadows and road wear, leads to high false detection rates with traditional static thresholding methods. LDA relies on a large number of labeled samples, but labeling actual road crack data is costly, resulting in insufficient model generalization ability. Therefore, existing methods, such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), have significant limitations in road scenarios.

[0005] With the development of remote sensing technology, environmental monitoring, and other fields, the demand for multispectral images is increasing, and the accuracy requirements are also becoming higher. Traditional monospectral image processing methods have limitations when dealing with complex scenes and targets, while multispectral images provide richer spectral information, which helps to improve the accuracy and efficiency of image processing. For example, the near-infrared band can enhance the reflection difference between cracks and intact road surfaces, while the thermal band can capture the internal stress distribution of materials. Combining the two can significantly improve the reliability of crack detection.

[0006] Multispectral images, by capturing electromagnetic wave information in different bands, provide richer ground feature information than traditional RGB images. These features have significant application value in fields such as environmental monitoring and civil engineering. However, the high dimensionality and complexity of multispectral images also bring challenges such as high processing complexity, noise interference, and difficulty in feature extraction. In road crack detection, direct processing of high-dimensional data (e.g., 10+ bands) leads to low computational efficiency, while noise (e.g., sensor noise, sudden changes in illumination) can mask the weak signal features of cracks. Existing dimensionality reduction methods such as PCA can reduce dimensionality, but their linear assumptions do not match the nonlinear characteristics of road scenes (e.g., the curve shape of cracks), resulting in the loss of key information. Furthermore, the unsupervised nature of PCA prevents optimization for distinguishing cracks from the background, limiting detection accuracy.

[0007] Traditional Principal Component Analysis (PCA) is insufficient for modeling nonlinear features of multispectral images (such as the meandering morphology of road cracks), and parameter selection (such as the number of principal components) requires a trade-off between information loss and redundancy. Its unsupervised nature ignores category information, limiting feature discriminative power in classification tasks. While Linear Discriminant Analysis (LDA) can improve feature discriminative power using category labels, it faces challenges in road scenes, including high cost of small-sample annotation, matrix estimation bias due to high-dimensional data sparsity, and a mismatch between the linear assumption and abrupt changes in local nonlinear contrast of cracks. Furthermore, static thresholding methods are susceptible to noise interference and have a high false positive rate because they cannot adapt to local changes. In summary, these methods often have limited effectiveness when handling complex scenes and variable spectral information. With the development of deep learning technology, especially the emergence of models such as Convolutional Neural Networks (CNNs) and Transformer networks (such as the Transformer), new solutions for image feature extraction have been provided. However, single CNN methods still face challenges such as band redundancy, noise sensitivity, and insufficient modeling of morphological continuity when processing high-dimensional multispectral data.

[0008] In the field of multispectral image data processing, deep learning networks are playing an increasingly important role as a powerful modeling tool. Modeling, through constructing appropriate network structures and optimizing algorithms, enables better modeling and feature extraction of multispectral image data. For example, convolutional and pooling layers in a Convolutional Neural Network (CNN) can model the spatial features of an image, while fully connected layers can combine and classify features. However, in road crack detection, dynamic thresholding techniques need to be further combined. This involves adaptively adjusting the segmentation threshold based on local statistical characteristics (such as mean and standard deviation) to enhance the contrast of cracked areas. For instance, in asphalt pavement, the local standard deviation of cracked areas is significantly higher than that of the background. Dynamic thresholding can optimize the segmentation results in real time based on neighborhood characteristics, which are then input into a CNN for refined feature extraction. This collaborative modeling mechanism of "dynamic thresholding + CNN" reduces data dimensionality while preserving key features, significantly improving detection efficiency and accuracy.

[0009] Deep learning networks can automatically learn feature representations from multispectral image data, avoiding the complexity of manually designing features in traditional methods. By constructing deep neural network models, spectral and spatial features in multispectral images can be jointly modeled, extracting richer and more accurate information. This modeling approach not only improves the accuracy of image processing but also enhances the robustness and generalization ability of the algorithm. For example, in road crack detection, a band attention mechanism can be designed to dynamically allocate weights for different bands (such as enhancing infrared bands at night), and combined with a morphological constraint loss function to optimize the prediction of crack continuity. However, single CNN methods have limited processing capabilities for high-dimensional multispectral data and are sensitive to noise, resulting in high model complexity and difficulty in efficient optimization and solution. Therefore, developing an efficient and accurate multispectral image feature extraction and modeling technique is of great significance.

[0010] Therefore, the main problem with existing technology is: High-dimensional data processing is highly complex: Multispectral images contain multiple bands (such as visible light, near-infrared, thermal infrared, etc.), resulting in high data dimensionality and significantly increasing the computational complexity of traditional methods. While existing dimensionality reduction methods (such as PCA and LDA) can reduce dimensionality, they still face problems such as low computational efficiency and high memory consumption when processing high-dimensional data. In road scenarios, the real-time processing requirements of high-dimensional data (such as real-time vehicle detection) further exacerbate the challenges. 2. Severe noise interference: Multispectral images are easily affected by sensor noise and changes in illumination during acquisition, leading to a decline in data quality. Traditional methods (such as static thresholding methods) cannot adaptively handle local noise, while dimensionality reduction methods such as PCA may lose useful information during denoising, affecting the accuracy of feature extraction. For example, dynamic shadows in road images may be misidentified as cracks.

[0011] 3. High Challenges in Feature Extraction: Multispectral images possess complex nonlinear structures in their spectral and spatial features, making it difficult for traditional linear methods (such as PCA and LDA) to fully extract the intrinsic features of the data. PCA has limited effectiveness when handling nonlinear data, while LDA's performance degrades with small samples and high-dimensional data, resulting in poor feature extraction. The morphological diversity of road cracks and the complexity of the background further increase the difficulty of feature separation.

[0012] 4. Single CNN methods are sensitive to high-dimensional data and noise: Although deep learning (such as CNNs) performs well in multispectral image processing, single CNN methods have limited capabilities when processing high-dimensional multispectral data and are sensitive to noise. Furthermore, deep learning models are highly complex, difficult to train and optimize, and struggle to achieve high computational efficiency while maintaining accuracy. For example, road crack detection requires lightweight models to meet the deployment needs of edge devices, but traditional CNNs have a large number of parameters, making them difficult to apply directly.

[0013] In summary, while multispectral imaging technology offers numerous advantages, the complexity and high dimensionality of its data present significant challenges to subsequent data processing and analysis. In the specific scenario of road crack detection, existing modeling methods struggle to balance efficiency and accuracy. There is an urgent need for a collaborative modeling technique that integrates dynamic thresholding and deep learning. Through adaptive feature extraction and noise suppression, this technique can achieve efficient and robust crack detection, supporting the intelligent operation and maintenance of smart transportation infrastructure. Summary of the Invention

[0014] To overcome the shortcomings of the prior art, this invention provides a multispectral image road crack detection method based on dynamic thresholding and CNN, which combines the adaptability of dynamic thresholding and the automatic feature learning capability of CNN to achieve efficient and robust feature extraction.

[0015] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a road crack detection method based on dynamic thresholding and multispectral image features of CNN is disclosed, including: Acquire multispectral image data of road cracks to be detected and perform preprocessing; Local statistical properties of the preprocessed multispectral image data are extracted, and dynamic thresholds are calculated for each spectral band based on the local statistical properties of the image. The multispectral image is binarized based on the calculated dynamic threshold to obtain the binarized multispectral image. A CNN network architecture model is established to process the binarized multispectral image to obtain multispectral image features. The multispectral image features are then classified to obtain the road crack detection results.

[0016] The classification of multispectral image features includes: roads with cracks and roads without cracks.

[0017] As a further technical solution, local statistical characteristics of the preprocessed multispectral image data are extracted, and a dynamic threshold is calculated for each spectral band based on these local statistical characteristics. Specifically: The local mean and local standard deviation of each pixel in the preprocessed multispectral image data are calculated by a sliding window. The local mean reflects the average gray value of the pixel neighborhood and captures the local brightness features of the image, while the local standard deviation reflects the degree of gray value variation in the pixel neighborhood and captures the local contrast features of the image. By combining the local mean and local standard deviation, a dynamic threshold is calculated for each pixel. The dynamic threshold is adaptively adjusted to adapt to local changes in the image, ensuring that the threshold can change dynamically according to the local characteristics of the image.

[0018] As a further technical solution, the grayscale value of each pixel is compared with a dynamic threshold to generate a binary image. If the pixel value is greater than the dynamic threshold, it is determined to be the target area and assigned a value of 1. If the pixel value is less than or equal to the dynamic threshold, it is determined to be the background and assigned a value of 0, thereby clearly marking the target area.

[0019] As a further technical solution, the generated binary image is post-processed, including noise removal and isolation point extraction to obtain an optimized binary image.

[0020] As a further technical solution, in the CNN network architecture model: the multispectral image after dynamic thresholding is used as the input value, the classification task uses Softmax to output the class probability, and the segmentation task uses Sigmoid to output the pixel-level classification result.

[0021] As a further technical solution, the CNN network architecture model includes: Convolutional and pooling layers are used to extract spatial features of an image. Convolutional layers capture local features through convolutional kernels and use ReLU activation functions to introduce non-linearity. Pooling layers use max pooling to gradually reduce the feature map size. Through multi-layer convolution and pooling, the network is able to extract the complex features of cracks step by step from low to high levels. The feature map output from the pooling layer is flattened and then input into the fully connected layer. The number of neurons is gradually reduced, and finally the classification result is output through Softmax. For segmentation tasks, upsampling layers are used to gradually restore the feature map size, and skip connections are combined to fuse the low-level features of the encoder with the high-level features of the decoder. Finally, pixel-level segmentation results are output through convolution.

[0022] Secondly, a road crack detection system based on dynamic thresholding and multispectral image features of CNN is disclosed, including: The data acquisition module is configured to acquire multispectral image data and perform preprocessing. The dynamic threshold calculation module is configured to: extract local statistical characteristics of the preprocessed multispectral image data, and calculate a dynamic threshold for each spectral band based on the local statistical characteristics of the image. The binarization processing module is configured to: perform binarization processing on the multispectral image based on the calculated dynamic threshold to obtain the binarized multispectral image; The feature extraction module is configured to: establish a CNN network architecture model, process the binarized multispectral image to obtain multispectral image features, classify the multispectral image features, and obtain road crack detection results.

[0023] The above one or more technical solutions have the following beneficial effects: (1) To address the issue that deep learning may lack sufficient flexibility to adapt to different image characteristics and scenes when processing complex and varied multispectral images of road cracks, this invention proposes a feature extraction technique based on dynamic thresholding and convolutional neural networks. The dynamic thresholding method can adaptively adjust the threshold according to the local features of the image, while CNN can learn and extract deep features in the image. For multispectral images of road cracks with complex shape features, variable lighting conditions, or complex backgrounds, the combination of dynamic thresholding and CNN can significantly improve the accuracy and robustness of road crack image processing. When processing road crack images, the dynamic thresholding method not only considers the pixel gray values ​​in the neighborhood of the original pixel, but also considers the position information of the target pixel in the image, thus better adapting to the feature extraction needs of different regions and different lighting conditions in the image. Through dynamic thresholding, the details and edge information in the image can be better preserved. Subsequently, CNN can further extract these fine features for subsequent image classification, recognition, and other tasks. This combination makes feature extraction more accurate and comprehensive.

[0024] (2) This invention controls the complexity of the convolutional neural network model by dynamically adjusting the threshold, thereby improving the model's generalization ability and reducing overfitting to a certain extent. The dynamic threshold can adaptively adjust the threshold size according to the actual situation during training, such as changes in the model's training error, validation error, or loss function. In the early stages of training, a lower threshold may allow the model to learn more features, thereby increasing the model's complexity; while in the later stages of training, a higher threshold may limit the model from learning too many features, reducing the model's complexity. The model maintains a high learning rate in the early stages of training to quickly capture the main features of the data, and gradually reduces the learning rate in the later stages of training to finely adjust the model parameters, thereby avoiding overfitting. This control mechanism helps the model maintain sufficient complexity while avoiding overfitting the training data.

[0025] (3) This invention optimizes model performance by introducing dynamic thresholding. Deep learning models typically require long training times and high computational resource consumption. However, methods based on dynamic thresholding and CNNs can reduce computational complexity and improve processing speed to some extent by simplifying the model structure or optimizing the algorithm. Before feature extraction, dynamic thresholding accurately segments the image into different regions of interest (ROIs) based on the spectral characteristics of different regions in the image, which serve as inputs to the CNN model, thus simplifying the network structure. In some cases, combining dynamic thresholding with parameter adjustment of the CNN model can reduce the number of model parameters and computational complexity, thereby improving the model's operating efficiency.

[0026] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a flowchart of the multispectral image feature extraction and modeling technique of the present invention; Figure 2 Here are example images for binarization processing, where (a) is the image before binarization and (b) is the image after binarization. Figure 3 These are schematic diagrams illustrating the effects of satellite image processing, where (a) is the processed satellite image; (b) is the image after global thresholding; and (c) is the image after dynamic thresholding. Figure 4 This is a flowchart illustrating the overall implementation of the method of the present invention; Figure 5This is a schematic diagram of the overall system of the present invention. Detailed Implementation

[0029] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0031] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0032] Dynamic thresholding techniques adjust the threshold dynamically based on local statistical characteristics (such as mean and standard deviation) to adapt to changes in different regions. Compared to global thresholding methods, dynamic thresholding requires less computation, is suitable for real-time processing, and is more robust to changes in illumination and noise. CNNs, on the other hand, perform exceptionally well in image processing, particularly in feature extraction and classification tasks.

[0033] Furthermore, the technical solution of this invention uses convolutional neural networks instead of traditional methods, reducing the complexity of high-dimensional data processing from exponential to linear or polynomial levels, while preserving key spatial-spectral information. The end-to-end learning framework combines the parallel computing capabilities of modern GPUs. Embedding a dynamic thresholding mechanism into the hierarchical feature extraction process of CNNs avoids the sensitivity of existing methods to high-dimensional data and noise.

[0034] Example 1 See appendix Figure 1 , 4 As shown, this embodiment discloses a road crack detection method based on dynamic thresholding and CNN in multispectral images. Taking road crack detection as an example, firstly, the multispectral image of the road crack is preprocessed using dynamic thresholding. By calculating the local mean and standard deviation, a binarized image is adaptively generated to enhance the contrast of the crack area and suppress background noise. Then, the binarized image is input into a pre-designed CNN model. Multi-level spatial features are extracted through convolutional and pooling layers, and the crack area is accurately classified by combining fully connected layers. Finally, the detection results are optimized through post-processing to remove noise and isolated points, generating high-quality road crack detection results. This method combines the adaptability of dynamic thresholding and the automatic feature learning capability of CNN, which can effectively improve the accuracy and robustness of crack detection. The specific implementation of the above steps includes the following steps: Step S1: Preprocess the multispectral image data of road cracks, including denoising, correction, and format conversion, to improve the quality and consistency of the data and prepare for modeling.

[0035] In the multispectral image processing workflow of road cracks, multispectral images of road cracks are first acquired through multispectral cameras, satellite remote sensing, or laboratory equipment to ensure the selection of appropriate bands and high resolution. Then, preprocessing is performed before denoising, including image registration (aligning different bands), normalization (eliminating differences in illumination and sensors), band fusion (optional, to enhance information content), and image cropping (extracting regions of interest).

[0036] The collection and denoising of multispectral image data of road cracks includes the following steps: S1-1: First, analyze the noise in the multispectral image of road cracks to determine the type, distribution and intensity of the noise, and select a suitable denoising algorithm based on the analysis results; S1-2: Based on the selected denoising algorithm, configure the corresponding parameters and settings, including the filter type, size, number of iterations, etc., as well as the input, output and training parameters of the deep learning model, and apply the configured denoising algorithm to the multispectral image of road cracks. S1-3: Objective evaluation metrics, such as signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR), are used to evaluate the denoised image. Based on the evaluation results, the parameters of the algorithm are optimized and adjusted to improve the denoising effect and image quality. S1-4: After denoising, perform post-processing operations such as image enhancement and detail restoration to further improve the readability and visual effect of the image, providing a better foundation for subsequent feature extraction and image processing.

[0037] In this embodiment, the denoising process in step S1 enhances the contrast between the target area and the background. The crack area can be further highlighted by the dynamic threshold, thereby reducing false detections and missed detections. In addition, the combination of denoising and dynamic threshold can improve the robustness of the algorithm to changes in illumination and noise, simplify subsequent processing, and improve overall efficiency and detection accuracy.

[0038] Specifically: Taking a multispectral image used to detect road cracks as an example, the local mean and variance are calculated according to the following formula:

[0039]

[0040] in denoted as 'r', where 'r' is the neighborhood radius and 'N' is the number of neighboring pixels.

[0041] In multispectral road crack detection, the local mean and variance are closely related to spectral characteristics. Through calculation, we can effectively distinguish cracked areas from intact road surfaces.

[0042] Because the local mean reflects the average reflectance intensity of a region, the mean is usually low in crack areas; the local variance reflects the complexity of the texture, and the variance is high at the crack edges. Different wavelengths respond differently to cracks; for example, the visible light band captures shadows, while the near-infrared band reflects material changes. Therefore, by fusing multi-band statistics, crack characteristics can be enhanced.

[0043] Next steps: Calculate local mean and variance for each band; weighted fuse multi-band statistics to generate comprehensive features and complete feature fusion; finally, perform threshold segmentation or deep learning classification based on the fused features to detect cracks.

[0044] Assess the noise intensity to select the appropriate denoising method. Gaussian filtering is suitable for Gaussian noise, median filtering is suitable for salt-and-pepper noise, and wavelet transform is suitable for frequency domain noise. For Gaussian filtering, the filter size is k×k (e.g., 3×3, 5×5), and the standard deviation σ is used to control the smoothing intensity, as shown in the following formula:

[0045] Here, (x, y) are the coordinates relative to the center of the filter. The filter size k is usually an odd number, thus having a definite center point; for example, the center of a 3×3 filter is (1, 1). The generated Gaussian kernel then needs to be normalized to ensure that the sum of all elements is 1, so that the brightness of the processed image remains unchanged.

[0046] For median filtering, the filter size is k×k. Neighboring pixels are sorted, and the median is taken as the output. The denoising effect on the multispectral image is evaluated. The filter size and standard deviation are adjusted, and the changes in SNR and PSNR are observed to complete parameter optimization. Signal-to-noise ratio

[0047] Peak signal-to-noise ratio

[0048] in, The maximum pixel value of the image is denoted by MSE, which represents the mean square error.

[0049] Histogram equalization is performed on local regions using adaptive contrast enhancement (CLAHE). For non-local mean filtering, details are recovered using information from similar regions in the image, as shown in the following formula:

[0050] The formula gives the left side as the filtered pixel (x, y) value; the right side, Z, is the normalization factor, where... For the weight function, I is the normalization factor, and I is the gray value of pixel (i,j) in the image.

[0051] in:

[0052] Where w is the weight between pixels (x,y) and (i,j).

[0053] .

[0054] Step S2: Calculate the dynamic threshold for each spectral band based on the local statistical characteristics of the image (i.e., mean and variance) to binarize the multispectral image.

[0055] The calculation of the dynamic threshold includes the following steps: S2-1: Calculate the local mean and standard deviation of each pixel using a sliding window. The local mean reflects the average gray value of the pixel's neighborhood, capturing the local brightness characteristics of the image, while the local standard deviation reflects the degree of gray value variation in the pixel's neighborhood, capturing the local contrast characteristics of the image, thus laying the foundation for subsequent calculations. S2-2: Combine local mean and standard deviation to calculate the threshold for each pixel. The dynamic threshold can be adaptively adjusted to adapt to local changes in the image, ensuring that the threshold can change dynamically according to the local characteristics of the image, thereby improving the ability to distinguish between the target area and the background. S2-3: Compare the gray value of each pixel with the dynamic threshold to generate a binary image. If the pixel value is greater than the dynamic threshold, it is determined to be the target area (assigned a value of 1). If the pixel value is less than or equal to the dynamic threshold, it is determined to be the background (assigned a value of 0). This makes the target area (such as road cracks) clearly marked, which is convenient for subsequent analysis and processing. S2-4: Post-process the binarized image to remove noise and isolated points, optimize the detection results, and further improve the quality of the binary image through morphological operations (such as opening operations to remove small noise and closing operations to fill small holes) and connected component analysis (removing connected components with too small area), ensuring that the target region is more complete and providing a high-quality binary image for subsequent feature extraction and image analysis. The above binary image is as follows Figure 2 As shown, where, Figure 2 In the image, (a) is the image before binarization, and (b) is the image after binarization. Figure 3 shows the effect of processing ordinary satellite images with global and dynamic thresholding. Figure 3 (a) in the image is a satellite image processing label; Figure 3 (b) in the figure shows the effect after global thresholding; Figure 3 (c) in the image is the result of dynamic thresholding of the satellite image.

[0056] The calculation of the dynamic threshold is as follows: The crack region is extracted from the preprocessed multispectral image according to the formula:

[0057] The specific meanings of the above parameters are as follows: I(x+i, y+j): The pixel value of the preprocessed multispectral image at coordinates (x+i, y+j); r: The neighborhood radius of the sliding window (usually 3-5), which defines the calculation range of local statistics; N: Total number of pixels in the neighborhood (N=(2r+1)) 2 ); μ(x,y): The local mean of pixel grayscale values ​​within the sliding window, reflecting the brightness level of the region, and used for dynamic threshold calculation.

[0058] In road crack detection, cracked areas are typically darker than intact pavement (e.g., asphalt cracks have low reflectivity due to light absorption), and local mean values ​​can characterize the brightness difference between the crack and the background. For example, the μ(x,y) value of a cracked area is usually lower than that of the surrounding intact pavement.

[0059] Then use the formula to calculate the local standard deviation: ; Substitute the obtained values ​​into the formula Z calculates the dynamic threshold for each pixel, where For pixels The dynamic threshold at the given point is used to adjust the threshold sensitivity (usually taken as 1.0-2.0).

[0060] The grayscale value of each pixel is compared with a dynamic threshold to generate a binary image, and the crack area is distinguished from the background.

[0061]

[0062] Where B(x,y) is the pixel value after binarization, and I(x,y) is the pixel value of the original image; Post-processing of binarized images is necessary. When the image has a lot of noise or the crack region is incomplete, dynamic operations are required. When there are a large number of small non-crack regions in the image, connected component analysis is required.

[0063] Dynamic learning operations:

[0064]

[0065] Connectivity analysis: Remove connected regions with excessively small areas to avoid false detections. Set a minimum area threshold (e.g., 50 pixels) to filter out small regions, remove noise and isolated points, and optimize crack detection results.

[0066] Opening operations are suitable for removing small noise points, smoothing boundaries, and separating adhered regions. Closing operations are suitable for filling small holes, connecting broken regions, and smoothing boundaries. In practical applications, the two can be used in combination: first remove noise, then fill holes and connect broken regions, thereby significantly improving the quality of the binarized image.

[0067] Step S3: Design the CNN network architecture according to the task requirements and data characteristics (number of bands in the multispectral image). By reasonably designing convolutional layers, pooling layers, fully connected layers, skip connections and upsampling layers, extract the features of road cracks and output high-precision detection results, as shown in Table 1.

[0068] Table 1

[0069] The design of a CNN model includes the following steps: S3-1: Define the input data and task objectives of the model. Use the multispectral image (size: H×W×C, where C is the number of bands) after dynamic thresholding as the input value. Determine the design of the output layer according to the objective. Use Softmax to output class probabilities for classification tasks and use Sigmoid to output pixel-level classification results for segmentation tasks. S3-2: Design a feature extraction network. Convolutional layers and pooling layers are used to extract spatial features of the image. The convolutional layers capture local features using 3×3 or 5×5 convolutional kernels, and the activation function uses ReLU to introduce non-linearity. The pooling layers use 2×2 max pooling to progressively reduce the feature map size, reduce computation, and enhance the robustness of the features. Through multiple layers of convolution and pooling, the network can progressively extract complex features of cracks from low to high levels. S3-3: Construct a feature fusion and output network. For classification tasks, the feature map output by the pooling layer is flattened and then input into the fully connected layer to gradually reduce the number of neurons. Finally, the classification result is output through Softmax. For segmentation tasks, upsampling layers (such as transposed convolution) are used to gradually restore the feature map size. Skip connections are combined to fuse the low-level features of the encoder with the high-level features of the decoder. Finally, pixel-level segmentation results are output through 1×1 convolution. S3-4: Select an appropriate loss function (such as cross-entropy loss or Dice Loss) and optimizer (such as Adam), configure the learning rate and training parameters, optimize the model parameters through backpropagation, and use data augmentation to improve the model's generalization ability. Monitor the validation set performance during training, use early stopping to prevent overfitting, and finally obtain a high-precision road crack detection model.

[0070] The design and construction of the CNN model are as follows: The input layer takes a multispectral image after dynamic thresholding. The first convolutional layer has 32 3×3 convolutional kernels with an output size of H×W×32. The second convolutional layer has 64 3×3 convolutional kernels with an output size of H×W×64. The convolutional kernel stride is 1, the padding method is "same", and the ReLU nonlinear activation function is selected. Pooling layer windows are typically 2×2 with a stride of 2. Therefore, the first layer is set to 2×2 max pooling, with an output size of H / 2×W / 2×32. The second layer is also set to 2×2 max pooling, with an output size of H / 4×W / 4×64. This reduces the dimensionality of the feature maps, decreases computation, and enhances the model's robustness. After feature extraction in convolutional and pooling layers, the fully connected layer flattens the feature map into a one-dimensional vector, and then performs feature combination and classification through a multi-layer neural network. The first fully connected layer has 128 neurons with ReLU activation. The second fully connected layer has 64 neurons with ReLU activation. Dropout is used after the fully connected layers to randomly discard some neurons to prevent overfitting. ReLU activation is used in the intermediate layers, and Softmax is used in the output layer. For classification tasks (such as road crack detection), the output layer is typically a softmax layer, outputting the probability distribution for each class. For regression tasks (such as crack width prediction), the output layer can be a linear layer, directly outputting the predicted value. Furthermore, for pixel-level tasks (such as image segmentation), upsampling layers are used. The first upsampling layer is a 2×2 transposed convolution with an output size of H / 2×W / 2×64. The second upsampling layer is a 2×2 transposed convolution with an output size of H×W×32, thus restoring the feature map to the original image size and generating pixel-level prediction results.

[0071] Step S4: Evaluate the constructed model to ensure that it can run stably and efficiently in practical applications, meet task requirements, and continuously monitor and optimize the model according to application requirements.

[0072] Model evaluation, optimization, deployment, and application include the following steps: S4-1: Evaluate model performance using a test set. For classification tasks, the evaluation metrics are accuracy, precision, recall, and F1 score, which measure the model's ability to identify crack categories. For segmentation tasks, IoU (Intersection over Union) is used to assess the overlap between the predicted and ground truth regions; the Dice coefficient measures the similarity between the predicted result and the ground truth label; pixel accuracy determines the proportion of correctly classified pixels; visualize the prediction results to check for false positives or false negatives; compare the performance of different models; and select the optimal model. S4-2: Further optimize the model based on the evaluation results to improve performance. Perform hyperparameter tuning by adjusting parameters such as learning rate, batch size, and network depth to increase the diversity of training data and improve the model's generalization ability. Remove redundant parameters to reduce model complexity, or use pre-trained models for fine-tuning to accelerate training and improve performance. S4-3: Deploy the trained model to a real-world application environment for real-time detection. Convert the trained model to a deployment-friendly format (such as TensorFlow Lite or ONNX), use techniques such as quantization and pruning to reduce model computation and memory usage, and test model performance in a real-world environment to ensure stability and reliability. S4-4: Continuous monitoring and updates ensure the model continues to run efficiently in practical applications. Monitor the model's predictive performance and running status in real time, collect user feedback and actual data for model iteration, and retrain and optimize the model regularly based on new data and needs.

[0073] Conduct model testing, complete optimization and deployment, specifically as follows: Load the test set and perform model inference. Evaluate the model's performance using the test set (including multispectral images and corresponding crack annotations). Perform dynamic threshold preprocessing on the images to generate binarized images, which are then input into the CNN model to obtain prediction results, ensuring its generalization ability. The formulas for each metric are as follows: accuracy

[0074]

[0075]

[0076] F1 score

[0077] Intersection and Union

[0078] Where TP is a true positive (correctly detected crack pixel), TN is a true negative (correctly detected background pixel), FP is a false positive (background pixel that is falsely detected as a crack), and FN is a false negative (crack pixel that is missed). Based on the evaluation results on the test set, adjust hyperparameters such as learning rate, batch size, and number of convolutional kernels. Then, improve the model structure by increasing the depth of convolutional layers and using more complex networks (such as ResNet and U-Net). Perform enhancement operations on the training data, such as rotation, flipping, and brightness adjustment, to improve model robustness. Use Dropout or L2 regularization to prevent overfitting. Optimize model parameters and structure to improve performance. The optimized model is deployed to the actual road detection system. The trained model is saved as a file (such as HDF5 or SavedModel format). Tools such as TensorFlow Serving and ONNX Runtime are used to load the model and integrate it into the application system. Dynamic threshold preprocessing and model inference are performed on the real-time acquired multispectral images to achieve real-time crack detection. Monitor the model's performance in real-world environments, regularly evaluate metrics such as accuracy and recall, collect new data from real-world environments, expand the training set, retrain the model using the new data, optimize parameters, and regularly update the model using model version control tools (such as MLflow).

[0079] Example 2 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0080] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0081] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0082] Example 4 See appendix Figure 5 As shown, the purpose of this embodiment is to provide a multispectral image road crack detection system based on dynamic thresholding and CNN, including: The data acquisition module is configured to acquire multispectral image data of road cracks to be detected and perform preprocessing. The dynamic threshold calculation module is configured to: extract local statistical characteristics of the preprocessed multispectral image data, and calculate a dynamic threshold for each spectral band based on the local statistical characteristics of the image. The binarization processing module is configured to: perform binarization processing on the multispectral image based on the calculated dynamic threshold to obtain the binarized multispectral image; The feature extraction module is configured to: establish a CNN network architecture model, process the binarized multispectral image to obtain multispectral image features, classify the multispectral image features, and obtain road crack detection results.

[0083] Example 5 The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods and functions involved in any of the above embodiments. The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0084] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0085] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A road crack detection method based on dynamic thresholding and CNN using multispectral images, characterized by: include: Acquire multispectral image data of road cracks to be detected and perform preprocessing; Local statistical properties of the preprocessed multispectral image data are extracted, and dynamic thresholds are calculated for each spectral band based on the local statistical properties of the image. The multispectral image is binarized based on the calculated dynamic threshold to obtain the binarized multispectral image. A CNN network architecture model is established to process the binarized multispectral image to obtain multispectral image features. The multispectral image features are then classified to obtain the road crack detection results.

2. The method for detecting road cracks in multispectral images based on dynamic thresholding and CNN as described in claim 1, characterized in that, Local statistical properties of the preprocessed multispectral image data are extracted, and a dynamic threshold is calculated for each spectral band based on these local statistical properties. Specifically: The local mean and local standard deviation of each pixel in the preprocessed multispectral image data are calculated by a sliding window. The local mean reflects the average gray value of the pixel neighborhood and captures the local brightness features of the image, while the local standard deviation reflects the degree of gray value variation in the pixel neighborhood and captures the local contrast features of the image. By combining the local mean and local standard deviation, a dynamic threshold is calculated for each pixel. The dynamic threshold is adaptively adjusted to adapt to local changes in the image, ensuring that the threshold can change dynamically according to the local characteristics of the image.

3. The method for detecting road cracks in multispectral images based on dynamic thresholding and CNN as described in claim 1, characterized in that, The grayscale value of each pixel is compared with a dynamic threshold to generate a binary image. If the pixel value is greater than the dynamic threshold, it is determined to be the target area and assigned a value of 1. If the pixel value is less than or equal to the dynamic threshold, it is determined to be the background and assigned a value of 0, thus clearly marking the target area.

4. The method for detecting road cracks in multispectral images based on dynamic thresholding and CNN as described in claim 1, characterized in that, The generated binary image is post-processed, including noise removal and isolation point removal to obtain an optimized binary image.

5. The method for detecting road cracks in multispectral images based on dynamic thresholding and CNN as described in claim 1, characterized in that, In the CNN network architecture model: the multispectral image after dynamic thresholding is used as the input value, the classification task uses Softmax to output the class probability, and the segmentation task uses Sigmoid to output the pixel-level classification result.

6. The method for detecting road cracks in multispectral images based on dynamic thresholding and CNN as described in claim 1, characterized in that, The CNN network architecture model includes: Convolutional and pooling layers are used to extract spatial features of the image. Convolutional layers capture local features through convolutional kernels and use ReLU activation function to introduce non-linearity. Pooling layers use 2×2 max pooling to gradually reduce the feature map size. Through multi-layer convolution and pooling, the network is able to extract the complex features of cracks step by step from low to high levels. The feature map output from the pooling layer is flattened and then input into the fully connected layer. The number of neurons is gradually reduced, and finally the classification result is output through Softmax. For segmentation tasks, upsampling layers are used to gradually restore the feature map size, and skip connections are combined to fuse the low-level features of the encoder with the high-level features of the decoder. Finally, pixel-level segmentation results are output through 1×1 convolution.

7. A multispectral image road crack detection system based on dynamic thresholding and CNN, characterized in that, include: The data acquisition module is configured to acquire multispectral image data of road cracks to be detected and perform preprocessing. The dynamic threshold calculation module is configured to: extract local statistical characteristics of the preprocessed multispectral image data, and calculate a dynamic threshold for each spectral band based on the local statistical characteristics of the image. The binarization processing module is configured to: perform binarization processing on the multispectral image based on the calculated dynamic threshold to obtain the binarized multispectral image; The feature extraction module is configured to: establish a CNN network architecture model, process the binarized multispectral image to obtain multispectral image features, classify the multispectral image features, and obtain road crack detection results.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1-6 above.

Images