A road surface damage detection method based on local gray extremum points and feature fusion

Abstract

The application relates to a road surface damage detection method based on local gray extremum points and feature fusion, which comprises the following steps: dividing a road surface image into a plurality of windows, and respectively calculating local extremum points in the window units; determining a plurality of local extremum points as feature points according to the gray distribution of the neighborhood blocks of the extremum points; intercepting the image blocks of the neighborhoods of the local extremum points as the centers, establishing a block image data set, and training a CNN block image feature extraction model; using the trained CNN feature model to calculate the feature vectors corresponding to the feature points of the image windows, performing feature fusion to obtain a composite feature vector; and using a support vector machine to classify the composite feature vector to detect the damage of the road surface. The application has the beneficial effects that the application combines some prior image features of the damage target area with a deep learning algorithm, reduces the scale of the data set and the requirement of the calculation capacity, and improves the detection effect of the road surface.

Description

Technical Field

[0001] This invention relates to the field of road damage detection, and more specifically, to a road damage detection method based on the fusion of local gray-level extreme points and features. Background Technology

[0002] Road maintenance integrating cutting-edge technologies such as image sensing, big data, and artificial intelligence can accurately and comprehensively perceive and predict road conditions. This provides decision-makers with reliable data analysis and decision-making support, improving road safety and transportation efficiency. Currently, road image acquisition technology is developing rapidly; for example, global shutter cameras can achieve rapid imaging and have a significant price advantage over ground-penetrating radar and laser systems. However, research on automatic road image processing algorithms still faces many challenges. Given the differences in road conditions and economic levels across regions, researching adaptable road detection methods is of greater practical significance.

[0003] Road surface image recognition technology has evolved from traditional image processing methods to deep learning-based detection methods. Traditional image processing methods segment images by manually designing features, including thresholding, edge detection, texture segmentation, and multi-feature fusion methods. These methods can achieve good detection results on specific datasets. Deep learning algorithms significantly reduce the steps of feature description, extraction, and recognition based on human experience, which is beneficial to improving the accuracy and versatility of road surface recognition algorithms.

[0004] Deep learning algorithms essentially achieve detection through the powerful fitting ability of neural networks. However, directly using high-resolution images to train deep learning neural networks places high demands on massive road surface datasets and computational power. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a road surface damage detection method based on the fusion of local gray-level extreme points and features.

[0006] Firstly, a pavement damage detection method based on local gray-level extreme points and feature fusion is provided, including:

[0007] Step 1: Acquire a road surface image and preprocess the road surface image;

[0008] Step 2: Divide the road surface image into several windows, and calculate the local extreme points for each window as a unit;

[0009] Step 3: Based on the grayscale distribution of the neighborhood of the extreme point, determine several local extreme points as feature points;

[0010] Step 4: Using local extreme points as the center, extract image blocks in their neighborhood, establish a block image dataset, and train a CNN block image feature extraction model.

[0011] Step 5: Using the trained CNN feature model, calculate the feature vectors corresponding to each feature point in the image window, perform feature fusion to obtain a composite feature vector; use a support vector machine to classify the composite feature vectors and detect the damage to the road surface.

[0012] Preferably, step 1 includes:

[0013] Step 1.1: Acquire a road surface image and perform Gaussian filtering on the road surface image;

[0014] Step 1.2: Perform gamma correction on the road surface image. The formula for gamma correction is:

[0015] I(x,y)=I(x,y) γ

[0016] The formula for determining the value of γ is:

[0017]

[0018] Among them, I P I0 is the peak value of the grayscale distribution before correction, and I0 is the peak value of the grayscale distribution after correction.

[0019] Preferably, step 2 includes:

[0020] Step 2.1: Divide the road surface image into road surface image windows of size Nw×Nw, first invert the grayscale of the image, and then search for the maximum value in units of image windows;

[0021] Step 2.2: Define two parameters: the minimum distance P for peak detection. dist And the threshold P for peak detection th The threshold P is calculated according to the following formula. th Convert to peak detection amplitude P per window A :

[0022] P A =P th *(G max -G min )+G min

[0023] Among them, G max and G min These are the maximum and minimum grayscale values ​​for the corresponding window;

[0024] Step 2.3: Obtain the first-order difference and the index position of the difference amplitude of 0. Adjacent 0 positions indicate that it is in a horizontal state. Only the middle point is selected as the unique peak point.

[0025] Step 2.4: Obtain the peak points that meet the peak detection criteria. The criteria are that the gradient sign changes and the gray value is greater than the peak detection amplitude P. A ;

[0026] Step 2.5: Use the minimum peak distance P dist Invalid peak points are filtered out to obtain local extreme points.

[0027] Preferably, in step 2.5, the distance between the local extrema is comparable to the size of the captured image block of the extrema neighborhood.

[0028] Preferably, step 3 includes:

[0029] Step 3.1: Determine the grayscale distribution of the neighborhood blocks of the extreme point, mainly considering the average grayscale value and variance; the grayscale value of the pixel in the i-th block is G. i Given (x, y), which has N pixels, the calculation formula is as follows:

[0030]

[0031]

[0032] Among them, E i The mean value reflects the overall brightness around the extreme point; σ i The variance represents the grayscale distribution of pixels in a block; "block" indicates a region.

[0033] Step 3.2: For the average gray value and variance of the neighborhood of the extreme point, a critical value is given for preliminary screening of local extreme points;

[0034] Step 3.3: Sort the remaining extreme points according to gray level, select a certain number of extreme points as feature points of the window, and extract the neighborhood blocks of the feature points for analysis of the image window.

[0035] Preferably, step 4 includes:

[0036] Step 4.1: Extract block images from public datasets and actual road surface images that need to be detected, and establish a block image dataset;

[0037] Step 4.2: Construct a CNN feature extraction model. The essence of CNN lies in performing convolution and pooling calculations layer by layer on the input image data to extract data features; the convolution operation formula for a single convolution kernel is:

[0038]

[0039] Among them, y i,j The output pixel value represents the convolution, f represents the activation function, and W represents the value of the convolution output pixel. u,v Let X represent the weight in the u-th row and v-th column of the convolution kernel, where U and V are the kernel sizes, b represents the kernel bias, and X represents the weight in the v-th column of the convolution kernel. i,j This represents the element in the i-th row and j-th column of the feature map.

[0040] As a preferred embodiment, the specific method for manually extracting image blocks to establish a dataset in step 4.1 includes: determining local extreme points in the image window and labeling them with serial numbers in the image; identifying extreme points located in the damaged area, inputting their serial numbers to extract image blocks and labeling them as positive samples; while the blocks corresponding to local extreme points in the normal road surface image are labeled as negative samples.

[0041] Preferably, step 5 includes:

[0042] Step 5.1: The CNN block image feature extraction model outputs feature vectors from the fully connected layer;

[0043] Step 5.2: Combine the feature vectors of all feature points involved in the image window to form a feature vector that describes the entire window;

[0044] Step 5.3: Use the composite feature vector as the feature descriptor of the image window, and use a support vector machine to construct a classifier for feature recognition to determine whether the image window contains a damaged area.

[0045] Secondly, a pavement damage detection device based on local gray-level extreme points and feature fusion is provided, for performing the pavement damage detection method based on local gray-level extreme points and feature fusion as described in the first aspect, including:

[0046] The acquisition module is used to acquire road surface images and preprocess the road surface images;

[0047] The segmentation module is used to divide the road image into several windows, and calculate local extreme points for each window as a unit.

[0048] The determination module is used to determine several local extreme points as feature points based on the gray-scale distribution of the neighborhood blocks of extreme points;

[0049] The training module is used to extract image blocks in the neighborhood of local extreme points, build a block image dataset, and train a CNN block image feature extraction model.

[0050] The detection module uses a trained CNN feature model to calculate the feature vectors corresponding to each feature point in the image window, performs feature fusion to obtain a composite feature vector, and uses a support vector machine to classify the composite feature vector to detect road surface damage.

[0051] Thirdly, a computer storage medium is provided, wherein a computer program is stored in the computer storage medium; when the computer program is run on a computer, the computer executes the road damage detection method based on local gray-level extreme points and feature fusion as described in the first aspect.

[0052] The beneficial effects of this invention are as follows: This invention combines some prior image features of the damaged target area with a deep learning algorithm. Starting from local minima that meet certain gray-level distribution conditions within the road surface image window as feature points, and using image patches surrounding the feature points as research units, a convolutional neural network model is trained using a self-built dataset composed of small image patches to extract the depth features of the image patches. This significantly reduces the size of the dataset and the computational requirements. The patch image dataset, built by combining existing public datasets and images of actual detected roads, has clear targeting and is beneficial for improving road surface detection performance. Multiple patches extracted from the image window often belong to the same damaged target, and there is always a certain correlation between them. Combining the feature vectors involved in the image window forms a composite feature vector describing the entire window. Correlation analysis of multiple patches may improve the accuracy of road surface detection. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the distribution of local extreme points before and after filtering by neighborhood average gray level and variance conditions provided in this application.

[0054] Figure 2 A schematic diagram of an image block for cropping the neighborhood of feature points provided in this application;

[0055] Figure 3 A schematic diagram illustrating the manual method used in this application to determine extreme points located in the damaged area;

[0056] Figure 4 The flowchart of the window image recognition algorithm provided in this application;

[0057] Figure 5 A graph showing the relationship between the number of feature points and the recognition accuracy provided for this application. Detailed Implementation

[0058] The present invention will be further described below with reference to embodiments. The description of the embodiments below is only for the purpose of helping to understand the present invention. It should be noted that those skilled in the art can make several modifications to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0059] Example 1:

[0060] This invention, based on the characteristic that various road surface damages are accompanied by local depressions, and considering image processing techniques at various stages, proposes a road surface damage region detection method based on feature points and deep learning. It uses local gray-level minima in road surface images that meet certain conditions as feature points, and studies image blocks around the neighborhood of these feature points. First, an image block dataset is established, and a convolutional neural network model is trained to extract depth features from the block images. When detecting road surface damage, a certain number of feature points are found on an image window basis, and the trained convolutional neural network is used to extract the depth features of these feature points. Then, the depth features corresponding to all feature points are fused to construct a composite feature vector, which serves as the feature descriptor for the image window. Finally, a support vector machine (SVM) is used to construct a classifier to identify whether the image window contains a damaged region.

[0061] Specifically, the following steps are included:

[0062] Step 1: Acquire road surface images and preprocess them.

[0063] Step 2: Divide the road surface image into several windows, and calculate the local extreme points for each window.

[0064] Step 3: Based on the grayscale distribution of the neighborhood of the extreme point, determine several local extreme points as feature points.

[0065] Step 4: Using local extreme points as the center, extract image blocks in their neighborhoods, establish a block image dataset, and train a CNN block image feature extraction model.

[0066] Step 4 involves building the block image dataset and training the CNN block image feature extraction model, which is part of the modeling phase and not required for road surface detection. However, when applying to road surfaces with different conditions, adding a portion of actual road surface block images to the dataset before training the CNN model will yield better results.

[0067] Step 5: Using the trained CNN feature model, calculate the feature vectors corresponding to each feature point in the image window, perform feature fusion to obtain a composite feature vector; use a support vector machine to classify the composite feature vectors and detect the damage to the road surface.

[0068] Step 1 includes:

[0069] Step 1.1: Acquire the road surface image and apply Gaussian filtering to the road surface image;

[0070] Step 1.2: Perform gamma correction on the road surface image. The formula for gamma correction is:

[0071] I(x,y)=I(x,y) γ

[0072] The formula for determining the value of γ is:

[0073]

[0074] Among them, I P I0 is the peak value of the grayscale distribution before correction, and I0 is the peak value of the grayscale distribution after correction.

[0075] Step 2 includes:

[0076] Step 2.1: Divide the road surface image into road surface image windows of size Nw×Nw. Considering that the road surface image is prone to continuous peaks, first invert the image grayscale, and then search for the maximum value in units of image windows.

[0077] Step 2.2: Define two parameters: the minimum distance P for peak detection. dist And the threshold P for peak detection th To adapt to changes in image brightness, the threshold P is adjusted according to the following formula. th Convert to peak detection amplitude P per window A :

[0078] P A =P th *(G max -G min )+G min

[0079] Among them, G max and G min These are the maximum and minimum grayscale values ​​for the corresponding window;

[0080] Step 2.3: Obtain the first-order difference and the index position of the difference amplitude of 0. Adjacent 0 positions indicate that it is in a horizontal state. Only the middle point is selected as the unique peak point.

[0081] Step 2.4: Obtain the peak points that meet the peak detection criteria. The criteria are that the gradient sign changes and the gray value is greater than the peak detection amplitude P. A ;

[0082] Step 2.5: Use the minimum peak distance P distTo filter out invalid peak points and obtain local extreme points, the specific method is as follows: sort the above peak points according to their amplitude, and extract the peak points one by one. Then, using this point as the center, P... dist All pixels within a neighborhood of radius are considered non-peak points.

[0083] In step 2.5, the remaining points are the extreme points that meet the threshold and distance conditions. The distance between the extreme points is roughly equivalent to the size of the captured image block in the neighborhood of each extreme point. These minima are subsequently referred to as local extrema. Figure 1 As shown, the left side corresponds to all local extreme points.

[0084] Step 3 includes:

[0085] Step 3.1: Determine the grayscale distribution of the neighborhood blocks of the extreme point, mainly considering the average grayscale value and variance. Assume the grayscale value of the pixel in the i-th block is G. i Given (x, y), which has N pixels, the calculation formula is as follows:

[0086]

[0087]

[0088] Among them, E i The mean value reflects the overall brightness around the extreme point; σ i Variance represents the grayscale distribution of pixels in a block; block indicates a region.

[0089] Step 3.2: For the average grayscale value and variance of the neighborhood of an extreme point, critical values ​​are given to perform preliminary screening of local extreme points. Only when all parameters are greater than the critical value is the extreme point retained, filtering out some extreme points that clearly belong to the normal road surface; such as... Figure 1 As shown, the right side corresponds to filtering out local extrema that do not meet the conditions of neighborhood average gray level and variance.

[0090] Step 3.3: Sort the remaining extreme points according to gray level, select a certain number of extreme points as feature points of the window, and extract the neighborhood blocks of the feature points for analysis of the image window.

[0091] This application determines that an extreme point is retained only when all parameters are greater than the critical value, filtering out some extreme points that clearly belong to normal road surfaces, such as... Figure 1 As shown. The remaining extreme points can be sorted by gray level, and a certain number of extreme points can be selected as feature points of the window. The neighborhood blocks of these feature points are then extracted for analysis of the image window.

[0092] In step 3.3, determining a certain number of extreme points within the image window as feature points is crucial for the detection performance. When constructing the image patch dataset, manual methods can be used to determine extreme points located in damaged areas. However, in actual detection, feature points are selected based on the grayscale values ​​of these extreme points.

[0093] Step 4 includes:

[0094] Step 4.1: To train a CNN model capable of extracting features from patch images, a patch image dataset needs to be created. Patch images are extracted from public datasets and actual road surface images that need to be detected. The patch images can be relatively small, for example, 32*32 pixels. See [link to relevant documentation]. Figure 2 .

[0095] Step 4.2: Construct a CNN feature extraction model. The essence of CNN lies in performing convolution, pooling, and other calculations layer by layer on the input image data to extract data features. Using the road surface features automatically extracted by CNN, it is possible to deeply reflect the changes in damaged areas. The convolution operation formula for a single convolution kernel is:

[0096]

[0097] Among them, y i,j The output pixel value represents the convolution, f represents the activation function, and W represents the value of the convolution output pixel. u,v Let X represent the weight in the u-th row and v-th column of the convolution kernel, where U and V are the kernel sizes, b represents the kernel bias, and X represents the weight in the v-th column of the convolution kernel. i,j This represents the element in the i-th row and j-th column of the feature map.

[0098] For example, the basic CNN model architecture parameters used in this application are shown in Table 1. The input layer directly processes the candidate region image, and the determined image block size is 32*32, which basically covers a part of the damaged area. This image size is moderate in terms of computational power and dataset size requirements, and it is easy to adjust the detection model parameters when the road condition changes.

[0099] Table 1: CNN Model Architecture Parameters

[0100]

[0101]

[0102] Furthermore, in step 4.1, the specific method for manually extracting image blocks to build the dataset includes: determining local extreme points in the image window and labeling them with serial numbers in the image, see [link to step 4]. Figure 3 Identify the extreme points located in the damaged area, input their serial numbers to extract image blocks and label them as positive samples; while the blocks corresponding to local extreme points in the normal road surface image are labeled as negative samples.

[0103] In step 4.2, the image blocks are relatively small, which greatly reduces the complexity requirements of the network model. Based on the actual computational requirements, a CNN model suitable for feature extraction of road damage areas was constructed.

[0104] Step 5 includes:

[0105] Step 5.1: The CNN block image feature extraction model outputs feature vectors from the fully connected layer.

[0106] Step 5.2: Combine the feature vectors of all feature points involved in the image window to form a feature vector describing the entire window; from the perspective of multi-feature fusion, it is also possible to improve the accuracy of road surface detection by analyzing the correlation of multiple blocks.

[0107] Step 5.3: Using the composite feature vector as the feature descriptor of the image window, a classifier is constructed using a support vector machine (SVM) for feature recognition to determine whether the image window contains a damaged area. SVM is a common binary classification model, particularly advantageous in nonlinear high-dimensional pattern recognition. Due to its good stability and ease of implementation, SVM is often used to replace the Softmax classifier in convolutional neural networks. This combination yields a recognition model that balances high accuracy and generalization ability, achieving high accuracy even on small datasets.

[0108] Example 2:

[0109] The next step is to conduct experiments on the public dataset. The algorithm flow is as follows: Figure 4 As shown, the specific processes for each part are as follows:

[0110] (1) Search for local minima in the road surface image window, which must meet certain gray-level statistical characteristics, including the minimum gray value of the minima, the mean gray value and variance of the neighborhood block. For comparability, the average gray value of the image window can be used as a reference, and relative gray values ​​are taken.

[0111] (2) Extract neighborhood blocks of each extreme point, consistent with the blocks saved when training the CNN feature extraction model, i.e., each block is 32*32 pixels. Utilize the deep features extracted by the trained CNN model, with each block corresponding to a 64-dimensional feature vector.

[0112] (3) Determine the number of extreme points to be used as feature points, selecting them from deepest to shallowest. Perform feature-level fusion on the feature vectors corresponding to the selected extreme points to obtain a composite feature vector. The feature vectors are fused using a concatenation method. If the number of selected feature points is m, the composite feature vector is an m*64 dimensional vector.

[0113] (4) A classifier was constructed using a Support Vector Machine (SVM) to study the original image dataset. The dataset was divided into a training set and a test set with a 1:1 ratio. The composite feature vectors were input into the SVM to train the SVM classifier, and then the accuracy of the classifier on the test set was tested. The parameters of the SVM were selected as follows: the SVM used an RBF kernel; C was set to 0.9; Tol was set to 1e-3; and Gamma was automatically set by scikit-learn.

[0114] Each image block is convolved and subsampled, then the output feature vectors from the fully connected layer are fused at the feature level to obtain a composite feature vector. This composite feature vector serves as the feature descriptor for the image window. A support vector machine (SVM) is used to construct a classifier for feature recognition, determining whether the image window contains damaged areas. Road surface damage detection is a binary classification problem, where damaged surfaces are considered positive samples, and normal surfaces are considered negative samples. SVM is a common binary classification model, particularly advantageous in nonlinear high-dimensional pattern recognition. Due to its stability and ease of implementation, SVM is often used instead of the Softmax classifier in convolutional neural networks. This combination results in a recognition model that balances high accuracy and generalization ability, achieving high accuracy even on small datasets.

[0115] A classifier using Support Vector Machine (SVM) was constructed to study the original image dataset. The dataset was divided into training and test sets in a 1:1 ratio. Composite feature vectors were input into the SVM to train the classifier, and then the accuracy of the classifier on the test set was tested. The SVM parameters were selected as follows: RBF kernel was used; C was set to 0.9; Tol was set to 1e-3; and Gamma was automatically set by scikit-learn.

[0116] Table 2: Number of Extreme Points and Recognition Accuracy

[0117]

[0118] Processing block images and datasets requires no special computer skills. It can be done on a Windows 10 system using Anaconda3 as the programming environment and Python as the programming language, incorporating image processing tool OpenCV, machine learning module scikit-learn, and deep learning module TensorFlow. The relationship between the number of feature points and recognition accuracy was calculated for the four datasets mentioned above. Starting from a single point, the accuracy improves with the increase of the number of associated feature points. After reaching five points, the recognition accuracy remains essentially unchanged. Moreover, the recognition accuracy of several datasets conforms to this trend, as shown in Table 2. Figure 5To improve overall recognition accuracy, a more comprehensive and targeted block image dataset needs to be established. For detection of the original image, the moving window can be appropriately enlarged, and the number of optimal feature points may also be increased accordingly.

Claims

1. A method for detecting pavement damage based on the fusion of local gray-level extreme points and features, characterized in that, include: Step 1: Acquire a road surface image and preprocess the road surface image; Step 2: Divide the road surface image into several windows, and calculate local extreme points for each window; Step 3: Based on the gray-scale distribution of the neighborhood blocks of the extreme points, determine several local extreme points as feature points; Step 3 includes: Step 3.1: Determine the grayscale distribution of the neighborhood blocks of the extreme point, considering the average grayscale value and variance; i The grayscale value of each pixel in the block is There are N pixels in total, and the calculation formula is as follows: in, The mean value reflects the overall brightness around the extreme point; The variance represents the grayscale distribution of pixels in a block; "block" indicates a region. Step 3.2: For the average gray value and variance of the neighborhood of the extreme point, a critical value is given for preliminary screening of local extreme points; Step 3.3: Sort the remaining extreme points according to gray level, and select a certain number of extreme points as feature points of the window, and extract the neighborhood blocks of the feature points for analysis of the image window. Step 4: Using local extreme points as the center, extract image blocks in their neighborhood, establish a block image dataset, and train a CNN block image feature extraction model. Step 5: Using the trained CNN feature model, calculate the feature vectors corresponding to each feature point in the image window, perform feature fusion to obtain a composite feature vector; use a support vector machine to classify the composite feature vectors and detect the damage to the road surface.

2. The road damage detection method based on local gray-level extreme points and feature fusion according to claim 1, characterized in that, Step 1 includes: Step 1.1: Acquire a road surface image and perform Gaussian filtering on the road surface image; Step 1.2: Perform gamma correction on the road surface image. The formula for gamma correction is: in, The formula for determining the value is: in, It is the peak value of the grayscale distribution before correction. It is the peak value of the corrected grayscale distribution.

3. The pavement damage detection method based on local gray-level extreme points and feature fusion according to claim 2, characterized in that, Step 2 includes: Step 2.1: Divide the road surface image into segments of size [size missing]. For the road surface image window, first invert the image grayscale, and then search for the maximum value in units of the image window; Step 2.2: Define two parameters: minimum distance for peak detection. P dist and the threshold for peak detection P th The threshold is calculated according to the following formula. P th Convert to peak detection amplitude per window P A : in, G max and G min These are the maximum and minimum grayscale values ​​for the corresponding window; Step 2.3: Obtain the first-order difference and the index position of the difference amplitude of 0. Adjacent 0 positions indicate that it is in a horizontal state. Only the middle point is selected as the unique peak point. Step 2.4: Obtain the peak points that meet the peak detection criteria. The criteria are that the gradient sign changes and the gray value is greater than the peak detection amplitude. P A ; Step 2.5: Use the minimum peak distance P dist Invalid peak points are filtered out to obtain local extreme points.

4. The pavement damage detection method based on local gray-level extreme points and feature fusion according to claim 3, characterized in that, In step 2.5, the distance between the local extrema is comparable to the size of the captured image block of the extrema neighborhood.

5. The pavement damage detection method based on local gray-level extreme points and feature fusion according to claim 4, characterized in that, Step 4 includes: Step 4.1: Extract block images from public datasets and actual road surface images that need to be detected, and establish a block image dataset; Step 4.2, Construction The essence of feature extraction models, specifically CNNs, lies in performing convolution and pooling operations layer by layer on the input image data to extract data features; the convolution operation formula for a single convolution kernel is: in, This represents the pixel value output by the convolution. This represents the activation function. Indicates the convolution kernel number 1 Line number Column weights, The kernel size is the convolution kernel size. This indicates the bias of the convolution kernel. The first feature map represents the first feature map. Line number Column elements.

6. The pavement damage detection method based on local gray-level extreme points and feature fusion according to claim 5, characterized in that, In step 4.1, the specific method for manually extracting image blocks to establish a dataset includes: determining local extreme points in the image window and labeling them with serial numbers in the image; identifying extreme points located in the damaged area, inputting their serial numbers to extract image blocks and labeling them as positive samples; while the blocks corresponding to local extreme points in the normal road surface image are labeled as negative samples.

7. The pavement damage detection method based on local gray-level extreme points and feature fusion according to claim 6, characterized in that, Step 5 includes: Step 5.1: The CNN block image feature extraction model outputs feature vectors from the fully connected layer; Step 5.2: Combine the feature vectors of all feature points involved in the image window to form a feature vector that describes the entire window; Step 5.3: Use the composite feature vector as the feature descriptor of the image window, and use a support vector machine to construct a classifier for feature recognition to determine whether the image window contains a damaged area.

8. A road surface damage detection device based on the fusion of local gray-level extreme points and features, characterized in that, The method for performing pavement damage detection based on local gray-level extreme points and feature fusion as described in claim 1 includes: The acquisition module is used to acquire road surface images and preprocess the road surface images; The segmentation module is used to divide the road image into several windows, and calculate local extreme points for each window as a unit. The determination module is used to determine several local extreme points as feature points based on the gray-scale distribution of the neighborhood blocks of extreme points; The training module is used to extract image blocks in the neighborhood of local extreme points, build a block image dataset, and train a CNN block image feature extraction model. The detection module uses a trained CNN feature model to calculate the feature vectors corresponding to each feature point in the image window, performs feature fusion to obtain a composite feature vector, and uses a support vector machine to classify the composite feature vector to detect road surface damage.

9. A computer storage medium, characterized in that, The computer storage medium stores a computer program; when the computer program is run on the computer, it causes the computer to execute the road damage detection method based on local gray-level extreme points and feature fusion as described in claim 1.

Images