Intelligent classification and feature extraction method and system for multiple types of pavement cracks

Abstract

The application provides a pavement multi-type crack intelligent classification and feature extraction method and system, relates to the pavement detection technical field, and comprises the following steps: inputting a deformable convolution network by extracting a crack skeleton and combining geometric features and texture features, and dynamically adjusting the spatial scale of feature extraction; learning meta-knowledge by adopting a few-shot learning strategy, modeling a category evolution mode based on a Markov decision process; projecting a multi-dimensional feature vector and a sample pair to a multi-level feature space by a contrast learning strategy, and completing classification in combination with a confidence evaluation network. The application can improve crack classification precision and processing efficiency, and reduce the dependence on sample data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road surface detection, and in particular to a method and system for intelligent classification and feature extraction of multiple types of road surface cracks. BACKGROUND

[0002] Road cracks are one of the common forms of road damage, which pose a serious threat to traffic safety and driving comfort. Intelligent classification and feature extraction of road cracks are key links in road detection and maintenance, and are of great significance for the preventive maintenance of road facilities. Traditional road crack detection mainly relies on manual inspection, which is inefficient and the results are easily affected by subjective factors. With the development of computer vision technology, image processing-based automatic detection and classification of road cracks has gradually become a research hotspot.

[0003] Existing crack feature extraction methods cannot effectively handle the complex morphological features of multiple types of cracks, especially for complex cracks with longitudinal and transverse intersections, the accuracy of extracting their geometric and texture features is insufficient, which limits the subsequent classification accuracy. Most road crack classification algorithms rely on a large number of labeled samples for training, and when facing new types of cracks or sample-poor crack categories, the classification performance decreases significantly, making it difficult to adapt to changes in data distribution in actual applications. Existing methods lack multi-scale analysis capability for crack features, making it difficult to capture both local details and global structure information of cracks, and also lack confidence evaluation mechanisms for classification results, which cannot provide reliability guarantee for subsequent decision-making. SUMMARY

[0004] The embodiments of the present application provide a method and system for intelligent classification and feature extraction of multiple types of road surface cracks, which can solve the problems in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a method for intelligent classification and feature extraction of multiple types of road surface cracks, comprising:

[0006] extracting a crack skeleton from the road surface to be detected to obtain a crack skeleton line image;

[0007] based on the crack skeleton line image, extracting geometric features of the crack through a region growing algorithm and a histogram of oriented gradients, and extracting texture features of the crack using a local binary pattern and a Gabor filter bank; inputting the geometric features and the texture features into a deformable convolution network, the deformable convolution network dynamically adjusting the spatial scale of feature extraction, extracting local feature vectors and global context feature vectors of the crack skeleton line image in parallel at different spatial scales, and packaging them into a multi-dimensional feature vector;

[0008] The few-shot learning strategy is used to learn the class-independent meta-knowledge from the known crack categories, a Markov decision process is constructed based on the meta-knowledge to model the category evolution mode, and a reinforcement learning method is combined to adaptively update the category prototype of each crack; the positive and negative sample pairs are constructed based on the category prototype, and the feature expression quality score is calculated, the multi-dimensional feature vector and the positive and negative sample pairs with the feature expression quality score higher than a preset discrimination threshold are projected to a plurality of feature hierarchical spaces to obtain multi-level feature representations through a contrast learning strategy; the feature matching distance of the multi-level feature representations is calculated to perform preliminary classification, and the classification confidence is evaluated in combination with a confidence evaluation network to realize intelligent classification of the pavement cracks.

[0009] Based on the crack skeleton line image, the geometric features of the crack are extracted through a region growing algorithm and a histogram of oriented gradients, and the texture features of the crack are extracted using a local binary pattern and a Gabor filter bank, including:

[0010] The crack skeleton line image is divided into a plurality of local regions, a seed point discrimination criterion is constructed in the local regions, a seed point is determined based on the seed point discrimination criterion, and a region growing is performed on the seed point to obtain a crack region; in the crack region, a histogram of orientation is established based on the gradient amplitude and the gradient direction of each pixel point, and the histogram of orientation is taken as the geometric feature of the crack region;

[0011] A feature extraction window of different scales is constructed for the crack region, a neighborhood sampling point is obtained by performing uniform sampling with the center pixel point of the feature extraction window as the center, a binary coding sequence is constructed according to the neighborhood sampling point and the center pixel point, and the number of value jumps in the binary coding sequence is taken as a local binary pattern feature value; the Gabor filter includes a real filter kernel and an imaginary filter kernel, and the crack region is respectively convolved with the real filter kernel and the imaginary filter kernel to extract Gabor filter features; the local binary pattern feature value and the Gabor filter features are combined at different scales to obtain the texture features of the crack region.

[0012] The deformable convolution network dynamically adjusts the spatial scale of feature extraction, and parallelly extracts the local feature vector and the global context feature vector of the crack skeleton line image at different spatial scales, and encapsulates them into a multi-dimensional feature vector, including:

[0013] The deformable convolution network receives the geometric features and the texture features, calculates a two-dimensional spatial offset, dynamically adjusts the sampling position of a standard convolution kernel based on the two-dimensional spatial offset, performs bilinear interpolation and multiple variable sampling rate downsampling at the adjusted sampling position, and obtains a feature response map of different spatial scales;

[0014] A parallel deformable convolution branch is constructed for the feature response map of each spatial scale, and the deformation amount is adaptively adjusted through back propagation of a loss function; an attention mechanism is used to balance and combine the output features of different deformable convolution branches to obtain a local feature map at each spatial scale;

[0015] The local feature maps are uniformly upsampled to the same spatial resolution and element-wise superimposed to obtain a multi-scale fusion feature map, and feature pooling operations are performed on the local feature maps and the multi-scale fusion feature map to obtain the local feature vector and the global context feature vector of the crack skeleton line image; after concatenating the local feature vector and the global context feature vector in the channel dimension, dimension reduction is performed through a projection matrix to obtain the multi-dimensional feature vector.

[0016] A few-shot learning strategy is used to learn class-agnostic meta-knowledge from known crack classes, a Markov decision process is constructed based on the meta-knowledge to model the class evolution pattern, and a reinforcement learning method is used to adaptively update the class prototype of each crack, including:

[0017] Support sets and query sets are extracted from data of known crack classes, features of samples in the support sets are extracted, and a class prototype of each class is calculated; distance metrics between samples in the query sets and the class prototypes of each class are calculated, probability distributions of the samples in the query sets belonging to each class are calculated based on the distance metrics, and class-agnostic meta-knowledge is obtained;

[0018] The state space is represented as a combination of the class prototypes of each class and their uncertainties, the action space is represented as the update step and direction of the class prototypes, the class prototype of the next state is calculated based on the state space and the action space, and the Markov decision process is obtained;

[0019] A target Q network and an evaluation Q network are constructed, which respectively output target Q values and evaluation Q values of each update action; a reward function that comprehensively considers classification accuracy and class distribution changes is constructed, an experience replay pool is constructed based on state transition samples in the Markov decision process, a training batch is sampled from the experience replay pool, and parameters of the target Q network and the evaluation Q network are alternately updated based on the reward function; the optimal update action is selected through the target Q value and the evaluation Q value, and the class prototype of each crack is updated.

[0020] Based on the class prototype, positive and negative sample pairs are constructed and feature expression quality scores are calculated, and through a contrast learning strategy, the multi-dimensional feature vector and the positive and negative sample pairs with feature expression quality scores higher than a preset discrimination threshold are projected to multiple feature level spaces to obtain multi-level feature representations, including:

[0021] Cosine similarity is used to measure feature distance, a preset number of samples closest to the feature distance are selected from a sample set of the same category as the category prototype to construct a positive sample pair, and a preset number of samples closest to the feature distance are selected from a sample set of other categories to construct a negative sample pair; Intra-class compactness score and inter-class separability score are calculated based on the positive and negative sample pairs, and the feature expression quality score is obtained by combining them;

[0022] A plurality of projection matrices with decreasing dimensions are constructed, and the multi-dimensional feature vector and the positive and negative sample pairs with feature expression quality scores higher than a preset discrimination threshold are projected to a plurality of feature level spaces through the plurality of projection matrices; In the feature level space, the cosine similarity of the multi-dimensional feature vector and the positive and negative sample pairs is calculated to obtain a contrast loss, and the Euclidean distance between adjacent feature levels is calculated to obtain a consistency loss, and the multi-level feature representation is obtained by minimizing the contrast loss and the consistency loss.

[0023] Preliminary classification is performed by calculating the feature matching distance of the multi-level feature representation, and the classification confidence is evaluated by combining the confidence evaluation network to realize intelligent classification of the road cracks, including:

[0024] The Euclidean distance and the cosine distance of each layer of the multi-level feature representation are calculated and combined to obtain the feature matching distance; based on the feature matching distance, a k-nearest neighbor classification strategy is used to obtain a preliminary classification result;

[0025] The confidence evaluation network performs probability encoding on the multi-level feature representation to obtain a feature probability distribution, obtains a plurality of network parameter samples by Monte Carlo sampling, inputs the feature probability distribution into a forward propagation network constructed by the network parameter samples, obtains a predicted probability distribution and calculates the information entropy of the predicted probability distribution; based on a soft voting mechanism, a plurality of candidate categories in the preliminary classification result are voted, a confidence factor is set according to the voting result, and the information entropy is scaled by the confidence factor to obtain the classification confidence;

[0026] When the classification confidence is greater than a preset reliability threshold, it is determined as correct classification, and intelligent classification of the road cracks is realized.

[0027] The second aspect of the embodiment of the application provides a road multi-type crack intelligent classification and feature extraction system, including:

[0028] A first unit is configured to extract a crack skeleton from a road surface to be detected to obtain a crack skeleton line image;

[0029] The second unit is configured to extract geometric features of the cracks by using a region growing algorithm and a histogram of oriented gradients based on the crack skeleton line image, extract texture features of the cracks by using a local binary pattern and a Gabor filter bank, input the geometric features and the texture features into a deformable convolution network, and dynamically adjust a spatial scale of feature extraction by the deformable convolution network.

[0030] The third unit is configured to learn class-agnostic meta-knowledge from known crack classes by using a few-shot learning strategy, construct a Markov decision process based on the meta-knowledge to model a class evolution mode, and adaptively update class prototypes of the cracks by using a reinforcement learning method.

[0031] A third aspect of the embodiments of the present application,

[0032] An electronic device is provided, comprising:

[0033] a processor;

[0034] a memory for storing processor-executable instructions;

[0035] The processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0036] A fourth aspect of the embodiments of the present application,

[0037] A computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0038] The beneficial effects of the present application are as follows:

[0039] The intelligent classification and feature extraction method for multiple types of road cracks can effectively identify various complex road cracks by extracting crack skeletons and performing multi-dimensional analysis on geometric features and texture features, and greatly improves the accuracy and stability of crack classification, thereby providing a reliable basis for road maintenance.

[0040] The method dynamically adjusts the spatial scale of feature extraction by using a deformable convolution network, combines a multi-scale parallel feature extraction mechanism, can capture local details and global structure information of cracks at the same time, effectively solves the adaptability problem of traditional methods in processing cracks of different scales and shapes, and improves the adaptability of the system to complex road surface environment.

[0041] By means of few-shot learning and Markov decision process modeling of category evolution, in combination with a contrast learning and confidence evaluation mechanism, the method can effectively process the identification problem of the fuzzy boundary between crack categories and unknown category cracks, reduce the dependence on large-scale labeled data, and improve the reliability and generalization ability of the system in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A flowchart of a pavement multi-type crack intelligent classification and feature extraction method according to an embodiment of the present application is shown in

[0043] Figure 2 A flowchart of positive and negative sample pair construction and multi-level feature representation learning is shown in DETAILED DESCRIPTION

[0044] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0045] The technical scheme of the present application will be described in detail in the following specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0046] Figure 1 A flowchart of a pavement multi-type crack intelligent classification and feature extraction method according to an embodiment of the present application is shown in Figure 1 The method comprises:

[0047] Extracting a crack skeleton from the road surface to be detected to obtain a crack skeleton line image;

[0048] Based on the crack skeleton line image, geometric features of the crack are extracted by a region growing algorithm and a histogram of oriented gradients, and texture features of the crack are extracted by a local binary pattern and a Gabor filter bank; the geometric features and the texture features are input into a deformable convolution network, the deformable convolution network dynamically adjusts a spatial scale of feature extraction, local feature vectors and global context feature vectors of the crack skeleton line image are extracted under different spatial scales in parallel, and are packaged as a multi-dimensional feature vector;

[0049] A few-shot learning strategy is used to learn class-independent meta-knowledge from known crack classes, a Markov decision process is constructed based on the meta-knowledge to model a class evolution mode, and a reinforcement learning method is combined to adaptively update class prototypes of each crack; positive and negative sample pairs are constructed based on the class prototypes, and a feature expression quality score is calculated, the multi-dimensional feature vector and the positive and negative sample pairs with the feature expression quality score higher than a preset discrimination threshold are projected to multiple feature level spaces to obtain multi-level feature representations through a contrast learning strategy; preliminary classification is performed by calculating feature matching distances of the multi-level feature representations, and classification confidence is evaluated by combining a confidence evaluation network, thereby realizing intelligent classification of the pavement crack.

[0050] In an alternative embodiment, based on the crack skeleton line image, geometric features of the crack are extracted by a region growing algorithm and a histogram of oriented gradients, and texture features of the crack are extracted by a local binary pattern and a Gabor filter bank, which includes:

[0051] The crack skeleton line image is divided into multiple local regions, a seed point discrimination criterion is constructed in the local regions, a seed point is determined based on the seed point discrimination criterion, and a region growing is performed on the seed point to obtain a crack region; in the crack region, a histogram of orientation is established based on a gradient amplitude and a gradient direction of each pixel point, and the histogram of orientation is taken as the geometric features of the crack region;

[0052] Feature extraction windows of different scales are constructed for the crack region, a neighborhood sampling point is obtained by performing uniform sampling with a center pixel point of the feature extraction window as the center, a binary coding sequence is constructed according to the neighborhood sampling point and the center pixel point, and a number of jumps of values in the binary coding sequence is taken as a local binary pattern feature value; the Gabor filter includes a real part filter kernel and an imaginary part filter kernel, Gabor filter features are extracted by performing convolution operations on the crack region and the real part filter kernel and the imaginary part filter kernel respectively; the local binary pattern feature value and the Gabor filter features are combined under different scales to obtain the texture features of the crack region.

[0053] The crack skeleton line image is divided into multiple local regions, and a fixed size grid can be used to divide the skeleton line image, for example, an 800x600 pixel image is divided into 16x12 grids, and each grid has a size of 50x50 pixels. A seed point discrimination criterion is constructed in each local region, which includes three factors of pixel gray value, local variance, and region connectivity. For the gray value, a threshold of 50 is set, and when the gray value of a pixel point is less than the threshold, it is considered to be part of the crack region. For the local variance, the pixel gray variance in a 5x5 neighborhood is calculated, and when the variance is greater than 20, it indicates that there is a significant gray change in the region. For the region connectivity, it is detected whether there are at least three pixel points in the 8-neighborhood that satisfy the previous two conditions. When a pixel point satisfies the above three conditions at the same time, it is determined as a seed point.

[0054] A region growing algorithm is performed on the determined seed points, and the specific growth criteria are set as follows: starting from the seed point, check each pixel point in its 8-neighborhood, if the gray difference between the neighborhood pixel and the pixel in the current region is less than the threshold 15, then add the neighborhood pixel to the crack region, and repeat the process until no new pixel point is added. For example, for a seed point coordinate (100, 150) with a gray value of 30, the neighborhood point (101, 150) is checked, with a gray value of 35, the difference is 5, which is less than the threshold 15, so (101, 150) is added to the crack region; while the neighborhood point (100, 151) has a gray value of 60, the difference is 30, which is greater than the threshold 15, and is not added to the crack region.

[0055] In the obtained crack region, the Sobel operator is used to calculate the gradient amplitude of each pixel point, and the horizontal and vertical gradients are denoted as Gx and Gy respectively. For each pixel point (i, j) in the crack region, the gradient amplitude is calculated as the square root of the sum of (Gx(i, j)) 2 and (Gy(i, j)) 2 , and the gradient direction is the arctangent value of Gy(i, j) and Gx(i, j). The gradient direction is divided into 8 bins, ranging from 0° to 360°, and each bin covers 45°. The gradient directions of all pixel points in the crack region are counted to form an 8-dimensional direction histogram, and each dimension represents the number of pixel points in the corresponding direction interval. To eliminate the influence of crack size on the feature, the histogram is normalized so that the sum of each dimension is 1. For example, for a crack region, the 8-dimensional direction histogram obtained by statistics is [120, 245, 310, 189, 95, 78, 62, 101], and after normalization, it becomes [0.10, 0.20, 0.26, 0.16, 0.08, 0.07, 0.05, 0.08]. This normalized direction histogram is the geometric feature of the crack.

[0056] Three scale feature extraction windows are constructed for the crack region, which are 7x7, 9x9 and 11x11 pixels. A center pixel point of the feature extraction window is taken as the center, and P neighborhood sampling points are obtained by performing uniform sampling on a circle with a radius of R. In the embodiment, P=8, and R is 1, 2 and 3 pixels respectively, corresponding to three different scales. For example, for the center pixel point (200, 300), 8 points are uniformly sampled on a circle with a radius of R=2, and neighborhood sampling points with coordinates (202, 300), (201, 302), (200, 302), (198, 302), (198, 300), (198, 298), (200, 298) and (202, 298) are obtained.

[0057] For each neighborhood sampling point, if the gray value is greater than or equal to the gray value of the center pixel point, the value is coded as 1, otherwise the value is coded as 0. For example, the gray value of the center pixel point is 100, and the gray values of the 8 neighborhood sampling points are 105, 98, 120, 85, 95, 110, 115 and 90 respectively. The corresponding binary coding sequence is 10101110. The number of transitions in the binary coding sequence, i.e. the number of times of changing from 0 to 1 or from 1 to 0, is calculated, which is 5 in this example. The number of transitions is taken as the local binary pattern feature value, and three local binary pattern feature values are calculated under three different scales.

[0058] The Gabor filter set contains multiple filters with different directions and scales. In the embodiment, 5 directions (0°, 36°, 72°, 108°, 144°) and 4 scales are set, and a total of 20 filters are obtained. Each filter contains a real filter kernel and an imaginary filter kernel, and the size of each filter kernel is 11x11 pixels. The crack region is respectively convolved with the real filter kernel and the imaginary filter kernel to obtain the real response and the imaginary response, and the Gabor filter feature is the square root of the sum of the square of the real response and the square of the imaginary response. For example, for the Gabor filter of the first direction and the first scale, the real response of a pixel point (150, 200) in the crack region is 25.6, and the imaginary response is -12.8. The Gabor filter feature corresponding to the pixel point is 28.7. The responses of all pixel points in the crack region are calculated for the 20 filters, and then the mean value and the standard deviation are calculated to obtain a 40-dimensional Gabor filter feature vector.

[0059] The local binary pattern feature values and the Gabor filter features under different scales are combined. In the embodiment, the local binary pattern feature values of the three scales are spliced with the 40-dimensional Gabor filter feature vector to obtain a 43-dimensional texture feature vector.

[0060] In an alternative embodiment, the deformable convolutional network dynamically adjusts the spatial scale of feature extraction, extracts local feature vectors and global context feature vectors of the crack skeleton line image at different spatial scales in parallel, and encapsulates them into a multi-dimensional feature vector, comprising:

[0061] The deformable convolutional network receives the geometric features and the texture features, calculates a two-dimensional spatial offset, dynamically adjusts the sampling position of a standard convolution kernel based on the two-dimensional spatial offset, performs bilinear interpolation and multiple variable sampling rate downsampling at the adjusted sampling position, and obtains feature response maps at different spatial scales;

[0062] A parallel deformable convolution branch is constructed for the feature response map at each spatial scale, which adaptively adjusts the deformation amount through backpropagation of a loss function; an attention mechanism is used to balance and combine the output features of different deformable convolution branches, obtaining a local feature map at each spatial scale;

[0063] The local feature maps are uniformly upsampled to the same spatial resolution and element superimposed to obtain a multi-scale fusion feature map, and feature pooling operations are performed on the local feature maps and the multi-scale fusion feature map to obtain the local feature vectors and the global context feature vectors of the crack skeleton line image; after concatenating the local feature vectors and the global context feature vectors in the channel dimension, dimension reduction is performed through a projection matrix to obtain the multi-dimensional feature vector.

[0064] Geometric features and texture features are taken as inputs of the deformable convolutional network, and the network calculates a two-dimensional spatial offset through convolutional layers and fully connected layers. The offset is represented as Δx and Δy, representing the position adjustment values in the horizontal and vertical directions, respectively. For each sampling position (x0, y0) in the standard convolution kernel, the offset is adjusted to a new sampling position (x0+Δx, y0+Δy). Since the adjusted sampling position usually does not fall on an integer pixel point, bilinear interpolation is performed at these positions to obtain accurate feature values.

[0065] For the adjusted sampling position (x0+Δx, y0+Δy), find its four surrounding integer pixel points (⌊x0+Δx⌋, ⌊y0+Δy⌋), (⌈x0+Δx⌉, ⌊y0+Δy⌋), (⌊x0+Δx⌋, ⌈y0+Δy⌉), (⌈x0+Δx⌉, ⌈y0+Δy⌉), calculate the weights according to the relative positions, and obtain the feature value of this position by weighted average. In addition, the input feature map is downsampled by multiple variable sampling rates, and the sampling rates are set to 1 / 2, 1 / 4 and 1 / 8, respectively, to generate three feature response maps at different spatial scales, which are 1 / 2, 1 / 4 and 1 / 8 of the original size, respectively.

[0066] Take the feature response graph with the original size of 1 / 4 as an example, a parallel deformable convolution branch is constructed for the feature response graph of each spatial scale, which contains three layers of deformable convolution layers, each layer has a convolution kernel size of 3x3, a channel number of 64, a step of 1, a padding of 1, and a ReLU activation function. Each layer of deformable convolution layer will generate its own offset, which is adaptively adjusted through the back propagation of the loss function. The loss function includes classification loss and regression loss, the classification loss uses cross-entropy loss, the regression loss uses L1 loss, and the total loss is the weighted sum of the two, with a weight ratio of 1:0.5.

[0067] In each deformable convolution branch, the channel attention mechanism is used to balance the output features of different branches. The channel attention mechanism compresses the feature map into a channel descriptor through global average pooling, generates channel weights through two fully connected layers, and applies the channel weights to the original feature map. The combination of output features of each branch adopts the weighted sum method, and the weights are generated by the attention module, with a value range of 0 to 1, and the sum of all weights is 1. For example, for the output features F1, F2, F3 of three branches, the final local feature map F_local = w1xF1 + w2xF2 + w3xF3, where w1, w2, w3 are the weights of each branch, and the optimal values are determined through experiments as 0.3, 0.4, and 0.3 respectively.

[0068] Bilinear interpolation is used to uniformly upsample the local feature maps at different scales to the same spatial resolution. For example, for a feature map with a size of H / 4xW / 4, upsample it to a feature map with a size of HxW. The upsampled feature map is element-wise superimposed, i.e. the feature values at corresponding positions are directly added, to obtain a multi-scale fusion feature map.

[0069] Feature pooling operations are performed on the local feature map and the multi-scale fusion feature map, including global average pooling and global maximum pooling. The global average pooling calculates the average value of all spatial positions in the feature map, and the global maximum pooling takes the maximum value. The two pooling results are fused in a 1:1 ratio to obtain a fixed-dimensional feature vector. For a 64-channel feature map, a 64-dimensional feature vector is obtained after pooling. The local feature vector is obtained from the local feature map, and the global context feature vector is obtained from the multi-scale fusion feature map.

[0070] The local feature vector and the global context feature vector are concatenated in the channel dimension to form a 128-dimensional feature vector. In order to reduce the computational complexity and extract more discriminative features, a dimension reduction transformation is performed through a projection matrix with a dimension of 128x64, and finally a 64-dimensional multi-dimensional feature vector is obtained. This feature vector contains the local detail information and global structure information of the crack skeleton line.

[0071] In an optional implementation, meta-knowledge independent of the classes is learned from known crack classes through a few-shot learning strategy, a Markov decision process is constructed based on the meta-knowledge to model the evolution mode of the classes, and a reinforcement learning method is combined to adaptively update the class prototypes of the cracks, including:

[0072] A support set and a query set are extracted from data of known crack classes, features of samples in the support set are extracted, and a class prototype of each class is calculated; a distance measure between samples in the query set and the class prototypes of the classes is calculated, a probability distribution of the samples in the query set belonging to each class is calculated based on the distance measure, and meta-knowledge independent of the classes is obtained;

[0073] A state space is represented as a combination of the class prototypes of the classes and uncertainty thereof, an action space is represented as an update step and an update direction of the class prototypes, the class prototype of a next state is calculated based on the state space and the action space, and the Markov decision process is obtained;

[0074] A target Q network and an evaluation Q network are constructed, the target Q network and the evaluation Q network respectively output a target Q value and an evaluation Q value of each update action; a reward function that comprehensively considers classification accuracy and changes in class distribution is constructed, an experience replay pool is constructed based on state transition samples in the Markov decision process, a training batch is sampled from the experience replay pool, parameters of the target Q network and the evaluation Q network are alternately updated based on the reward function; and an optimal update action is selected through the target Q value and the evaluation Q value, and the class prototype of each crack is updated.

[0075] A support set and a query set are constructed by randomly sampling from a data set of known crack classes, for each sample in the support set, a pre-trained convolutional neural network is used to extract a feature vector, in the feature extraction process, the input crack image is preprocessed, including image normalization, cropping, data augmentation and other operations, taking concrete crack data as an example, the input image size is 224x224 pixels, and a 512-dimensional feature vector is obtained after the feature extraction network, for each known crack class, a class prototype is constructed by calculating the average value of the feature vectors of the support set samples of the class. For example, for three classes of longitudinal cracks, transverse cracks and network cracks, each class contains 5 samples in the support set, and three 512-dimensional class prototype vectors are obtained through feature extraction and average calculation.

[0076] Euclidean distance is used as the basic measurement standard to calculate the distance between the query set samples and the prototype of each category. The smaller the distance value, the higher the similarity of the sample to the category. In order to convert the distance into a probability distribution, the softmax function is used for normalization processing. In actual operation, the temperature parameter τ is introduced to control the smoothness of the probability distribution, and the value of τ is set to 0.5. In this way, for each sample in the query set, the probability distribution of belonging to each category can be obtained, forming category-independent meta-knowledge.

[0077] The uncertainty of the category prototype is quantified by calculating the variance between the feature of the category support set sample and the category prototype. The state space is represented as the combination of the current category prototype and its uncertainty. For each category, the state is represented as a 513-dimensional vector, including a 512-dimensional category prototype vector and a 1-dimensional uncertainty measure. The combination of the update step and the update direction of the category prototype forms the action space. The update step takes values in the range of [0.01, 0.05, 0.1], and the update direction is obtained by uniformly sampling the feature space, with a total of 8 main directions. By executing a specific action in the state space, the updated category prototype can be obtained, realizing the state transition process.

[0078] The target Q network and the evaluation Q network are constructed, both networks have the same structure, including three fully connected layers, the hidden layer dimension is 256, the network input is the current state and action, and the output is the Q value estimation corresponding to the state-action pair. The evaluation Q network is responsible for real-time evaluation of the current policy, and the target Q network provides a stable learning target. The parameters of the target Q network are updated every 100 steps, and the update method adopts a soft update mechanism with an update rate of 0.01.

[0079] The reward function design considers the classification accuracy and the change of category distribution. The classification accuracy reward is based on the classification performance of the updated category prototype. For every 1% improvement in the current batch classification accuracy, the reward increases by 0.05. The category distribution change reward is measured by calculating the Euclidean distance between the category prototypes before and after the update, encouraging reasonable category prototype movement. In addition, to prevent over-update, when the category prototype moves too much (distance exceeds threshold 1.5), a negative reward of -0.2 is given.

[0080] The experience replay pool is used to store state transition samples, with a capacity of 10000. Each sample contains the current state, the executed action, the obtained reward, the next state, and a flag indicating whether it is a terminal state. During training, 128 samples are randomly sampled from the experience replay pool to form a training batch each time. The evaluation Q network parameters are updated by minimizing the mean square error between the evaluation Q value and the target Q value, using the Adam optimizer with a learning rate of 0.001.

[0081] In the practical application stage, according to the current state, the adaptive update of the category prototype is realized by selecting the action with the maximum Q value through the evaluation of the Q network, and the ε-greedy strategy is used to balance exploration and utilization, and the initial ε value is set to 0.9, and is linearly decayed to 0.1 with the training. In this way, the crack category prototype can be adaptively adjusted according to the new sample information, and the recognition ability of the unseen category crack can be improved.

[0082] In an optional embodiment, the positive and negative sample pairs are constructed based on the category prototype, and the feature expression quality score is calculated, and the multi-dimensional feature vector and the positive and negative sample pairs with the feature expression quality score higher than the preset discrimination threshold are projected to multiple feature hierarchical spaces by a contrast learning strategy to obtain multi-level feature representations, including:

[0083] The cosine similarity is used to measure the feature distance, a preset number of samples with the closest feature distance are selected from the sample set of the same category as the category prototype to construct a positive sample pair, and a preset number of samples with the closest feature distance are selected from the sample set of other categories to construct a negative sample pair; the intra-class compactness score and the inter-class separability score are calculated based on the positive and negative sample pairs to obtain the feature expression quality score;

[0084] A plurality of projection matrices with decreasing dimensions are constructed, and the multi-dimensional feature vector and the positive and negative sample pairs with the feature expression quality score higher than the preset discrimination threshold are projected to multiple feature hierarchical spaces by the plurality of projection matrices; in the feature hierarchical space, the cosine similarity of the multi-dimensional feature vector and the positive and negative sample pairs is calculated to obtain a contrast loss, and the Euclidean distance between adjacent feature hierarchical spaces is calculated to obtain a consistency loss, and the multi-level feature representations are obtained by minimizing the contrast loss and the consistency loss.

[0085] As shown in Figure 2 , the method comprises:

[0086] The sample features in the image dataset are obtained, which are usually multi-dimensional feature vectors extracted by a pre-trained convolutional neural network, and for each category in the dataset, the mean of all sample feature vectors of the category is calculated as a category prototype. For example, for a dataset containing 10 categories, each category has 100 images, and the dimension of the feature vector extracted from each image is 512, 10 512-dimensional category prototype vectors will be obtained.

[0087] The construction of the positive sample pair is based on the cosine similarity measure of feature distance. The sample with the closest feature distance is selected from the sample set of the same class as the class prototype. Specifically, if the number of pre-selected samples is 5, for the prototype of class A, the cosine similarity between all samples in class A and the prototype is calculated, and the top 5 samples with the highest similarity are selected as positive samples. The cosine similarity is calculated as the dot product of two vectors divided by the product of their norms, with a value range of [-1, 1], and a larger value indicates a higher similarity between the two vectors. The construction of the negative sample pair is also based on the cosine similarity, but the sample with the closest feature distance is selected from the samples of other classes, for example, for the prototype of class A, the cosine similarity between all samples of non-A classes and the prototype of class A is calculated, and the top 5 samples with the highest similarity are selected as negative samples.

[0088] The feature expression quality score includes two parts: intra-class compactness score and inter-class separability score. The intra-class compactness score measures the degree of aggregation of sample features within the same class, which can be obtained by calculating the average cosine similarity between the class prototype and its positive sample pair. For example, if the cosine similarity between a class prototype and its 5 positive samples is 0.92, 0.90, 0.89, 0.87, and 0.85, respectively, the intra-class compactness score is 0.886. The inter-class separability score measures the degree of separation between sample features of different classes, which can be obtained by calculating the negative value of the average cosine similarity between the class prototype and its negative sample pair. For example, if the cosine similarity between a class prototype and its 5 negative samples is 0.65, 0.63, 0.62, 0.60, and 0.58, respectively, the inter-class separability score is -0.616. The feature expression quality score is obtained by weighted combination of the intra-class compactness score and the inter-class separability score, for example, it can be simply added to obtain 0.27, or it can be weighted combined by setting weights such as 0.6 and 0.4 to obtain 0.271.

[0089] A pre-set discrimination threshold is set, for example, 0.2. Only the positive and negative sample pairs with a feature expression quality score higher than the threshold will be used for subsequent contrast learning, which can filter out poor quality sample pairs and improve learning efficiency and model performance.

[0090] Multiple projection matrices with decreasing dimensions are constructed to project the feature vector into different dimensional feature spaces, for example, if the original feature dimension is 512, three projection matrices can be constructed to project the feature into 256-dimensional, 128-dimensional, and 64-dimensional feature spaces. These projection matrices can be implemented through neural network layers, such as fully connected layers plus activation functions.

[0091] The original multi-dimensional feature vector and the positive and negative sample pairs with feature expression quality scores higher than a preset discrimination threshold are projected to multiple feature hierarchy spaces through the projection matrices. In each feature hierarchy space, the cosine similarity between the feature vector and the positive and negative sample pairs is calculated to obtain a contrast loss. For example, in a 256-dimensional feature space, if the cosine similarity between a feature vector and its positive sample is 0.85 and the cosine similarity between the feature vector and its negative sample is 0.45, the contrast loss can be defined as the negative value of the difference between the positive sample similarity and the negative sample similarity, that is, -(0.85-0.45)=-0.4.

[0092] In addition, in order to maintain consistency between different feature hierarchies, the Euclidean distance between adjacent feature hierarchies is also calculated as a consistency loss. For example, the Euclidean distance of the same sample in the 256-dimensional and 128-dimensional feature spaces is calculated, and if the distance value is 1.2, the consistency loss is 1.2.

[0093] By minimizing the weighted sum of the contrast loss and the consistency loss, a multi-level feature representation is obtained. The weight parameters can be set, for example, the contrast loss weight is 0.7 and the consistency loss weight is 0.3. In the training process, an optimization algorithm such as stochastic gradient descent can be used to minimize the total loss, and the parameters of the projection matrix are constantly updated to obtain a better multi-level feature representation.

[0094] In an optional implementation, the feature matching distance of the multi-level feature representation is calculated to perform preliminary classification, and the classification confidence is evaluated in combination with a confidence evaluation network to realize intelligent classification of the road cracks.

[0095] The Euclidean distance and the cosine distance of each layer of the multi-level feature representation are calculated and combined to obtain the feature matching distance. Based on the feature matching distance, a k-nearest neighbor classification strategy is used to obtain a preliminary classification result.

[0096] The confidence evaluation network performs probability encoding on the multi-level feature representation to obtain a feature probability distribution. Multiple sets of network parameter samples are obtained by Monte Carlo sampling. The feature probability distribution is input into a forward propagation network constructed by the network parameter samples to obtain a predicted probability distribution and calculate the information entropy of the predicted probability distribution. Based on a soft voting mechanism, multiple candidate classes in the preliminary classification result are voted, and a confidence factor is set according to the voting result. The confidence factor is used to scale the information entropy to obtain the classification confidence.

[0097] When the classification confidence is greater than a preset reliability threshold, it is determined as correct classification, and intelligent classification of the road cracks is realized.

[0098] The multi-level feature representation of the road surface crack is matched to calculate the distance. For each level of feature, the Euclidean distance and the cosine distance are calculated respectively. The Euclidean distance reflects the absolute distance of the feature vector in space, and the square root of the sum of the squares of the difference between the corresponding elements of the two feature vectors is taken when calculating. The cosine distance measures the angle between the feature vectors, and the dot product of the two vectors is divided by the product of their respective lengths, and then 1 is subtracted from the result. In practical applications, for the i-th level of feature, assuming that the test sample feature vector is A_i and the reference sample feature vector is B_i, when the dimension of A_i is 128, the Euclidean distance value is 0.45 and the cosine distance value is 0.32. To combine the advantages of the two distance measures, a weighted combination method is used, and the weight coefficient a is set to 0.6. The combined distance is 0.6*0.45 + (1-0.6)*0.32 = 0.398. Repeat this process for all levels of features to obtain the complete feature matching distance.

[0099] Based on the calculated feature matching distance, a k-nearest neighbor classification strategy is used for preliminary classification. In practical implementation, the value of k is set to 5, that is, the 5 reference samples with the smallest feature matching distance are selected for voting. For example, for a test sample, among the 5 reference samples with the smallest distance, 3 belong to the "transverse crack" category, 1 belongs to the "longitudinal crack" category, and 1 belongs to the "crack" category. The preliminary classification result of the test sample is "transverse crack", and the sample number distribution of each candidate category is recorded.

[0100] To further evaluate the reliability of the classification result, a confidence evaluation network is introduced. This network encodes the multi-level feature representation as a probability distribution, converting the original feature into a probability distribution form. In specific implementation, a fully connected neural network with two hidden layers is used, each layer has 64 and 32 neurons respectively, the input is the original feature vector, and the output is the probability value of each dimension. For example, for a feature vector with a dimension of 128, after probability encoding, the probability distribution obtained has a total probability value of 1 for each dimension, with a maximum probability value of 0.15 and a minimum probability value of 0.002.

[0101] The Monte Carlo sampling method is used to obtain a plurality of network parameter samples. In actual application, 20 sets of network parameters are sampled, each set of parameters is generated by adding random noise subject to a normal distribution on the basis of the trained confidence evaluation network parameters, the noise mean is 0, and the standard deviation is 0.01. The feature probability distribution is input into the forward propagation network constructed by the 20 sets of network parameters, and 20 predicted probability distributions are obtained. For a test sample, the first set of network prediction results are as follows: the transverse crack probability is 0.65, the longitudinal crack probability is 0.25, and the crack probability is 0.10; the second set of network prediction results are as follows: the transverse crack probability is 0.68, the longitudinal crack probability is 0.22, and the crack probability is 0.10. The information entropy is calculated based on the weighted sum of the prediction probabilities of all categories, and the entropy value is 0.75.

[0102] The candidate categories in the preliminary classification result are voted based on a soft voting mechanism. In the foregoing example, the transverse crack obtains 3 votes, the longitudinal crack obtains 1 vote, and the crack obtains 1 vote. According to the voting ratio, a confidence factor is set. In specific implementation, the ratio of the highest number of votes to the total number of votes is used as the confidence factor, that is, 3 / 5 = 0.6. The confidence factor is used to scale the information entropy calculated in the foregoing, and the classification confidence is 0.6 x 0.75 = 0.45.

[0103] The obtained classification confidence is compared with a preset reliability threshold. In actual application, the reliability threshold is set to 0.4. Since the classification confidence 0.45 is greater than the threshold 0.4, it is confirmed that the classification result is a correct classification, and finally, "transverse crack" is output as the classification result of the pavement crack sample.

[0104] The method has the advantages that a plurality of distance metrics and confidence evaluation mechanisms are fused, can adapt to the change of crack morphology in different pavement environments, and has strong robustness for crack identification in a complex background. Meanwhile, by introducing the confidence evaluation, the reliability of the classification result can be actively identified, manual intervention is prompted for a low-confidence sample, and engineering maintenance decision errors caused by false classification are avoided.

[0105] The pavement multi-type crack intelligent classification and feature extraction system of the embodiment of the application comprises:

[0106] A first unit is configured to extract a crack skeleton from a to-be-detected pavement to obtain a crack skeleton line image.

[0107] The second unit is configured to extract geometric features of the cracks by using a region growing algorithm and a histogram of oriented gradients based on the crack skeleton line image, extract texture features of the cracks by using a local binary pattern and a Gabor filter bank, input the geometric features and the texture features into a deformable convolution network, and dynamically adjust a spatial scale of feature extraction by the deformable convolution network.

[0108] The third unit is configured to learn class-agnostic meta-knowledge from known crack classes by using a few-shot learning strategy, construct a Markov decision process to model a class evolution mode based on the meta-knowledge, and adaptively update class prototypes of the cracks by using a reinforcement learning method.

[0109] In a third aspect, an electronic device is provided, including:

[0110] a processor;

[0111] a memory for storing processor-executable instructions;

[0112] The processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0113] In a fourth aspect, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0114] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein, which are used to perform various aspects of the present application.

[0115] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for intelligent classification and feature extraction of multiple types of road surface cracks, characterized in that, include: Crack skeletons are extracted from the road surface to be inspected to obtain crack skeleton line images; Based on the crack skeleton line image, the geometric features of the crack are extracted using a region growing algorithm and histogram of oriented gradients, and the texture features of the crack are extracted using a local binary mode and a Gabor filter bank, including: The crack skeleton line image is divided into multiple local regions. A seed point discrimination criterion is constructed in the local regions. Seed points are determined based on the seed point discrimination criterion. Region growth is performed on the seed points to obtain the crack region. In the crack region, an orientation histogram is established based on the gradient magnitude and gradient direction of each pixel. The orientation histogram is used as the geometric feature of the crack region. Feature extraction windows of different scales are constructed for the crack region. Uniform sampling is performed with the center pixel of each feature extraction window as the center to obtain neighboring sampling points. A binary encoding sequence is constructed based on the neighboring sampling points and the center pixel. The number of jumps in the values ​​of the binary encoding sequence is used as the local binary pattern feature value. The Gabor filter includes a real part filter kernel and an imaginary part filter kernel. The crack region is convolved with both the real part filter kernel and the imaginary part filter kernel to extract Gabor filter features. The local binary pattern feature value and the Gabor filter features are combined at different scales to obtain the texture features of the crack region. The geometric features and the texture features are input into a deformable convolutional network. The deformable convolutional network dynamically adjusts the spatial scale of feature extraction and extracts local feature vectors and global context feature vectors of the crack skeleton line image in parallel at different spatial scales, and encapsulates them into multidimensional feature vectors. A few-shot learning strategy is used to learn category-independent meta-knowledge from known crack categories. Based on this meta-knowledge, a Markov decision process is constructed to model the category evolution pattern, and a reinforcement learning method is used to adaptively update the category prototypes of each crack. Positive and negative sample pairs are constructed based on the category prototypes, and feature representation quality scores are calculated. A contrastive learning strategy is used to project the multidimensional feature vectors and the positive and negative sample pairs with feature representation quality scores higher than a preset discrimination threshold into multiple feature hierarchical spaces to obtain multi-level feature representations, including: Cosine similarity is used to measure feature distance. A predetermined number of samples with the closest feature distance are selected from the sample set of the same class as the prototype to construct positive sample pairs. A predetermined number of samples with the closest feature distance are selected from the sample set of other classes to construct negative sample pairs. Based on the positive and negative sample pairs, intra-class compactness scores and inter-class separability scores are calculated and combined to obtain the feature expression quality score. Multiple projection matrices with sequentially decreasing dimensions are constructed. The multidimensional feature vector and the positive and negative sample pairs with feature expression quality scores higher than a preset discrimination threshold are projected onto multiple feature hierarchical spaces through these multiple projection matrices. In the feature hierarchical space, the cosine similarity between the multidimensional feature vector and the positive and negative sample pairs is calculated to obtain the contrast loss, and the Euclidean distance between adjacent feature levels is calculated to obtain the consistency loss. By minimizing the contrast loss and the consistency loss, the multi-level feature representation is obtained. Preliminary classification is performed by calculating the feature matching distance of the multi-level feature representation, and the classification confidence is evaluated by combining the confidence evaluation network to achieve intelligent classification of road surface cracks.

2. The method according to claim 1, characterized in that, The deformable convolutional network dynamically adjusts the spatial scale of feature extraction, extracting local feature vectors and global context feature vectors of the crack skeleton line image in parallel at different spatial scales, and encapsulating them into a multi-dimensional feature vector including: The deformable convolutional network receives the geometric features and the texture features, calculates the two-dimensional spatial offset, dynamically adjusts the sampling position of the standard convolutional kernel based on the two-dimensional spatial offset, and performs bilinear interpolation and multiple variable sampling rate downsampling at the adjusted sampling position to obtain feature response maps at different spatial scales. Parallel deformable convolutional branches are constructed for the feature response maps at each spatial scale. The deformable convolutional branches adaptively adjust the deformation amount through backpropagation of the loss function. An attention mechanism is used to balance and combine the output features of different deformable convolutional branches to obtain the local feature map at each spatial scale. The local feature maps are uniformly upsampled to the same spatial resolution and element-wise superimposed to obtain a multi-scale fused feature map. Feature pooling is performed on the local feature map and the multi-scale fused feature map to obtain the local feature vector and the global context feature vector of the crack skeleton line image. The local feature vector and the global context feature vector are concatenated in the channel dimension and then dimensionality reduction is performed through a projection matrix to obtain the multi-dimensional feature vector.

3. The method according to claim 1, characterized in that, A few-shot learning strategy is used to learn category-independent meta-knowledge from known crack categories. Based on this meta-knowledge, a Markov decision process is constructed to model the category evolution pattern. Then, a reinforcement learning method is used to adaptively update the category prototypes of each crack, including: Extract support sets and query sets from data with known crack categories; extract features from samples in the support set and calculate the category prototype for each category; calculate the distance metric between samples in the query set and the category prototypes for each category; calculate the probability distribution of samples in the query set belonging to each category based on the distance metric; and obtain category-independent meta-knowledge. The state space is represented as a combination of the category prototypes and their uncertainties for each category, and the action space is represented as the update step size and update direction of the category prototypes. The category prototypes for the next state are calculated based on the state space and the action space to obtain the Markov decision process. A target Q-network and an evaluation Q-network are constructed, with the target Q-network and the evaluation Q-network respectively outputting the target Q-value and the evaluation Q-value for each update action. A reward function that comprehensively considers classification accuracy and changes in category distribution is constructed. An experience replay pool is built based on the state transition samples in the Markov decision process. Training batches are sampled from the experience replay pool, and the parameters of the target Q-network and the evaluation Q-network are alternately updated based on the reward function. The optimal update action is selected based on the target Q-value and the evaluation Q-value to update the category prototype of each crack.

4. The method according to claim 1, characterized in that, Preliminary classification is performed by calculating the feature matching distance of the multi-level feature representation, and the classification confidence is evaluated by combining the confidence evaluation network, thereby achieving intelligent classification of pavement cracks, including: For each layer of the multi-level feature representation, calculate the Euclidean distance and cosine distance, and combine them to obtain the feature matching distance; based on the feature matching distance, use the k-nearest neighbor classification strategy to obtain the preliminary classification result; The confidence evaluation network probabilistically encodes the multi-level feature representation to obtain a feature probability distribution, uses Monte Carlo sampling to obtain multiple sets of network parameter samples, inputs the feature probability distribution into the forward propagation network constructed from the network parameter samples to obtain a predicted probability distribution, and calculates the information entropy of the predicted probability distribution; based on a soft voting mechanism, it votes on multiple candidate categories in the preliminary classification results, sets a confidence factor according to the voting results, and uses the confidence factor to scale the information entropy to obtain the classification confidence. When the classification confidence level is greater than a preset reliability threshold, it is determined to be a correct classification, thus realizing intelligent classification of road surface cracks.

5. A smart classification and feature extraction system for multiple types of road surface cracks, used to implement the method as described in any one of claims 1-4, characterized in that, include: The first unit is used to extract the crack skeleton from the road surface to be detected and obtain the crack skeleton line image. The second unit is used to extract the geometric features of the crack based on the crack skeleton line image by using a region growing algorithm and histogram of oriented gradients, and to extract the texture features of the crack by using a local binary mode and a Gabor filter bank. The geometric features and the texture features are input into a deformable convolutional network, which dynamically adjusts the spatial scale of feature extraction and extracts the local feature vector and global context feature vector of the crack skeleton line image in parallel at different spatial scales, and encapsulates them into a multi-dimensional feature vector. The third unit is used to learn category-independent meta-knowledge from known crack categories through a few-shot learning strategy. Based on the meta-knowledge, a Markov decision process is constructed to model the category evolution pattern, and a reinforcement learning method is used to adaptively update the category prototype of each crack. Based on the category prototype, positive and negative sample pairs are constructed and feature expression quality scores are calculated. Through a contrastive learning strategy, the multi-dimensional feature vector and the positive and negative sample pairs with feature expression quality scores higher than a preset discrimination threshold are projected onto multiple feature hierarchical spaces to obtain multi-level feature representations. Preliminary classification is performed by calculating the feature matching distance of the multi-level feature representations, and the classification confidence is evaluated by combining a confidence evaluation network to achieve intelligent classification of pavement cracks.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 4.

Images