Real-time intelligent detection method for highway pavement diseases

Abstract

The invention discloses a highway pavement disease real-time intelligent detection method, which belongs to the technical field of road detection and comprises the following steps: acquiring an original image of a to-be-detected highway pavement; inputting the original image into a pre-constructed detection model for processing to obtain a detection result; wherein the detection model comprises a down-sampling module, a feature extraction module based on a separable sparse encoder and a spatial position information encoding module; the down-sampling module is used for processing the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map; a feature extraction module based on a separable sparse encoder is used for performing block processing on the primary feature map to capture disease features in each image block to obtain an advanced feature map; and the spatial position information coding module is used for performing semantic feature capture on the advanced feature map based on the spatial position information so as to obtain a detection result. According to the method, the calculation amount is reduced, and the processing efficiency and precision are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of road detection, and in particular relates to a real-time intelligent detection method for highway pavement defects. Background Art

[0002] As an important part of civil infrastructure, the road surface condition is directly related to traffic safety and transportation efficiency. Due to factors such as the natural environment, traffic load and material aging, the road surface of the highway is prone to cracks, potholes, damage, collapse and other defects. If these defects are not detected and treated in time, they may gradually expand and eventually lead to the destruction of the road surface structure, increase maintenance costs, and even cause traffic accidents, posing a threat to local economic development and residents' travel safety.

[0003] In terms of highway crack detection, early detection methods mainly include those based on image morphology algorithms and time and frequency domain transformation algorithms, such as edge detection and threshold segmentation to identify cracks. However, due to the complex changes in crack shape and size, the robustness of image morphology-based algorithms is poor, and it is more likely to make errors or miss detections, especially when processing images with complex backgrounds and uneven lighting. The performance of the algorithm may decline, which poses a challenge to accurately identifying cracks. Time and frequency domain transformation algorithms are very sensitive to noise in images and are prone to misdetection of noisy images. Images may have broken edges or incomplete connections, which affects the accurate detection of cracks.

[0004] With the development of artificial intelligence technology, especially the application of deep learning in the field of image processing, new possibilities have been provided for the automated detection of highway cracks. Methods based on convolutional neural networks (CNN) have been widely used in crack detection. For example, deep neural networks (DNN) and fully convolutional networks (FCN) have been used. These methods have made significant progress in the accuracy and efficiency of crack detection.

[0005] However, existing crack detection methods still face challenges in terms of real-time and accuracy, especially in the ever-changing environmental conditions of rural roads. On the one hand, compared with urban roads, rural roads have more complex environmental backgrounds and uneven lighting conditions, resulting in complex and diverse rural road conditions, and the quality of the collected road images is uneven. Existing deep learning-based crack detection methods may not be able to accurately identify some small or irregular cracks when processing these poor quality images, resulting in low detection accuracy. On the other hand, existing fully convolutional networks and other complex deep learning architectures require a lot of computing resources, which is also unsatisfactory for edge devices and resource-constrained rural environments. In addition, existing crack detection methods cannot meet the needs of real-time monitoring, especially on vehicle-mounted or drone monitoring equipment, which may result in frame loss during real-time segmentation and missed crack detection. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a real-time intelligent detection method for road pavement defects. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention proposes a real-time intelligent detection method for highway pavement defects, comprising:

[0008] Acquire the original image of the road surface to be detected;

[0009] Input the original image into the pre-built detection model for processing to obtain the detection result;

[0010] The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module;

[0011] The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map;

[0012] The feature extraction module based on the separable sparse encoder is used to process the primary feature map in blocks to capture the disease features in each image block and obtain a high-level feature map;

[0013] The spatial position information encoding module is used to capture semantic features of high-level feature maps based on spatial position information to obtain detection results.

[0014] In a second aspect, the present invention proposes a real-time intelligent detection system for road pavement defects, which is used to implement a real-time intelligent detection method for road pavement defects proposed in the first aspect of the present invention; the system comprises:

[0015] An image acquisition module is used to acquire an original image of the road surface to be detected;

[0016] The detection module is configured with a detection model, and the detection model is used to process the original image acquired by the image acquisition module to obtain a detection result;

[0017] The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module;

[0018] The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map;

[0019] The feature extraction module based on the separable sparse encoder is used to process the primary feature map in blocks to capture the disease features in each image block and obtain a high-level feature map;

[0020] The spatial position information encoding module is used to capture semantic features of high-level feature maps based on spatial position information to obtain detection results.

[0021] Beneficial effects of the present invention:

[0022] 1. The present invention provides a real-time intelligent detection method for road pavement defects, which constructs a detection model including a feature extraction module based on a separable sparse encoder and a spatial position information encoding module. On the one hand, the feature extraction module based on the separable sparse encoder divides the input feature map into multiple small blocks for processing, so that the model can focus on processing tiny defect areas, thereby improving the accuracy of defect segmentation; and each small block is processed independently, which can reduce the amount of calculation and improve processing efficiency, meeting the requirements of real-time detection. On the other hand, the spatial position information encoding module enhances the model's ability to recognize the spatial position of the defect by encoding the coordinate information of the defect in the image, further improving the accuracy of defect segmentation.

[0023] 2. The real-time intelligent detection method for highway pavement defects provided by the present invention can be successfully applied to the detection of defects on edge devices and resource-constrained rural cement roads after being quantized by TensorRT.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a real-time intelligent detection method for highway pavement defects provided by an embodiment of the present invention;

[0026] Figure 2It is a principle framework diagram of a real-time intelligent detection method for highway pavement defects provided by an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of an original image of a road surface to be detected collected and provided in an embodiment of the present invention;

[0028] Figure 4 It is a principle framework diagram of a feature extraction module based on a separable sparse encoder provided in an embodiment of the present invention;

[0029] Figure 5 is a principle framework diagram of a spatial position information encoding module provided by an embodiment of the present invention;

[0030] Figure 6 It is a structural block diagram of a real-time intelligent detection system for highway pavement defects provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The first aspect of the present invention provides a real-time intelligent detection method for highway pavement defects. Figure 1 and Figure 2 , Figure 1 is a flow chart of a real-time intelligent detection method for road pavement defects provided by an embodiment of the present invention. Figure 2 The figure is a principle framework diagram of a real-time intelligent detection method for road pavement defects provided by an embodiment of the present invention. The real-time intelligent detection method for road pavement defects provided by the present invention mainly comprises the following steps:

[0033] Step 1: Obtain the original image of the road surface to be detected.

[0034] Specifically, this embodiment can use low-cost vehicle-mounted camera equipment to collect images of a road surface in a rural area to be tested. Due to the complex and diverse conditions of rural roads and the constraints of detection costs, the collected images have the following characteristics: (1) The quality of the disease image is low and affected by the ambient light intensity; (2) The imaging angle is tilted, the top view and other non-fixed angle imaging, and the non-uniform single lane imaging; (3) The road surface imaging quality is uneven, the types of diseases are diverse, and have obvious long-tail characteristics.

[0035] For example, taking pavement crack disease as an example, the collected original image is as follows: Figure 3As shown in the figure, we can see that the image on the left has a poor imaging angle, the image in the middle has equipment shadows, and the image on the right has tree shades. This shows that for rural roads with complex and diverse road conditions, the collected original images may have various quality problems.

[0036] Based on this, this embodiment constructs a detection model based on a separable sparse encoder and spatial position information encoding to process the above-mentioned original image, thereby realizing intelligent detection of road surface diseases.

[0037] Step 2: Input the original image into the pre-built detection model for processing to obtain the detection result.

[0038] The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module;

[0039] The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map;

[0040] The feature extraction module based on the separable sparse encoder is used to process the primary feature map in blocks to capture the disease features in each image block and obtain a high-level feature map;

[0041] The spatial position information encoding module is used to capture semantic features of high-level feature maps based on spatial position information to obtain detection results.

[0042] It is understandable that before the collected original image is input into the detection model for processing, the image needs to be preprocessed to adapt to the input requirements of the detection model, including operations such as resizing and normalization.

[0043] After the preprocessed image is sent to the detection model, it will first undergo preliminary processing in the downsampling module to reduce the spatial resolution of the image and increase the number of feature channels to provide richer information for subsequent feature extraction and obtain a primary feature map.

[0044] Optionally, as an implementation manner, the downsampling module processes the input original image specifically according to the following steps S101-S102:

[0045] S101, the input image is first passed through a convolution layer, then through a batch normalization layer, and finally through a SILU activation function to prevent gradient explosion and obtain the first feature map.

[0046] S102: The first feature map is sequentially subjected to two downsampling layers to perform dimension reduction feature perception and semantic information perception to obtain a primary feature map.

[0047] Specifically, in the downsampling layer, the input feature map first undergoes a group dilation convolution with a step size of 2 to reduce the dimension of the features in the image; then the bottleneck after a large field of view and multi-level dilation is used to perceive the high-level semantic information of long-distance cracks.

[0048] After being processed by the downsampling module, it provides richer information for subsequent feature extraction and obtains a primary feature map.

[0049] Furthermore, the primary feature map output by the downsampling module is input into a feature extraction module based on a separable sparse encoder for feature extraction to capture the disease features in each image block and obtain a high-level feature map.

[0050] Optional, see Figure 4 , Figure 4 It is a principle framework diagram of a feature extraction module based on a separable sparse encoder provided in an embodiment of the present invention.

[0051] As an implementation method, the feature extraction module based on the separable sparse encoder specifically performs block processing on the input primary feature map according to the following steps S201-S203:

[0052] S201, performing feature separation on the primary feature map in space to divide the primary feature map into a plurality of image blocks.

[0053] Specifically, the primary feature map can be divided in width and height to achieve spatial feature separation. For example, a H×W feature map can be divided into small blocks, where S is the size of the partition.

[0054] S202: Pass the plurality of image blocks to a sparse encoder for further processing to obtain a plurality of corresponding second feature maps.

[0055] The sparse encoder is an optimized encoder structure that is able to process these small blocks of features, reducing the number of attention heads that must be activated at inference time, thereby improving the inference efficiency of the model and extracting detailed information about the cracks.

[0056] In this embodiment, the sparse encoder specifically processes the input image blocks according to the following steps a)-c):

[0057] a) Several input image blocks are passed through the convolution layer and the normalization layer in sequence to standardize the features and obtain several standardized feature maps.

[0058] Specifically, for each small input image patch, it is first processed by a convolutional layer.

[0059] Then, a normalization layer is used to normalize the features and obtain a standardized feature map. This helps stabilize the model training process and improve the representation ability of the features. The specific steps are:

[0060] For each small block, calculate the mean m and variance v of its elements. The mean m is the average of all elements in the small block, and the variance v is the average of the squared differences between the elements and their mean. Each small block is normalized using the calculated mean and variance, and then the normalized features are adjusted by multiplying by the scale parameter γ and adding the offset parameter β.

[0061] b) Send several standardized feature maps into the sparse multi-head self-attention module to extract features based on global context information to obtain several attention feature maps.

[0062] Specifically, the standardized feature map obtained after normalization is fed into a sparse multi-head self-attention module, which can model the relationship between different feature blocks, which helps capture the global context information of the disease. And the attention is integrated into the hybrid expert mechanism, which regards the attention head as an expert and allows each token to dynamically select the most relevant multiple attention heads. The specific steps are as follows:

[0063] b1) Divide the normalized feature map of the current input into multiple tokens, where each token is a pixel block or a higher-level feature representation. These tokens will be used as input to the multi-head attention layer.

[0064] b2) For each token, use a different weight matrix to project it to the query, key, and value vectors.

[0065] Specifically, using different weight matrices W Q , W K , W V , projecting each token into query (Q), key (K), and value (V) vectors, represented as:

[0066] Q=XW Q ,K=XW K ,V=XW V ;

[0067] b3) Dynamically select the most relevant multiple attention heads for each token.

[0068] Specifically, a router is used to dynamically select the Top-K most relevant attention heads for each token. The router score is given by:

[0069]

[0070] In the formula, gi represents the routing score of the i-th attention head, α 1 and α 2 represents the balance coefficient, Softmax() represents the normalized exponential function, W s and W r denote the projection matrices of the shared head and the routing head, respectively, i denotes the i-th attention head, h s represents the number of shared heads, Top-K represents the number of most relevant attention heads, and h represents the total number of attention heads.

[0071] b4) Calculate the corresponding outputs for the selected multiple attention heads based on the query, key, and value vectors. Specifically, for each selected head, the calculation formula for the attention output is:

[0072] H i =Attention(QW i Q ,KW i K ,VW i V );

[0073] In the formula, H i represents the output of i attention heads, W i Q , W i K , W i V The projection matrices representing the query, key, and value of the i attention heads respectively.

[0074] b5) Perform weighted summation on the outputs of all attention heads to obtain the final attention feature.

[0075] Specifically, the outputs of the selected attention heads are weighted and summed to obtain the final attention output, which is expressed as follows:

[0076]

[0077] In the formula, MoH represents the final attention feature, (X, X') represents the input Token, and W i O Represents the output projection matrix of the i attention heads.

[0078] c) Concatenate several attention features with several input image blocks to obtain several second feature maps as the output of the sparse encoder.

[0079] Specifically, the attention features output by the sparse multi-head self-attention module are concatenated with the corresponding input image blocks and then passed as the output of the sparse encoder to subsequent network layers, such as feedforward networks and layer normalization.

[0080] S203: Perform feature reorganization on a plurality of second feature maps to obtain a first feature matrix.

[0081] Specifically, the feature blocks processed by the separable sparse encoder are recombined to form a new feature matrix, which is recorded as the first feature matrix. This matrix contains rich information of the crack area and has a higher spatial resolution.

[0082] S204: Add the first characteristic matrix to an all-1 matrix, and perform a self-multiplication operation to obtain a second characteristic matrix.

[0083] In order to obtain crack markers, the reorganized feature matrix is ​​added to an all-1 matrix and then multiplied by itself, which helps the model better capture the characteristics of the cracks. The obtained feature matrix is ​​recorded as the second feature matrix.

[0084] S205. Expand the dimension of the second feature matrix to obtain a high-level feature map.

[0085] The third and fourth dimensions of the second feature matrix are expanded so that the high-level feature maps obtained after processing can be passed to the subsequent modules of the model for processing. This step ensures the effective flow of feature information in the model.

[0086] The feature extraction module based on the separable sparse encoder designed in this embodiment adopts a strategy of separating the features of the input feature map and then processing the separated features in blocks, which has two obvious advantages: first, it reduces the amount of calculation and memory usage, improving the efficiency and performance of the model; second, it enables the module to focus the field of view on a smaller area, improving the accuracy of crack segmentation. Through the block sparse multi-head attention mechanism, the model can effectively capture crack features in local areas, while reducing computational complexity and improving reasoning efficiency.

[0087] Furthermore, the high-level feature map output by the feature extraction module based on the separable sparse encoder is input into the spatial position information encoding module for further semantic feature capture to obtain the detection result.

[0088] Optional, see Figure 5 , Figure 5 It is a principle framework diagram of the spatial position information encoding module provided in an embodiment of the present invention.

[0089] As an implementation method, the spatial position information encoding module specifically captures semantic features of the input high-level feature map according to the following steps S301-S308 to obtain a detection result.

[0090] S301. Perform multi-layer convolution processing on the high-level feature map to enhance feature representation. At the same time, through the convolution operation, the number of channels of the high-level feature map is doubled to capture richer object information and obtain a third feature map.

[0091] Among them, multi-layer convolution processing on high-level feature maps includes applying batch normalization and activation functions.

[0092] S302: Perform a channel separation operation on the third feature map to obtain corresponding third feature matrices of multiple channels.

[0093] Specifically, after the number of channels is doubled, a channel separation operation is performed. This step divides the channels into two different characteristic matrices along the second dimension according to the equal distribution principle, denoted as A1 and A2, which are also called the third characteristic matrix in this embodiment.

[0094] S303, applying a pooling kernel to each separated channel, and encoding the third feature matrix of each channel in the horizontal and vertical directions.

[0095] Specifically, for each separated channel, a pooling kernel of size (H, 1) and (1, W) is applied to encode each channel in the horizontal and vertical directions. Here, B represents the batch size, C represents the number of feature channels, H represents the image height, and W represents the image width.

[0096] S304, using two independent global pooling layers to process the third feature matrix of each channel respectively, and correspondingly obtaining multiple fourth feature matrices rich in height information and width information.

[0097] Specifically, two independent global pooling layers are used to process the input third feature matrix respectively, and convert it into the form of A(B,C,H,1) and A(B,C,1,W), which are recorded as the fourth feature matrix. In this way, A(B,C,H,1) is rich in the height information of the feature matrix, while A(B,C,1,W) is rich in width information.

[0098] S305 , concatenating multiple fourth characteristic matrices to form a fifth characteristic matrix.

[0099] Specifically, the last two dimensions of A(B,C,1,W) are transposed so as to be concatenated with A(B,C,H,1) along the third dimension (corresponding to the image height) to form a new matrix A(B,C,2H,1), which is recorded as the fifth characteristic matrix.

[0100] S306. Process the fifth feature matrix through a convolution kernel to encode horizontal and vertical information and capture corresponding coordinate information, thereby obtaining coordinate matrices in the horizontal and vertical directions.

[0101] Specifically, the fifth feature matrix is ​​subjected to 1×1 convolution processing, the horizontal and vertical information of the feature map is encoded, and the corresponding coordinate information is captured at the same time, and the horizontal coordinate matrix and the vertical coordinate matrix are obtained accordingly.

[0102] S307 , respectively perform feature enhancement on the coordinate matrices in the horizontal and vertical directions to further extract and enhance coordinate information, and obtain two enhanced coordinate matrices.

[0103] Convolution and activation functions are applied to the horizontal coordinate matrix and the vertical coordinate matrix respectively to further extract and enhance the coordinate information. Then, matrix multiplication and addition operations are performed to enhance the height and width information of the feature dimension to obtain the enhanced horizontal coordinate matrix and vertical coordinate matrix.

[0104] S308, using dilated convolution, batch normalization layer and GELU activation function, the semantic features of the two enhanced coordinate matrices are captured, and the captured features are sent to the detection head for detection to obtain the detection results.

[0105] Specifically, in order to expand the receptive field and capture more spatial information, dilated convolution, batch normalization layer and GELU activation function are used here to maximize the semantic feature capture of cracks.

[0106] The spatial position information encoding module designed in this embodiment enhances the model's ability to identify the spatial position of cracks by encoding the coordinate information of cracks. The feature fusion technology enhances the feature expression ability and improves the accuracy of crack segmentation.

[0107] It should be noted that before the original image is input into the pre-built detection model for processing, it also includes:

[0108] Construct a highway pavement disease dataset; based on the heuristic training strategy, use the dataset to train the pre-built detection model, and use the stochastic gradient descent optimizer to optimize the training process of the detection model to obtain a trained detection model.

[0109] Specifically, when constructing a dataset, low-cost vehicle-mounted camera equipment can be used to collect road damage conditions on cement roads in one or more rural areas of China to obtain a large number of real datasets, and then the datasets are divided into training sets, validation sets, and test sets in a ratio of 8:1:1. The training set is used to train the detection model, and the validation set and test set are used to verify the model.

[0110] During the training process, heuristic training strategies such as image enhancement and hyperparameter tuning were adopted.

[0111] Optionally, the hyperparameters for training can be set to: training batch size of 300 epochs, stochastic gradient descent (SGD) optimizer, learning rate of 0.01, weight decay of 0.0005, and image size of 640×640 pixels.

[0112] During training, a load balancing loss is introduced to avoid most tokens being routed to only a few heads, and its expression is:

[0113]

[0114] Where, L b represents the load balancing loss, f i represents the selection frequency of the i-th attention head, P i represents the selection probability of the i-th attention head, h s represents the number of shared heads, and h represents the total number of attention heads.

[0115] Then, the task-specific loss and the load-balanced loss are combined to obtain the overall training objective:

[0116] L=L task +βL b ;

[0117] In the formula, L represents the total loss, L task represents the task loss, and β represents the hyperparameter that weighs the task loss and load balancing loss.

[0118] After obtaining the trained detection model, the model needs to be deployed at low cost, which specifically includes the operations of steps S401-S404:

[0119] S401. Use the TensorRT engine to quantize the trained detection model.

[0120] Among them, the quantization process involves converting the model's weights and activations from floating point numbers (fp32) to lower precision representations such as int8.

[0121] This operation can improve the running efficiency of the model on the hardware. The model quantized by TensorRT can be successfully applied to the disease detection of edge devices and resource-constrained rural cement roads.

[0122] S402. Use the command line tool trtexec.exe provided by TensorRT to convert the model in ONNX format to the engine format of TensorRT.

[0123] This operation involves specifying the model file, saving the engine file, and selecting the quantitative strategy. For the detailed process, please refer to the existing relevant technical implementation.

[0124] S403. Configure TensorRT and OpenCV in the C++ environment to ensure that the detection model can be loaded and executed correctly.

[0125] This operation includes reading the TensorRT engine file, setting the input and output formats, and linking the necessary library files. For detailed process, please refer to the existing related technical implementation.

[0126] S404: Perform performance testing on the quantized detection model on the target hardware.

[0127] Specifically, the test content includes the model's inference speed (FPS) and accuracy (such as Map).

[0128] Select appropriate quantization strategies and operation modes based on test results. Integrate the optimized model into practical applications, such as crack detection systems for vehicles. Ensure that the model can receive input data (such as video streams) in real time and output crack segmentation results.

[0129] In summary, the present invention provides a real-time intelligent detection method for road pavement defects, which constructs a detection model including a feature extraction module based on a separable sparse encoder and a spatial position information encoding module. On the one hand, the feature extraction module based on the separable sparse encoder divides the input feature map into multiple small blocks for processing, so that the model can focus on processing tiny defect areas, thereby improving the accuracy of defect segmentation; and each small block is processed independently, which can reduce the amount of calculation and improve processing efficiency, meeting the requirements of real-time detection. On the other hand, the spatial position information encoding module enhances the model's ability to recognize the spatial position of the defect by encoding the coordinate information of the defect in the image, further improving the accuracy of defect segmentation.

[0130] Based on the same inventive concept, the second aspect of the present invention also provides a real-time intelligent detection system for road pavement defects. Figure 6 , Figure 6 : is a structural block diagram of a real-time intelligent detection system for road pavement defects provided by an embodiment of the present invention, the system comprising:

[0131] An image acquisition module is used to acquire an original image of the road surface to be detected;

[0132] The detection module is configured with a detection model, and the detection model is used to process the original image acquired by the image acquisition module to obtain a detection result;

[0133] The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module;

[0134] The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map;

[0135] The feature extraction module based on the separable sparse encoder is used to process the primary feature map in blocks to capture the disease features in each image block and obtain a high-level feature map;

[0136] The spatial position information encoding module is used to capture semantic features of high-level feature maps based on spatial position information to obtain detection results.

[0137] A real-time intelligent detection system for highway pavement defects provided by the present invention is used to implement a real-time intelligent detection method for highway pavement defects provided by the first aspect of the present invention, which is basically similar to the method embodiment. Therefore, the system can also achieve the same or similar beneficial effects as the above method. For relevant matters, please refer to the partial description of the method embodiment.

[0138] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0139] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0140] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices (equipment), systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware, which are collectively referred to as "modules" herein. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as a part of hardware, or may adopt other distribution forms, such as through the Internet or other wired or wireless telecommunication systems.

[0141] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A real-time intelligent detection method for highway pavement defects, characterized in that: include: Acquire the original image of the road surface to be detected; Inputting the original image into a pre-built detection model for processing to obtain a detection result; The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module; The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map; The feature extraction module based on the separable sparse encoder is used to perform block processing on the primary feature map to capture the disease features in each image block and obtain a high-level feature map; The spatial position information encoding module is used to capture semantic features of the high-level feature map based on the spatial position information, so as to obtain a detection result.

2. A real-time intelligent detection method for road pavement defects according to claim 1, characterized in that: The downsampling module specifically processes the original image according to the following steps to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map: The input original image first passes through a convolution layer, then a batch normalization layer, and finally a SILU activation function to obtain the first feature map; The first feature map is sequentially passed through two downsampling layers to perform dimension reduction feature perception and semantic information perception to obtain a primary feature map.

3. A real-time intelligent detection method for road pavement defects according to claim 1, characterized in that: The feature extraction module based on the separable sparse encoder specifically performs block processing on the primary feature map according to the following steps to capture the disease features in each image block and obtain a high-level feature map: Performing feature separation on the primary feature map in space to divide the primary feature map into a plurality of image blocks; Passing the plurality of image blocks to a sparse encoder for further processing to obtain a plurality of second feature maps; Performing feature recombination on the plurality of second feature maps to obtain a first feature matrix; Add the first characteristic matrix to an all-1 matrix, and perform a self-multiplication operation to obtain a second characteristic matrix; The second feature matrix is ​​dimensionally expanded to obtain a high-level feature map.

4. A real-time intelligent detection method for road pavement defects according to claim 3, characterized in that: The sparse encoder specifically processes the input image blocks according to the following steps to obtain corresponding second feature maps: The input image blocks are sequentially passed through the convolution layer and the normalization layer to standardize the features and obtain several standardized feature maps; Sending the plurality of standardized feature maps into a sparse multi-head self-attention module to extract features based on global context information to obtain a plurality of attention features; The plurality of attention features are correspondingly concatenated with a plurality of input image blocks to obtain a plurality of second feature maps.

5. A real-time intelligent detection method for road pavement defects according to claim 4, characterized in that: The sparse multi-head self-attention module specifically processes the input standardized feature map according to the following steps to obtain the attention feature, including: Divide the currently input standardized feature map into multiple tokens, where each token is a pixel block or a higher-level feature representation; For each token, use a different weight matrix to project it into the query, key, and value vectors; Dynamically select the most relevant multiple attention heads for each token; Calculate corresponding outputs for the selected multiple attention heads based on the query, key and value vectors; The outputs of all attention heads are weighted summed to obtain the final attention feature.

6. A real-time intelligent detection method for road pavement defects according to claim 1, characterized in that: The spatial position information encoding module specifically captures semantic features of the high-level feature map according to the following steps to obtain a detection result: Performing multi-layer convolution processing on the high-level feature map to enhance feature representation, and at the same time doubling the number of channels of the high-level feature map through the convolution operation, thereby capturing richer object information, and obtaining a third feature map; Performing a channel separation operation on the third feature map to obtain corresponding third feature matrices of multiple channels; For each separated channel, a pooling kernel is applied, and the third feature matrix of each channel is encoded in the horizontal and vertical directions; Use two independent global pooling layers to process the third feature matrix of each channel respectively, and obtain multiple fourth feature matrices rich in height information and width information; splicing the plurality of fourth characteristic matrices to form a fifth characteristic matrix; The fifth feature matrix is ​​processed by a convolution kernel to encode horizontal and vertical information and capture corresponding coordinate information, thereby obtaining coordinate matrices in the horizontal and vertical directions; Performing feature enhancement on the coordinate matrices in the horizontal and vertical directions respectively to further extract and enhance coordinate information, thereby obtaining two enhanced coordinate matrices; The semantic features of the two enhanced coordinate matrices are captured using dilated convolution, batch normalization layer, and GELU activation function, and the captured features are sent to the detection head for detection to obtain the detection results.

7. A real-time intelligent detection method for road pavement defects according to claim 1, characterized in that: Before the original image is input into the pre-built detection model for processing, the method further includes: A highway pavement disease dataset is constructed; based on a heuristic training strategy, the pre-constructed detection model is trained using the dataset, and the training process of the detection model is optimized using a stochastic gradient descent optimizer to obtain a trained detection model.

8. A real-time intelligent detection method for road pavement defects according to claim 7, characterized in that: When training the detection model, the total loss used is the weighted sum of the task loss and the load balancing loss, which is expressed as: L=L task +βL b ; In the formula, L represents the total loss, L task represents the task loss, L b represents the load balancing loss, β represents the hyperparameter that weighs the task loss and load balancing loss, and f i represents the selection frequency of the i-th attention head, P i represents the selection probability of the i-th attention head, h s represents the number of shared heads, and h represents the total number of attention heads.

9. A real-time intelligent detection method for road pavement defects according to claim 7, characterized in that: After obtaining the trained detection model, it also includes: Use the TensorRT engine to quantize the trained detection model; Use the command line tool trtexec.exe provided by TensorRT to convert the ONNX format model to the TensorRT engine format; Configure TensorRT and OpenCV in the C++ environment to ensure that the detection model can be loaded and executed correctly; Performance test of the quantized detection model on the target hardware.

10. A real-time intelligent detection system for road pavement defects, characterized in that: The real-time intelligent detection system for road pavement defects is used to implement a real-time intelligent detection method for road pavement defects according to any one of claims 1 to 9; the system comprises: An image acquisition module is used to acquire an original image of the road surface to be detected; A detection module is configured with a detection model, wherein the detection model is used to process the original image acquired by the image acquisition module to obtain a detection result; The detection model includes a downsampling module, a feature extraction module based on a separable sparse encoder, and a spatial position information encoding module; The downsampling module is used to process the original image to reduce the spatial resolution of the original image and increase the number of feature channels to obtain a primary feature map; The feature extraction module based on the separable sparse encoder is used to perform block processing on the primary feature map to capture the disease features in each image block and obtain a high-level feature map; The spatial position information encoding module is used to capture semantic features of the high-level feature map based on the spatial position information, so as to obtain a detection result.

Images