Highway pavement detection method based on image recognition
By using image recognition methods based on Mask R-CNN and Faster R-CNN, combined with local region dynamic calculation and multiple anomaly localization modules, the problem of adaptability and real-time performance in complex scenarios of highway pavement detection in existing technologies is solved, and high-precision pavement defect detection is achieved.
Patent Information
- Application Number
- CN202510994438.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-11-04
AI Technical Summary
Existing image recognition-based methods for road surface detection have poor adaptability to complex scenarios, struggle to cope with changes in lighting and shadow interference, find it difficult to balance detection accuracy and real-time performance, and lack modular design, making it difficult to integrate and optimize detection algorithms for different types of defects.
A segmentation model based on Mask R-CNN network architecture and local region dynamic computation is used for foreground and background segmentation. An anomaly detection model with multiple anomaly localization modules, including feature enhancement subnetwork and anomaly localization subnetwork, is combined. Image feature enhancement is performed through histogram equalization, contrast stretching and Fourier transform. A pre-trained network is built using Faster R-CNN and attention mechanism for training to achieve feature fusion and anomaly localization of multiple modules.
It improves the accuracy and efficiency of highway pavement inspection, realizes high-precision real-time inspection in complex scenarios, can effectively identify various pavement defects, and meets the needs of vehicle-mounted mobile inspection.
Smart Images
Figure CN120894615A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of road surface inspection technology, and in particular to a highway road surface inspection method based on image recognition. Background Technology
[0002] Road surface inspection is a crucial aspect of ensuring the safety of transportation infrastructure. Traditional manual inspection methods suffer from inherent drawbacks such as low efficiency, strong subjectivity, and limited coverage. With the rapid development of computer vision and deep learning technologies, intelligent detection methods based on image recognition have gradually become a research hotspot. Current technologies for detecting road surface defects largely rely on single image processing algorithms or general object detection models, which have the following limitations: 1) Poor adaptability to complex scenes, struggling to cope with changes in lighting, shadow interference, and different road surface textures; 2) Difficulty in balancing detection accuracy and real-time performance; high-precision models have high computational complexity, making it difficult to meet the needs of vehicle-mounted mobile inspection; 3) Lack of modular design, making it difficult to integrate and optimize detection algorithms for different defect types. Summary of the Invention
[0003] The purpose of this invention is to provide a road surface detection method based on image recognition, which can improve the accuracy of road surface detection.
[0004] To achieve the above objectives, the present invention provides the following solution:
[0005] A highway pavement detection method based on image recognition, comprising:
[0006] An image acquisition system is built on a carrier to acquire images of the target area;
[0007] The target region image is binarized and cropped to obtain multiple grayscale image blocks;
[0008] Each grayscale image block is input into a segmentation model for foreground and background segmentation to obtain a coarsely processed image; the segmentation model is constructed based on the Mask R-CNN network architecture and dynamic calculation of local regions;
[0009] Each coarsely processed image is input into an anomaly detection model to determine the anomaly location in the target region; the anomaly detection model includes a fusion module and multiple anomaly localization modules connected to the fusion module; each anomaly localization module includes a feature enhancement subnet and an anomaly localization subnet connected in sequence.
[0010] Optionally, the image acquisition system uses a device with both camera and transmission capabilities.
[0011] Optionally, the target region image is binarized and cropped, the specific process including:
[0012] The target region image is converted to grayscale, and the grayscale image is then subjected to median filtering for noise reduction. The image after noise reduction is then cropped to obtain multiple grayscale image blocks.
[0013] Optionally, each grayscale image patch is input into a segmentation model for foreground and background segmentation, the specific process of which includes:
[0014] Based on the Mask R-CNN network, local region dynamic calculation is performed on each grayscale image block using an adaptive threshold, and the foreground region in each image block is extracted to obtain a coarsely processed image; the foreground region is the road surface in the image.
[0015] Optionally, the adaptive threshold includes a local mean and a Gaussian weight; wherein the local mean is used to handle uneven illumination, and the Gaussian weight is used to preserve edge details.
[0016] Optionally, before inputting each of the coarsely processed images into the anomaly detection model, the method further includes: training the anomaly detection model, specifically the following training steps:
[0017] Acquire training data; the training data includes training images and corresponding detection labels; the detection labels include whether the image has an anomaly and the location where the image has an anomaly.
[0018] A pre-trained network was built based on Faster R-CNN and an attention mechanism;
[0019] The training data is input into the pre-trained network, and training is performed with the goal of minimizing the loss between the network output and the detection label. The trained network is then determined as the final anomaly detection model.
[0020] Optionally, the coarsely processed images are input into the anomaly detection model, and the specific process includes:
[0021] After inputting the coarsely processed images into the anomaly detection model, the corresponding image blocks are first processed in each anomaly localization module to obtain the anomalies in each image block and determine the anomaly localization box. Then, the fusion algorithm is used to stitch together each image block and the corresponding anomaly localization box to determine the anomaly location in the target area.
[0022] Optionally, the corresponding image patch in each anomaly localization module is processed, and the specific process includes:
[0023] In each anomaly localization module, histogram equalization, contrast stretching, and Fourier transform are used to enhance the image features of the corresponding input image blocks. The enhanced image blocks are then input into the anomaly localization submodule for detection, and the detected anomaly locations are marked using localization boxes.
[0024] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0025] This invention discloses a highway pavement detection method based on image recognition. The method includes: building an image acquisition system on a carrier to acquire an image of a target region; performing binarization and cropping processing on the target region image to obtain multiple grayscale image blocks; inputting each grayscale image block into a segmentation model for foreground and background segmentation to obtain a coarsely processed image; the segmentation model is constructed based on the Mask R-CNN network architecture and local region dynamic calculation; inputting each coarsely processed image into an anomaly detection model to determine the anomaly location in the target region; the anomaly detection model includes a fusion module and multiple sets of anomaly localization modules connected to the fusion module; each set of anomaly localization modules includes a feature enhancement subnet and an anomaly localization subnet connected in sequence. This invention can improve the accuracy of highway pavement detection. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic flowchart of the highway pavement detection method based on image recognition according to the present invention. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] The purpose of this invention is to provide a road surface detection method based on image recognition, which can improve the accuracy of road surface detection.
[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0031] like Figure 1 As shown, the present invention provides a highway pavement detection method based on image recognition, comprising:
[0032] Step 100: Set up an image acquisition system on the carrier and acquire images of the target area. The image acquisition system uses a device with both camera and transmission capabilities.
[0033] Step 200: Binarize and crop the target region image to obtain multiple grayscale image blocks.
[0034] Step 300: Input each grayscale image block into the segmentation model for foreground and background segmentation to obtain a coarsely processed image; the segmentation model is constructed based on the Mask R-CNN network architecture and local region dynamic calculation.
[0035] Step 400: Input each of the coarsely processed images into the anomaly detection model to determine the anomaly location in the target region; the anomaly detection model includes a fusion module and multiple anomaly localization modules connected to the fusion module; each set of anomaly localization modules includes a feature enhancement subnet and an anomaly localization subnet connected in sequence.
[0036] As a specific implementation method, the target region image is binarized and cropped, and the specific process includes:
[0037] The target region image is converted to grayscale, and the grayscale image is then subjected to median filtering for noise reduction. The image after noise reduction is then cropped to obtain multiple grayscale image blocks.
[0038] As a specific implementation method, each grayscale image block is input into a segmentation model for foreground and background segmentation. The specific process includes:
[0039] Based on the Mask R-CNN network, an adaptive threshold is used to dynamically calculate local regions for each grayscale image patch, and the foreground region in each image patch is extracted to obtain a coarsely processed image; the foreground region is the road surface in the image. The adaptive threshold includes local mean and Gaussian weighting; the local mean is used to handle uneven illumination, and the Gaussian weighting is used to preserve edge details.
[0040] As a specific implementation, before inputting each of the coarsely processed images into the anomaly detection model, the method further includes: training the anomaly detection model, wherein the specific training steps are as follows:
[0041] Acquire training data; the training data includes training images and corresponding detection labels; the detection labels include whether the image has an anomaly and the location when the image has an anomaly; construct a pre-trained network based on Faster R-CNN and attention mechanism; input the training data into the pre-trained network, train it with the goal of minimizing the loss between the network output and the detection label, and determine the trained network as the final anomaly detection model.
[0042] As a specific implementation method, the coarsely processed images are input into the anomaly detection model, and the specific process includes:
[0043] After inputting the coarsely processed images into the anomaly detection model, the corresponding image blocks are first processed in each anomaly localization module to obtain the anomalies in each image block and determine the anomaly localization boxes. Then, a fusion algorithm is used to stitch together each image block and its corresponding anomaly localization box to determine the anomaly location in the target region. Specifically, the processing of the corresponding image blocks in each anomaly localization module includes:
[0044] In each anomaly localization module, histogram equalization, contrast stretching, and Fourier transform are used to enhance the image features of the corresponding input image blocks. The enhanced image blocks are then input into the anomaly localization submodule for detection, and the detected anomaly locations are marked using localization boxes.
[0045] Based on the above technical solution, the following embodiments are provided.
[0046] Step 100: Image Acquisition System Setup and Image Acquisition
[0047] Equipment Selection: Industrial cameras with high-definition video recording and wireless transmission capabilities will be selected as image acquisition devices and installed on traffic monitoring poles or drone carriers to ensure coverage of the target highway area.
[0048] Image Acquisition: The camera is controlled by a preset program to continuously shoot at a rate of 30 frames per second, and the images are transmitted to the backend server in real time using H.264 encoding. A 10-minute video clip with stable lighting conditions (such as midday) and no obstructions is selected, and key frames are extracted as images of the target area and saved in JPEG format.
[0049] Step 200: Binarization and cropping
[0050] Grayscale conversion: Convert the target area image from RGB color space to grayscale space, and calculate the pixel grayscale value using the formula Gray = 0.299R + 0.587G + 0.114B.
[0051] Median filtering for noise reduction: A 5×5 window median filter is applied to effectively eliminate salt-and-pepper noise and random impulse interference in the image while preserving road surface texture details.
[0052] Dynamic threshold binarization: A local adaptive thresholding algorithm is used to calculate the local mean of the 15×15 neighborhood around each pixel, and Gaussian weighting (σ=2.5) is combined to smooth the edge region to generate a binarized mask.
[0053] Intelligent cropping: Based on the binarization results, the main area of the road surface is located through connected component analysis, and the minimum bounding rectangle is determined by the convex hull algorithm, automatically cropping out multiple 512×512 pixel grayscale image blocks.
[0054] Step 300: Foreground and background segmentation based on Mask R-CNN
[0055] Model Architecture: A Mask R-CNN model is constructed using ResNet-50 as the backbone network and combined with a Feature Pyramid Network (FPN). An adaptive thresholding module is added after the Region Proposal Network (RPN) to dynamically adjust the anchor box generation strategy.
[0056] Local dynamic calculation:
[0057] Illumination compensation: Calculate the local mean for each image patch, construct an illumination compensation matrix, and correct overexposed or underexposed areas.
[0058] Edge enhancement: Apply Gaussian weighting (kernel size 7×7, σ=1.0) to the image patch to enhance the road surface edge features.
[0059] Foreground extraction: The model output contains a binary mask of the road surface area. Small holes are eliminated by morphological closing operation (kernel size 5×5) to obtain a coarsely processed image.
[0060] Step 400: Anomaly Detection and Localization
[0061] Model training:
[0062] Training data: 10,000 images containing road surface anomalies such as cracks and potholes were collected, and the locations of anomalies (bounding boxes) were marked using the LabelImg tool.
[0063] Network construction: Based on Faster R-CNN, a multi-head self-attention mechanism (Transformer Encoder) is introduced after the feature extraction layer to enhance global feature association.
[0064] Loss function: Smooth L1 loss is used to calculate the positioning error, and Focal Loss is used to handle the class imbalance problem.
[0065] Anomaly detection process:
[0066] Feature enhancement: Perform the following sequentially on the coarsely processed image:
[0067] Histogram equalization: Expands the contrast range to the [0.2, 0.8] interval.
[0068] Contrast stretching: A linear transformation is used to map pixel values to the [30, 220] range.
[0069] Frequency domain filtering: Low-frequency components are preserved (cutoff frequency 30Hz) and high-frequency noise is suppressed by Fourier transform.
[0070] Anomaly localization: The enhanced image is input into the anomaly localization submodule (YOLOv5s), and the output is a predicted bounding box containing the anomaly location and confidence level.
[0071] Results fusion: The non-maximum suppression (NMS) algorithm is used to merge overlapping boxes, and the detection results of each image patch are stitched together using a weighted fusion strategy (weight coefficient 0.7) to generate a global anomaly distribution map.
[0072] Implementation effect
[0073] Processing efficiency: Average processing time per image is 320ms (accelerated by NVIDIA Tesla T4 GPU).
[0074] Detection accuracy: Achieved anomaly recall of 92.3% and localization accuracy of 88.5% on the test set.
[0075] Application scenario: Deployed in the intelligent inspection system of highways, it can detect abnormal conditions such as road damage and obstacles in real time, and push the early warning information to the operation and maintenance platform through the 5G network.
[0076] Therefore, the present invention has the following specific beneficial effects:
[0077] This method first constructs a multispectral image acquisition system on a mobile platform, achieving multi-dimensional information acquisition of road surface texture and temperature field distribution through the collaborative operation of a high-resolution visible light camera and an infrared thermal imager. In the image processing stage, an adaptive threshold binarization algorithm combined with a dynamic ROI cropping strategy is employed to effectively remove irrelevant background interference and generate a set of grayscale image patches with uniform scale characteristics.
[0078] The core detection model consists of a two-stage deep neural network: the first stage employs an improved Mask R-CNN architecture for coarse-grained foreground and background segmentation. This model introduces a local region dynamic computation mechanism, enabling the network to adaptively focus on suspected disease areas, significantly reducing computational cost while maintaining segmentation accuracy. The second stage constructs a deep anomaly detection network with multiple parallel anomaly localization modules. Each module includes a feature enhancement subnetwork and an anomaly localization subnetwork. The feature enhancement subnetwork uses densely connected convolutional modules to enhance the expression of subtle disease features through cross-level feature fusion, while the anomaly localization subnetwork uses a spatial attention mechanism to achieve pixel-level anomaly probability prediction. The outputs of multiple modules are integrated and decided through a feature fusion layer, effectively overcoming the problem of insufficient sensitivity of a single model to specific disease types.
[0079] This method achieves scalability of the detection process through modular design, allowing each sub-model to be independently optimized and updated. At the same time, it uses knowledge distillation technology to achieve lightweight deployment of the model. While maintaining a detection accuracy of over 92%, the inference speed meets the requirements of real-time detection, providing an efficient and accurate technical solution for the intelligent maintenance of large-scale highway infrastructure.
[0080] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0081] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for detecting highway pavement based on image recognition, characterized in that, include: An image acquisition system is built on a carrier to acquire images of the target area; The target region image is binarized and cropped to obtain multiple grayscale image blocks; Each grayscale image block is input into a segmentation model for foreground and background segmentation to obtain a coarsely processed image. The segmentation model is built based on the Mask R-CNN network architecture and dynamic calculation of local regions; Each coarsely processed image is input into an anomaly detection model to determine the anomaly location in the target region; the anomaly detection model includes a fusion module and multiple anomaly localization modules connected to the fusion module; each anomaly localization module includes a feature enhancement subnet and an anomaly localization subnet connected in sequence.
2. The highway pavement detection method based on image recognition according to claim 1, characterized in that, The image acquisition system uses equipment with both camera and transmission capabilities.
3. The highway pavement detection method based on image recognition according to claim 1, characterized in that, The target region image is binarized and cropped, and the specific process includes: The target region image is converted to grayscale, and the grayscale image is then subjected to median filtering for noise reduction. The image after noise reduction is then cropped to obtain multiple grayscale image blocks.
4. The highway pavement detection method based on image recognition according to claim 1, characterized in that, Each grayscale image block is input into a segmentation model for foreground and background segmentation. The specific process includes: Based on the Mask R-CNN network, local region dynamic calculation is performed on each grayscale image block using an adaptive threshold, and the foreground region in each image block is extracted to obtain a coarsely processed image; the foreground region is the road surface in the image.
5. The highway pavement detection method based on image recognition according to claim 4, characterized in that, The adaptive threshold includes a local mean and a Gaussian weight; wherein the local mean is used to handle uneven illumination, and the Gaussian weight is used to preserve edge details.
6. The highway pavement detection method based on image recognition according to claim 1, characterized in that, Before inputting the coarsely processed images into the anomaly detection model, the method further includes: training the anomaly detection model, specifically the following training steps: Acquire training data; the training data includes training images and corresponding detection labels; the detection labels include whether the image has an anomaly and the location where the image has an anomaly. A pre-trained network was built based on Faster R-CNN and an attention mechanism; The training data is input into the pre-trained network, and training is performed with the goal of minimizing the loss between the network output and the detection label. The trained network is then determined as the final anomaly detection model.
7. The highway pavement detection method based on image recognition according to claim 1, characterized in that, The coarsely processed images are input into the anomaly detection model. The specific process includes: After inputting the coarsely processed images into the anomaly detection model, the corresponding image blocks are first processed in each anomaly localization module to obtain the anomalies in each image block and determine the anomaly localization box. Then, the fusion algorithm is used to stitch together each image block and the corresponding anomaly localization box to determine the anomaly location in the target area.
8. The highway pavement detection method based on image recognition according to claim 7, characterized in that, In each anomaly localization module, the corresponding input image patch is processed. The specific process includes: In each anomaly localization module, histogram equalization, contrast stretching, and Fourier transform are used to enhance the image features of the corresponding input image blocks. The enhanced image blocks are then input into the anomaly localization submodule for detection, and the detected anomaly locations are marked using localization boxes.