Road surface material identification method in driving scene
By using a lightweight semantic segmentation model and a convolutional neural network structure with a dual-branch semantic segmentation head, the problems of high hardware cost and high computational complexity in road material recognition in intelligent driving are solved, achieving high-precision and low-latency road material classification, which is suitable for vehicle embedded platforms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU ARCVIDEO TECHNOLOGY CO LTD
- Filing Date
- 2025-12-19
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies for road surface material recognition in intelligent driving suffer from high hardware costs, high computational complexity, large number of parameters, and limited resources, making it difficult to achieve high-precision, low-latency road surface material recognition on in-vehicle embedded platforms.
A lightweight semantic segmentation model is adopted, and a convolutional neural network structure with a dual-branch semantic segmentation head is designed. By collecting image data and constructing a semantic segmentation dataset, the road surface material segmentation model is trained. The intersection calculation is performed by combining the drivable area and the road surface material segmentation mask, and the road surface category is determined by voting count, thus realizing end-to-end road surface material recognition.
It achieves high-precision, low-latency road surface material classification with limited computing power, meets the real-time performance and energy efficiency requirements of intelligent driving systems, reduces computational complexity and the number of parameters, and is suitable for mainstream automotive-grade chips.
Smart Images
Figure CN121937862A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for recognizing road surface materials in driving scenarios. Background Technology
[0002] In intelligent driving and advanced driver assistance systems (ADAS), accurate identification of road surface materials (such as asphalt, concrete, gravel, ice, snow, or dirt roads) is crucial for vehicle dynamics control, braking strategy optimization, and driving safety. Traditional road surface material identification methods often rely on high-resolution images and multimodal sensor fusion (such as LiDAR and millimeter-wave radar). While these methods offer good recognition accuracy, they suffer from high hardware costs. Other solutions that rely solely on visual images typically employ multi-stage cascaded deep neural network models, which suffer from high computational complexity, a large number of parameters, and high memory consumption, posing a significant challenge to resource-constrained automotive embedded platforms. Currently, the computing power of mainstream automotive chips is relatively low compared to GPU platforms, and they need to run multiple critical modules such as perception, planning, and control simultaneously, resulting in extremely tight budgets for computing power and power consumption. Summary of the Invention
[0003] In view of the above-mentioned problems, the present invention provides a road surface material recognition method in driving scenarios, which can perform road surface material recognition with low latency, low power consumption and high reliability.
[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A method for identifying road surface material in a driving scenario includes the following steps: S10: Collect and label image data to construct a semantic segmentation dataset for road surface materials; S20, Use the constructed segmentation dataset to train the road surface material segmentation model; S30, input the image from the video stream to be detected to obtain the drivable region segmentation mask and the road surface material segmentation mask of the current image frame; take the intersection of the drivable region mask and the road surface material segmentation mask to obtain the final road surface material segmentation mask; calculate the road surface material category with the largest current area ratio; when the area ratio exceeds a certain threshold, take the current road surface category as the road surface material category of the image frame. S40, for driving scenarios, uses a road surface material segmentation model to determine the road surface material category of the current driving segment; sets a judgment time threshold, and counts the road surface category results of multiple consecutive frames; after the judgment time threshold is reached, a vote count is performed, and the road surface category with the highest percentage and a vote ratio exceeding the fixed threshold is the road surface category of the current driving segment.
[0005] Preferably, in S10, the road surface material semantic segmentation dataset includes three parts: the original road surface image, the drivable area mask annotation image, and the road surface material category mask annotation; that is, each original road surface image corresponds to two mask annotation images, one of which is a binary image that only distinguishes between the foreground and the background, where the foreground is the drivable area and the rest is the background; the other mask image includes grayscale images of the background and the road surface material category, where the road surface material category is a variety of road surfaces.
[0006] Preferably, in S20, the feature extraction module of the road surface material segmentation model is a lightweight semantic segmentation model feature extraction module; the output head is a dual semantic segmentation head, one for drivable area segmentation and the other for road surface material segmentation; the road surface material segmentation model training process includes: preprocessing the labeled road surface material segmentation training sample images and inputting them into the feature extraction module of the road surface material segmentation model to obtain the output feature tensor; the feature tensor is further input into the semantic segmentation head for drivable area segmentation and the semantic segmentation head for road surface material category segmentation; using the known drivable area mask annotation images and road surface material category mask annotations, the model network parameters are optimized using the cross-entropy CE loss function and the binary cross-entropy BCE loss function to reduce the value of the loss function and make it continuously approach 0.
[0007] Preferably, the two semantic segmentation heads have the same structure, and the final output is a segmentation result array of size C*H*W, where H and W represent the height and width of the mask image, respectively, and channel C represents the number of categories.
[0008] Preferably, for the drivable area segmentation head, C is 2, representing the ability to distinguish between the background and the drivable area.
[0009] Preferably, for the road surface material segmentation head, the size of C represents the type of road surface material. Five types of road surfaces are selected as the road surfaces to be identified: concrete road, asphalt road, dirt road, gravel road, and mud road. That is, the size of C is set to 5.
[0010] The present invention has the following advantages: it can be used in automotive embedded platforms, and by designing a convolutional neural network structure with a dual-branch semantic segmentation head, it can retain key texture and semantic features while significantly reducing the number of parameters and computational complexity; it enables the model to achieve high-precision and low-latency road material classification with limited computing power; it can be deployed end-to-end on mainstream automotive-grade chips without relying on external sensors or cloud computing, effectively meeting the requirements of intelligent driving systems for real-time performance, energy efficiency and functional safety. Attached Figure Description
[0011] Figure 1 This is a flowchart illustrating the steps of a road surface material identification method in a driving scenario according to an embodiment of the present invention. Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] See Figure 1 The diagram shows a flowchart of a road surface material identification method in a driving scenario according to an embodiment of the present invention, including the following steps: S10: Collect and label image data to construct a semantic segmentation dataset for road surface materials; S20, Use the constructed segmentation dataset to train the road surface material segmentation model; S30, input the image from the video stream to be detected to obtain the drivable region segmentation mask and the road surface material segmentation mask of the current image frame; take the intersection of the drivable region mask and the road surface material segmentation mask to obtain the final road surface material segmentation mask; calculate the road surface material category with the largest current area ratio; when the area ratio exceeds a certain threshold, take the current road surface category as the road surface material category of the image frame; optionally, the area threshold can be set to 0.7.
[0014] S40, for driving scenarios, uses a road surface material segmentation model to determine the road surface material category of the current driving segment; sets a judgment time threshold, and counts the road surface category results of multiple consecutive frames; after the judgment time threshold is reached, a vote count is performed, and the road surface category with the highest percentage and a vote ratio exceeding a fixed threshold is the road surface category of the current driving segment. Optionally, the judgment time threshold can be set to 10 seconds, and the vote ratio threshold can be set to 0.8.
[0015] In a specific application example, in S10, image data acquisition sources include, but are not limited to, on-site shooting data and open-source datasets from the internet. Excluding already labeled open-source datasets, the remaining data is labeled using Xlabel annotation software, pre-loading open-source large-scale model semantic segmentation algorithms, including but not limited to the Segment Anything large-scale model, to obtain the contour information of the drivable area and the contour information of different road surface materials in the current image, followed by manual secondary correction. Based on the labeled drivable area contour information, a drivable area mask image is calculated; based on the labeled contour information of different road surface materials, a road surface material mask information is calculated. The road surface material semantic segmentation dataset includes three parts: the original road surface image, the drivable area mask annotation image, and the road surface material category mask annotation. That is, each original road surface image corresponds to two mask annotation images. One mask image is a binary image that only distinguishes between the foreground and background, where the foreground is the drivable area and the rest is the background; the other mask image includes grayscale images of the background and the road surface material category, where the road surface material category can be multiple categories. The road surface category can be concrete road, asphalt road, dirt road, gravel road, mud road, etc.
[0016] In a specific application example, in S20, the feature extraction module of the road surface material segmentation model is a lightweight semantic segmentation model feature extraction module; the recognition model feature extraction module can use the lightweight semantic segmentation model PIDnet, and general semantic segmentation algorithms such as BiSeNet and SFNet can also be used. The output head is a dual semantic segmentation head, one for drivable area segmentation and the other for road surface material segmentation; the training process of the road surface material segmentation model includes: preprocessing the labeled road surface material segmentation training sample images and inputting them into the feature extraction module of the road surface material segmentation model to obtain the output feature tensor; this feature tensor is further input into the semantic segmentation head for drivable area segmentation and the semantic segmentation head for road surface material category segmentation simultaneously; using the known drivable area mask annotation images and road surface material category mask annotations, the model network parameters are optimized using the cross-entropy CE loss function and the binary cross-entropy BCE loss function to reduce the value of the loss function and make it continuously approach 0.
[0017] Furthermore, the two semantic segmentation heads have identical structures, ultimately outputting a segmentation result array of size C*H*W, where H and W represent the height and width of the mask image, respectively, and C represents the number of categories. For the drivable area segmentation head, C is 2, indicating that it can distinguish between the background and the drivable area. For the road material segmentation head, C represents the number of road materials, selecting five types—concrete road, asphalt road, dirt road, gravel road, and mud road—as the road surfaces to be identified, i.e., C is set to 5. This dual-branch segmentation head method reduces the influence of the background in non-driving areas on the various road material categories in the foreground, allowing the model material segmentation head to focus more on the road surface area, thus improving the accuracy of road material recognition.
[0018] The road surface material recognition method for driving scenarios described above can simultaneously complete two tasks: drivable area segmentation and road surface material recognition. Compared with multi-model cascade methods that require first identifying the drivable area and then recognizing the road surface material, this method significantly reduces the computational load. Furthermore, a semantic segmentation model is used to achieve road material category recognition, providing corresponding segmentation results for each road surface region on complex mixed road surfaces. In contrast, classification models can only output one result regardless of whether the road surface is single or complex, and complex mixed road surfaces can affect the classification results of the classification model. Therefore, the segmentation model can achieve higher recognition accuracy on complex road surfaces. Ultimately, this invention achieves low latency, low power consumption, and high reliability.
[0019] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.
Claims
1. A method for identifying road surface material in a driving scenario, characterized in that, Includes the following steps: S10: Collect and label image data to construct a semantic segmentation dataset for road surface materials; S20, use the constructed segmentation dataset to train the road surface material segmentation model; S30: Input the image from the video stream to be detected to obtain the drivable region segmentation mask and the road surface material segmentation mask of the current image frame; take the intersection of the drivable region mask and the road surface material segmentation mask to obtain the final road surface material segmentation mask. Calculate the road surface material category with the largest current area proportion; when the area proportion exceeds a certain threshold, use the current road surface category as the road surface material category of the frame image; S40, for driving scenarios, uses a road surface material segmentation model to determine the road surface material category of the current driving segment; Set a judgment time threshold and count the road surface category results of multiple consecutive frames; after the judgment time threshold is reached, vote counting is performed, and the road surface category with the highest percentage and the vote ratio exceeding the fixed threshold is the road surface category of the current driving segment.
2. The method for identifying road surface material in a driving scenario as described in claim 1, characterized in that, In S10, the road surface material semantic segmentation dataset includes three parts: the original road surface image, the drivable area mask annotation image, and the road surface material category mask annotation. That is, each original road surface image corresponds to two mask annotation images. One mask image is a binary image that only distinguishes between the foreground and the background, where the foreground is the drivable area and the rest is the background. The other mask image includes grayscale images of the background and the road surface material category, where the road surface material category includes multiple types of road surfaces.
3. The method for road surface material recognition in a driving scenario as described in claim 2, characterized in that, In S20, the feature extraction module of the road surface material segmentation model is a lightweight semantic segmentation model feature extraction module. The output head is a dual semantic segmentation head, one for drivable area segmentation and the other for road surface material segmentation; the road surface material segmentation model training process includes: preprocessing the labeled road surface material segmentation training sample images, inputting them into the feature extraction module of the road surface material segmentation model, and obtaining the output feature tensor; This feature tensor is further input into both the semantic segmentation head for drivable area segmentation and the semantic segmentation head for road surface material category segmentation. Using known drivable area mask annotation images and road surface material category mask annotations, the model network parameters are optimized using the cross-entropy (CE) loss function and the binary cross-entropy (BCE) loss function to reduce the value of the loss function and make it continuously approach 0.
4. The method for identifying road surface material in a driving scenario as described in claim 3, characterized in that, The two semantic segmentation heads have the same structure, and the final output is a segmentation result array of size C*H*W, where H and W represent the height and width of the mask image, respectively, and C represents the number of categories.
5. The method for road surface material recognition in a driving scenario as described in claim 4, characterized in that, For the drivable area segmentation head, C is 2, which means that the background and drivable area can be distinguished.
6. The method for road surface material recognition in a driving scenario as described in claim 4, characterized in that, For the road surface material segmentation head, the size of C represents the type of road surface material. Five types of road surfaces are selected as the road surfaces to be identified: concrete road, asphalt road, dirt road, gravel road, and mud road. That is, the size of C is set to 5.