Road surface meteorological state image segmentation method based on YOLOv8-BRFA model

Abstract

The invention discloses a road surface meteorological state image segmentation method based on a YOLOv8-BRFA model. The method comprises the following steps: acquiring a near-infrared road surface meteorological image data set; designing a dynamic sparse attention mechanism and a hardware perception neural network based on a YOLOv8 network architecture, and constructing a road surface meteorological state segmentation model in combination with an adaptive decoupling detection head; training the road surface meteorological state segmentation model by using a near-infrared road surface meteorological image data set to obtain a trained road surface meteorological state segmentation model; and obtaining a to-be-detected road surface meteorological image, inputting the to-be-detected road surface meteorological image into the trained road surface meteorological state segmentation model, and carrying out road surface meteorological state identification and segmentation to obtain a positioning and segmentation result of the to-be-detected road surface meteorological image. According to the invention, through structure improvement, the calculation complexity is reduced while the precision of the image segmentation model is improved, and real-time and accurate segmentation and detection of the road surface meteorological state under the complex illumination condition are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning combined with road surface weather, and particularly relates to a road surface weather state image segmentation method based on a YOLOv8-BRFA model. BACKGROUND

[0002] The road surface weather state is a key factor affecting road traffic safety. Weather conditions such as rain, snow, ice, and water accumulation can significantly reduce the road anti-skid performance and increase the risk of traffic accidents. Statistical data shows that traffic accidents on highways under adverse weather conditions account for 50% of the total number of accidents in China, and the proportion of major accidents is as high as 71%. Snowy weather even leads to an 84% increase in accident rate and a 75% increase in injury rate. Therefore, it is of great significance to realize high-precision and real-time detection of road surface weather state for traffic safety warning and road maintenance.

[0003] Current road surface weather detection technologies mainly include contact and non-contact methods. The contact method requires burying sensors on the road surface, which not only damages the road structure, but also is easily disturbed by the environment and has high maintenance costs. Non-contact methods, such as radar, light polarization, and infrared spectroscopy, avoid physical contact, but still have problems such as high hardware costs, signal being greatly affected by the environment, and difficulty in quantitative detection. In recent years, road surface detection methods based on machine vision have gradually become a research hotspot due to their non-contact, wide coverage, and rich information. For example, Jonsson et al. combined multi-wavelength laser and short-wave infrared cameras with SVM and KNN methods to identify road surface states. Casselgren used a multi-wavelength detector to obtain backscattered light intensity and set a threshold for classification. Nolte et al. used CNN to classify dry, wet, and snowy road surface states. However, existing visual methods still have obvious shortcomings: first, in visible or near-infrared imaging, water accumulation and ice accumulation areas on the road surface can cause specular reflection and uneven lighting, resulting in local overexposure or loss of details in the image; second, under complex lighting conditions such as weak light and backlight, the feature extraction capability of traditional convolutional neural networks is insufficient, especially for capturing subtle features such as water accumulation edges and thin ice textures; third, although the existing real-time detection model YOLOv8 achieves a good balance between speed and accuracy, it still has problems such as large parameter quantity and high computational complexity in the multi-wavelength near-infrared scene, making it difficult to meet the needs of edge device deployment and real-time processing.

[0004] Therefore, under the condition of multi-wavelength near-infrared imaging, how to achieve high-precision real-time segmentation of multiple road bases and multiple weather states is a problem that needs to be solved. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a road surface weather state image segmentation method based on a YOLOv8-BRFA model, which introduces a dynamic sparse attention mechanism, a hardware perception neural network and a self-adaptive decoupling detection head, thereby improving the precision of the image segmentation model while reducing the computational complexity, and realizing real-time and accurate segmentation and detection of the road surface weather state under complex lighting conditions.

[0006] To solve the above technical problems, the present application adopts the following technical solutions:

[0007] A road surface weather state image segmentation method based on a YOLOv8-BRFA model, comprising the following steps:

[0008] Collecting a near-infrared road surface weather image dataset;

[0009] Based on a YOLOv8 network framework, a dynamic sparse attention mechanism and a hardware perception neural network are designed, and a self-adaptive decoupling detection head is combined to construct a road surface weather state segmentation model;

[0010] The road surface weather state segmentation model is trained using the near-infrared road surface weather image dataset to obtain a trained road surface weather state segmentation model;

[0011] Obtaining a road surface weather image to be detected, inputting the road surface weather image to be detected into the trained road surface weather state segmentation model, performing road surface weather state recognition and segmentation, and obtaining the positioning and segmentation results of the road surface weather image to be detected.

[0012] As a preferred solution, before training the road surface weather state segmentation model, the following steps are further included: after removing specular reflection from the near-infrared road surface weather image dataset using a mathematical morphology algorithm, performing a preprocessing operation of uniformizing illumination based on the Retinex theory, combining gamma transformation and image difference.

[0013] As a preferred solution, the road surface weather state segmentation model comprises a backbone network fused with a dynamic sparse attention mechanism, a neck network fused with a hardware perception neural network, and a detection head network with self-adaptive decoupling; the trained road surface weather state segmentation model takes the preprocessed near-infrared road surface weather image dataset as the input of the backbone network, extracts deep feature maps through the dynamic sparse attention mechanism, then uses the hardware perception neural network of the neck network to perform multi-scale feature enhancement and fusion on the deep feature maps, generates robust multi-scale feature maps and inputs them into the detection head network with self-adaptive decoupling, classifies and locates through task decoupling and distributed focal loss, and finally obtains the positioning and segmentation results of the road surface weather state as the output of the road surface weather state segmentation model.

[0014] As a preferred solution, the backbone network comprises a convolutional layer, four dynamic attention feature extraction modules and an SPPF module connected in sequence; the dynamic attention feature extraction module comprises a convolutional layer and a C2f_biformer module connected in cascade, wherein the C2f_biformer module comprises a deep separable convolutional layer, a first layer normalization layer, a double-layer routing attention module, a second layer normalization layer and an MLP layer connected in cascade;

[0015] In the C2f_biformer module, after the input feature is extracted by the deep separable convolutional layer, it is spliced with the input of the deep separable convolutional layer as the input of the first layer normalization layer, the output of the layer normalization layer is input into the double-layer routing attention module for multi-level feature extraction, and then it is spliced with the input of the first layer normalization layer as the input of the second layer normalization layer, the output of the second layer normalization layer is input into the MLP module, and the output of the MLP module is spliced with the input of the second layer normalization layer as the final output of the C2f_biformer module.

[0016] As a preferred solution, the processing process of the double-layer routing attention module specifically comprises:

[0017] Firstly, the feature map is divided into SxS attention regions, and query matrix, key matrix and value matrix are generated by linear transformation; then, the attention regions are dynamically sparsified and screened by a double-layer routing mechanism, the query matrix and the key matrix of each region are respectively averaged and pooled to obtain the regional query and key representation, and an inter-regional affinity adjacency matrix is constructed, and the Top-k screening is used to retain the most relevant k target regions of each region to form a routing index matrix; subsequently, the corresponding key-value pairs of the target regions are aggregated based on the routing index matrix, and are converted into a dense matrix form, the aggregated key-value pairs and the original query are calculated by fine-grained token-level attention to generate a sparse attention weight matrix, and a depth separable convolution is introduced to further enhance the local context information; finally, the sparse attention weight matrix and the aggregated value are weighted and fused, and the output feature is restored to the original spatial size to obtain an output feature with structural sparsity.

[0018] As a preferred solution, the neck network comprises two reparameterization transpose fusion modules and two reparameterization splicing modules connected in sequence; the reparameterization transpose fusion module comprises a SimConv layer, a Transpose layer, a connection layer and a reparameterization module connected in cascade, the reparameterization splicing module comprises a SimConv layer, a connection layer and a reparameterization module connected in cascade; the reparameterization module is composed of a plurality of RepConv modules connected in cascade;

[0019] The output of the backbone network is taken as the input of the first reparameterization transposed fusion module, and the outputs of the second and third dynamic attention feature extraction modules in the backbone network are transmitted to the connection layers in the first and second reparameterization transposed fusion modules in the neck network respectively to perform splicing fusion operation.

[0020] As a preferred solution, the detection head network comprises three parallel AsDDet detection heads; the AsDDet detection head comprises a classification branch and a regression branch connected in parallel; the classification branch comprises a cascaded 3x3 deep separable convolution layer, a 3x3 convolution layer, a 1x1 convolution layer and a Sigmoid activation function, and the regression branch comprises a first 3x3 convolution layer, a channel rearrangement unit, a second 3x3 convolution layer, a 1x1 convolution layer, a distribution focal loss module and a bounding box decoding function.

[0021] The output of the second reparameterization transposed fusion module in the neck network and the outputs of the two reparameterization splicing modules are taken as the inputs of the three AsDDet detection heads in the detection head network respectively.

[0022] As a preferred solution, the processing process of the AsDDet detection head specifically comprises:

[0023] In the AsDDet detection head, the input feature map is processed in parallel through the decoupled classification branch and the regression branch; in the classification branch, the input feature map is subjected to 3x3 deep separable convolution layer to extract local detailed features, and then subjected to 3x3 convolution layer to further fuse semantic information, and then subjected to 1x1 convolution layer to map the channel number to the category dimension, and finally subjected to Sigmoid activation function to output the classification probability corresponding to each position; in the regression branch, the input feature map is first subjected to the first 3x3 convolution layer to extract positioning features, and then subjected to channel mixing operation to enhance feature interaction capability, and then subjected to the second 3x3 convolution and 1x1 convolution in sequence for feature refining and channel adjustment, and then subjected to the distribution focal loss module for probability distribution modeling, and finally subjected to the bounding box decoding function to convert and output the probability distribution to obtain the bounding box coordinates; the classification probability and the bounding box coordinates are combined, and finally the positioning and segmentation result of the road weather image as the output of the road weather state segmentation model is output.

[0024] Compared with the prior art, the present application has the following technical effects:

[0025] 1、The application introduces a dynamic sparse attention mechanism (C2f-Biformer) module in the backbone network, which greatly reduces the computational complexity through a double-layer routing attention (BRA) module, extracts low-computing features through a depth separable convolution, enhances the generalization ability of the model through a layer normalization layer, adjusts the model parameters through a multilayer perceptron (MLP) layer, and extracts features at multiple levels through a double-layer attention module, so that the model can effectively enhance the perception ability of the segmentation region. In addition, the pyramid model (SPFF) structure enhances the feature fusion ability of the model, and through this sparse computing strategy, the interference from irrelevant regions is effectively reduced, thereby enhancing the robustness and accuracy of the model in processing the road weather segmentation task.

[0026] 2、The application adopts a hardware perception neural network (RepFPN) in the neck network, optimizes the feature expression ability of the model in a weak light and non-uniform light environment through a reparameterization design and a multi-level feature fusion mechanism, and effectively suppresses the interference of overexposure and underexposure on detection.

[0027] 3、The detection head network of the application adopts an adaptive decoupled detection head (AsDDet), and the AsDDet detection head adopts a task decoupled design, which completely separates the classification and regression branches: the classification branch extracts local detailed features through a relevant separable convolution (DWConv), and the regression branch introduces a distributed focus loss (DFL) in the decoding stage instead of a traditional coordinate regression, converts the probability distribution into continuous coordinates, and finally combines the distribution expectation value with the preset anchor point through a bounding box decoding function (dist2bbox) to generate the decoded bounding box coordinates. The AsDDet detection head based on task decoupling and DFL design makes the classification and regression tasks independent optimization, further improves the positioning accuracy and classification robustness, and especially has stronger adaptability in the edge blur and uneven light road weather scene. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings, in which:

[0029] Figure 1 The application discloses a road weather state image segmentation method based on a YOLOv8-BRFA model;

[0030] Figure 2 The application discloses a road weather state image segmentation method based on a YOLOv8-BRFA model;

[0031] Figure 3 The application discloses a road weather state image segmentation method based on a YOLOv8-BRFA model;

[0032] Figure 4(a) is the overall architecture diagram of the C2f_biformer module of the embodiment of the present application;

[0033] Figure 4 (b) is the structure schematic diagram of the BRA of the embodiment of the present application;

[0034] Figure 5 the structure schematic diagram of the re-parameterization module of the embodiment of the present application;

[0035] Figure 6 the structure schematic diagram of the AsDDet detection head of the embodiment of the present application;

[0036] Figure 7 the physical diagram of the pavement weather sampling system of the embodiment of the present application;

[0037] Figure 8 the ice water medium reflectivity contrast diagram of the embodiment of the present application;

[0038] Figure 9 the pavement weather state part data set image of the embodiment of the present application;

[0039] Figure 10 the pavement weather segmentation diagram of each improved model and each subgrade of the embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0041] The present application will be further described in detail below with reference to the accompanying drawings.

[0042] The existing road surface weather detection method has significant limitations in dealing with complex lighting conditions: the traditional visual detection method is easily disturbed by mirror reflection and uneven lighting, causing local overexposure or underexposure of the image, which seriously affects the effectiveness of feature extraction; although the existing deep learning model performs well in general scenarios, it lacks the ability to capture key weather conditions such as water accumulation and icing in low-light, backlight or non-uniform lighting environments, and the model often has a large number of parameters and high computational cost, making it difficult to meet the real-time edge deployment requirements. In addition, the classification and regression tasks in the common detection head structure are coupled, which easily interferes with each other in complex scenes, further restricting the detection accuracy and robustness.

[0043] To this end, the present application proposes a road surface weather state image segmentation method based on YOLOv8-BRFA model, which reduces the model complexity and enhances the feature perception ability by introducing a dynamic sparse attention mechanism (C2f-Biformer), uses a hardware-aware neural network (RepFPN) to improve the robustness of multi-scale feature fusion in low-light environments, and designs an adaptive decoupling detection head (AsDDet) to realize the decoupling optimization of classification and regression tasks. The present application improves the detection accuracy while reducing the parameter quantity, realizing real-time and high-precision segmentation and identification of various road surface weather conditions such as water accumulation, icing and wetness.

[0044] Specifically, the road surface weather state image segmentation method based on YOLOv8-BRFA model proposed by the present application comprises the following steps: collecting near-infrared road surface weather image dataset;

[0045] Based on the YOLOv8 network framework, a dynamic sparse attention mechanism and a hardware-aware neural network are designed, and an adaptive decoupling detection head is combined to build a road surface weather state segmentation model;

[0046] The near-infrared road surface weather image dataset is used to train the road surface weather state segmentation model, and a trained road surface weather state segmentation model is obtained;

[0047] Obtain the road surface weather image to be detected, input the road surface weather image to be detected into the trained road surface weather state segmentation model, and perform road surface weather state recognition and segmentation to obtain the positioning and segmentation results of the road surface weather image to be detected.

[0048] The present application takes YOLOv8n model as the benchmark model, designs dynamic sparse attention mechanism, hardware-aware neural network, and introduces a new lightweight detection head-adaptive decoupling detection head, and then obtains YOLOv8-BRFA improved model.

[0049] The road surface weather state image segmentation method based on YOLOv8-BRFA model of the present application will be described in more detail below.

[0050] 1. Data preprocessing

[0051] As a spotlight, the halogen lamp emits from a point to a specific conical direction with attenuation. When the halogen light source irradiates the water and ice area, the problem of specular reflection and uneven illumination will occur, which will cause partial highlight area in the camera receiving end and lead to the shielding of key details in the image. To solve the problem, the embodiment proposes to solve the above problems based on the dilation and corrosion operation in mathematical morphology and the algorithm based on the Retinex theory framework.

[0052] 1) Mathematical morphology algorithm: convert the collected multi-channel base weather image from RGB color space to HSV space, and define the specular reflection area threshold as 100. In the highlight area mask, define the size of the structural element as Element, to prevent edge residue or incomplete repair in the subsequent repair process. Then, perform dilation operation on the highlight area mask. After obtaining the dilated area, further morphological reconstruction is performed, using the pixel values of the surrounding non-highlight area to replace the pixel values in the original highlight area, thereby restoring the image information covered by strong light. After the morphological algorithm, the influence of specular reflection on road weather detection can be removed.

[0053] 2) Retinex theory algorithm: due to the influence of factors such as camera performance, halogen light source and natural light environment, a gamma transformation and image difference illumination uniformization method is proposed to effectively solve the problem of local overexposure and underexposure of the image caused by uneven illumination distribution, thereby improving the accuracy of subsequent detection. After obtaining the enhanced result which is not affected by the original illumination distribution, the reconstructed image is subjected to linear normalization processing to make the pixel value distribution within the standard display range of 0-255, thereby completing the entire illumination uniformization process. The obtained entire image has consistent brightness, the uneven illumination of the original image is well corrected, and more image details of the road surface area to be segmented are retained.

[0054] The above data preprocessing can solve the illumination interference of the halogen light source on the road weather detection, as shown in Figure 2 .

[0055] 2. Road weather state segmentation model

[0056] YOLO algorithm has shown broad application and research prospects since its inception, breaking the limitations of the traditional two-stage algorithm (such as R-CNN series) that speed and accuracy are difficult to balance. In this embodiment, the YOLOv8n model is taken as the benchmark model, and the dynamic sparse attention mechanism (C2f_biformer), the hardware-aware neural network (RepFPN) are designed, and a new lightweight detection head, the adaptive decoupling detection head (AsDDet), is introduced, and then the road surface weather state segmentation model based on the improved YOLOv8-BRFA model is obtained.

[0057] In this embodiment, the road surface weather state segmentation model includes a backbone network fused with a dynamic sparse attention mechanism, a neck network fused with a hardware-aware neural network, and a detection head network with adaptive decoupling, and the model structure is as shown in Figure 3 As shown in the figure; in the backbone network, the introduction of the dynamic sparse attention mechanism can effectively enhance the perception ability of the segmentation area, in the neck network part, a hardware-aware neural network is proposed to increase the feature detection in the weak light environment. In the detection head part, a classification-enhanced adaptive decoupling detection head is used, and through task decoupling and distributed focal loss design, the classification and positioning ability of the detection head to the features is improved.

[0058] The specific processing process is as follows: the trained road surface weather state segmentation model takes the preprocessed near-infrared road surface weather image dataset as the input of the backbone network, extracts deep feature maps through the dynamic sparse attention mechanism, then uses the hardware-aware neural network of the neck network to perform multi-scale feature enhancement and fusion on the deep feature maps, generates robust multi-scale feature maps and inputs them into the adaptive decoupling detection head network, classifies and locates through task decoupling and distributed focal loss, and finally obtains the positioning and segmentation results of the road surface weather state, which are taken as the output of the road surface weather state segmentation model.

[0059] Next, each network of the model will be described in detail.

[0060] 2.1, backbone network

[0061] In this embodiment, the backbone network includes convolutional layers, four dynamic attention feature extraction modules and SPPF modules connected in turn; the dynamic attention feature extraction module includes cascaded convolutional layers and a C2f_biformer module, wherein the C2f_biformer module includes cascaded depth separable convolutional layers, a first layer normalization layer, a double-layer routing attention module, a second layer normalization layer and an MLP layer.

[0062] In implementation, the C2f_biformer module is a transformer-based visual network architecture composed of multiple sub-modules, including deep convolution, layer normalization, bi-layer routing attention (BRA), and MLP. The overall architecture of the C2f_biformer module is shown in Figure 4 (a), where the plus sign represents the merging of two feature vectors.

[0063] In the C2f_biformer module, after low-computational feature extraction of the input features by a deep separable convolution layer, the input of the deep separable convolution layer is spliced as the input of the first layer normalization layer to enhance the generalization ability. The output of the layer normalization layer is input into the bi-layer routing attention module for multi-level feature extraction, and then spliced with the input of the first layer normalization layer as the input of the second layer normalization layer. The output of the second layer normalization layer is input into the MLP module. The output of the MLP module is spliced with the input of the second layer normalization layer as the final output of the C2f_biformer module.

[0064] The processing process of the BRA module is shown in Figure 4 (b), which specifically includes:

[0065] First, the feature map is divided into SxS non-overlapping coarse-grained regions, and the query (Q), key (K), and value (V) matrices are generated by linear transformation. Then, the attention regions are dynamically sparsified and screened by a bi-layer routing mechanism. The Q and K of each region are averaged and pooled to obtain the regional-level query and key, and an inter-regional affinity adjacency matrix is constructed. The Top-k screening is used to retain the most relevant k target regions for each region to form a routing index matrix. Subsequently, the key-value pairs of the target regions are aggregated based on the index matrix and converted into a dense matrix form friendly to GPU. The aggregated key-value pairs and the original query are used for fine-grained token-level attention calculation to generate a sparse attention weight matrix. A depth separable convolution is introduced to further enhance the local context information. Finally, the sparse attention weight matrix and the aggregated value are weighted and fused, and the output features are restored to the original spatial size to obtain output features with structural sparsity.

[0066] The embodiment introduces a dynamic sparse attention mechanism (C2f-Biformer) module in the backbone network, which greatly reduces the computational complexity through a double-layer routing attention (BRA) module, extracts low-computational features through a depth separable convolution, enhances the generalization ability of the model through a layer normalization layer, adjusts the model parameters through a multi-layer perceptron (MLP) layer, and extracts features at multiple levels through a double-layer attention module, so that the model can effectively enhance the perception ability of the segmentation region. In addition, the pyramid model (SPFF) structure enhances the feature fusion ability of the model, and the sparse computing strategy effectively reduces the interference from irrelevant regions, thereby enhancing the robustness and accuracy of the model in processing the road weather segmentation task.

[0067] 2.2, neck network

[0068] In the embodiment, the neck network includes two re-parameterization transposed fusion modules and two re-parameterization splicing modules connected in sequence; the re-parameterization transposed fusion module includes a cascade of a SimConv layer, a Transpose layer, a connection layer, and a re-parameterization module, and the re-parameterization splicing module includes a cascade of a SimConv layer, a connection layer, and a re-parameterization module; the re-parameterization module is composed of a plurality of RepConv modules connected in cascade; wherein the output of the backbone network is taken as the input of the first re-parameterization transposed fusion module, and the outputs of the second and third dynamic attention feature extraction modules in the backbone network are transmitted to the connection layers in the first and second re-parameterization transposed fusion modules in the neck network for splicing fusion operation; the output of the SimConv layer in the first re-parameterization transposed fusion module in the neck network is also transmitted to the connection layer in the second re-parameterization splicing module for splicing fusion transmission, and the output of the SimConv layer in the second re-parameterization transposed fusion module is also transmitted to the connection layer in the first re-parameterization splicing module for splicing fusion transmission.

[0069] In specific implementation, although the traditional YOLOv8 feature pyramid structure can effectively fuse multi-scale features, in strong backlight, direct sunlight or weak light environment, the traditional FPN may not be able to effectively capture the surface features of weather conditions such as water accumulation and icing due to overexposure or underexposure, resulting in a decrease in the classification performance and detection accuracy of the segmentation model. Therefore, the embodiment introduces a hardware perception neural network (RepFPN) to improve the robustness of the model. The RepFPN mainly consists of Repblock, SimConv and Transpose.

[0070] In the inference process, the network combines multiple branches into one path, thereby improving the inference speed of the model, and a re-parameterization module (Repblock) is proposed in the structure of the model, as shown in Figure 5 Figure 5 ​(a) is a RepConv training graph, Figure 5 (b) is a RepConv inference graph, Figure 5 (c) is a Repblock structure.

[0071] Although the RepConv structure design helps to reduce the number of parameters of the model and improve the inference speed, a single RepConv module is not enough to effectively capture the characteristics of the edge area of the frozen water under complex light conditions, especially the edge area of the frozen water. Therefore, a Repblock is formed by cascading multiple RepConv modules, and the output of the previous module is used as the input of the next module. By cascading multiple RepConv modules, Repllock can gradually extract deeper semantic features while keeping the network lightweight, and enhance the feature representation ability of the model to complex scenes (such as weak light road surface images). This structure not only inherits the multi-branch optimization advantage of RepConv in training, but also realizes efficient calculation through structure fusion in inference, and is suitable for road surface weather state detection tasks with high real-time requirements.

[0072] In this embodiment, a hardware perception neural network (RepFPN) is used in the neck network, which optimizes the feature expression ability of the model in weak light and non-uniform light environment through reparameterization design and multi-level feature fusion mechanism, and effectively suppresses the interference of overexposure and underexposure on detection.

[0073] 2.3, detection head network

[0074] In this embodiment, the detection head network includes 3 parallel AsDDet detection heads; the AsDDet detection head includes a parallel classification branch and a regression branch; the classification branch includes a cascaded 3x3 deep separable convolution layer, a 3x3 convolution layer, a 1x1 convolution layer and a Sigmoid activation function, and the regression branch includes a first 3x3 convolution layer, a channel rearrangement unit, a second 3x3 convolution layer, a 1x1 convolution layer, a distribution focal loss module and a bounding box decoding function; wherein the output of the second reparameterization transpose fusion module of the neck network and the output of the two reparameterization splicing modules are respectively used as the input of the three AsDDet detection heads in the detection head network.

[0075] In specific implementation, due to the interference of non-parallel light source and weak light environment, the design requirement of the detection head for road surface weather monitoring is very high. The traditional YOLOv8 detection head shares the feature extraction layer of classification and regression, which causes the mutual interference of weather state classification tasks, especially the wet area caused by the spread of water. The AsDDet detection head is proposed, which realizes high-precision and high-efficiency road surface weather state segmentation through task decoupling and DFL design. Its structure is as shown in Figure 6 .

[0076] In the AsDDet detection head, the input feature map is processed in parallel through decoupled classification and regression branches. In the classification branch, the input feature map is processed through a 3×3 depthwise separable convolutional layer to extract local detailed features, and then further fused with semantic information through another 3×3 convolutional layer. Subsequently, a 1×1 convolutional layer maps the number of channels to the category dimension, and finally, the classification probability corresponding to each position is output through the Sigmoid activation function. In the regression branch, the input feature map is first processed through a first 3×3 convolutional layer to extract localization features, and then the feature interaction capability is enhanced through channel shuffling. Then, the features are refined and the channels are adjusted through a second 3×3 convolution and a 1×1 convolution. Next, the probability distribution is modeled using a distributed focus loss module, and finally, the probability distribution is transformed and output to obtain the bounding box coordinates through a bounding box decoding function. The classification probability and the bounding box coordinates are merged to finally output the localization and segmentation results of the road meteorological image as the output of the road meteorological state segmentation model.

[0077] The most important components of the AsDDet detection head in this embodiment are Depthwise Separable Convolution (DWConv) and Distributed Focal Loss (DFL). The classification branch uses DWConv to extract local semantic features, including depthwise convolution and pointwise convolution. Traditional linear edge regression methods typically rely on direct coordinate prediction, which is prone to quantization errors due to feature map downsampling. In this process, continuous coordinates are mapped to discrete grid locations, leading to inaccuracies. The DFL method mitigates these quantization errors by shifting from traditional coordinate regression to probability distribution regression. Instead of directly predicting bounding box coordinates, DFL models each bounding box as a discrete probability distribution across multiple possible locations. This allows the model to predict the extent of potential bounding box locations, which can then be decoded into more accurate coordinates by calculating the expectation of the predicted probability distribution.

[0078] 3. Examples

[0079] The experimental system used in this embodiment is a near-infrared camera road weather segmentation system, such as... Figure 7 As shown, with With a road surface cooler as its core, it can simulate typical road surface weather conditions such as icing, dryness, water accumulation, and dampness. It is compatible with both cement and asphalt roadbeds, providing real-world support for image acquisition. The acquisition system uses a near-infrared industrial camera equipped with a filter wheel containing filters in five specific wavelength bands: 960nm, 980nm, 1064nm, 1250nm, and 1550nm, to provide multispectral data on road surface weather conditions.

[0080] 3.1 Experimental Environment

[0081] The detailed experimental parameter settings for this experiment are shown in Table 1.

[0082] Table 1 Experimental Parameter Settings

[0083]

[0084] 3.2, Dataset Collection

[0085] The dataset used in the experiment is the reflectance spectrum of the ice-water medium, as shown in Figure 8 .

[0086] To improve the robustness of the model, the image dataset at 960 nm, 1250 nm, and 1550 nm is selected according to the reflectance spectrum of the ice-water medium. The images at the three wavelengths have the following imaging characteristics:

[0087] (1) When the wavelength is at 980 nm, water and ice are highly reflective to light, which can be used as basic information of the road surface.

[0088] (2) When the wavelength is at 1250 nm, the reflectivity of both begins to decrease, and this wavelength can also filter out excess light.

[0089] (3) At 1550 nm, the reflectivity of both is at a trough, and the difference in reflectivity can be used to distinguish the road surface state information.

[0090] The original data includes 615 near-infrared road weather images, of which 27 are dry road background, 261 are water-covered road, and 66 are wet road. After processing, the image pixels are unified to 640*640. Some of the datasets are shown in Figure 9 , where Figure 9 (a) is the original image of the road weather state, Figure 9 (b) is the preprocessed image of the road weather state. To ensure the diversity and balance of the data, the dataset is randomly divided according to the ratio of training set: validation set: test set = 7:2:1. Then, the images that have been specularly reflected and light equalized are labeled using the labelme software.

[0091] 3.3, Comparison and Analysis of Improved Models

[0092] The evaluation indicators for measuring the segmentation performance of the model mainly include parameter quantity, precision P, recall R, and average precision mAP.

[0093] C2f, C2f_FocalNeXt, C2f_RMB, C2f_FasterNeXt, and C2f_Biformer were analyzed. Under the same input conditions, the above attention mechanisms were replaced at the same position of the original model structure, and the standard feature extraction module C2f was used as the benchmark for comparison. The GFLOPs is the computational complexity, the Parameters is the model parameter quantity, and the mAP is the average precision. The comparison results are shown in Table 2.

[0094] Table 2 Attention mechanism experimental results

[0095]

[0096] The experimental results show that the parameter quantity is the least (2.5 GFLOPs) and the computational complexity is the smallest (10.1M) after introducing the C2f_FasterNeXt module in the model. Compared with C2f_FasterNeXt, after introducing the C2f_Biformer module, the parameter quantity and the computational complexity are reduced while the average accuracy of the model is improved by 5.8%. Through the comparison of experimental results, the dynamic sparse attention mechanism (C2f_Biformer) is selected to be introduced into the benchmark model.

[0097] The neck structure of the YOLOv8 model is replaced with RepFPN, effQAFPN and SSFF respectively for comparative experiments, and the comparison results are shown in Table 3.

[0098] Table 3 Experimental results of neck neural network

[0099]

[0100] The experimental results show that after changing the neck network of the model, the network layers are greatly increased, but after introducing the hardware-aware neural network (RepFPN), the parameter quantity and computational complexity are reduced by 3.3% and 3.5% respectively while increasing the network depth, and the average accuracy and precision are improved by 8.5% and 2.4% respectively. Through performance analysis experiments, the hardware-aware neural network (RepFPN) is introduced into the neck network of the benchmark model.

[0101] 3.4, Ablation experiment

[0102] To verify the effectiveness of each module of the YOLOv8-BRFA model, three groups of ablation experiments are designed using the near-infrared road weather data set after image preprocessing as the experimental data set. The dynamic sparse attention mechanism (C2f_Biformer), the hardware-aware neural network (RepFPN) and the AsDDet detection head are added to the YOLOv8 model respectively, and the experimental results are shown in the table, where the bold indicates the best indicator.

[0103] Table 4 Ablation experiment results of YOLOv8-BRFA model

[0104]

[0105] After introducing the dynamic sparse attention mechanism (C2f_Biformer) and the hardware-aware neural network (RepFPN), the network structure layer and parameter quantity are increased by 71.3% and 40%, respectively, increasing the computational complexity, but the average precision of the two is increased by 9.5% and 0.8%, respectively. The reason for the increase in network structure layer and parameter quantity after introducing the RepFPN network structure is that the Repblock needs to cascade multiple RepConv to increase the depth of the model, enhance the generalization ability and flexibility of the model, and enable the model to extract deeper semantic information from the feature map, thereby improving the detection accuracy. At this time, the adaptive decoupled detection head (AsDDet) is introduced, which reduces the interference of non-parallel light sources and weak light environments through task decoupling and DFL design, and at the same time reduces the computational complexity and parameter quantity through the depth separable convolution DWConv unique to the AsDDet detection head. Compared with the original YOLOv8 model, although the network layer of YOLOv8-BRFA increases, the parameter quantity and computational complexity are reduced by 6.7% and 5.3%, respectively, while the average precision of the two is increased by 12.7% and 1.3% compared with the baseline model, respectively.

[0106] 3.5、Experimental results analysis

[0107] The segmentation accuracy of the two models YOLOv8 and YOLOv8-BRFA is evaluated on the test set of the dataset. Table 5 is the comparison chart of the average accuracy of each category segmentation. The average segmentation accuracy of the YOLOv8-BRFA model under the three weather conditions is improved, which proves the effectiveness of the model improvement.

[0108] Table 5 Average accuracy of each category segmentation

[0109]

[0110] In order to intuitively show the effect of the model in this paper in near-infrared road weather segmentation, the water accumulation, icing, and wet road surfaces on the test set are selected for visualization from the weather segmentation effect of each improved model and each roadbed. The visualization results are as shown in Figure 10 .

[0111] From the visualization results, it can be seen that the YOLO model of the embodiment can maintain a higher segmentation confidence and the optimal boundary box fitting degree compared with other models. In all test samples, the segmentation result of the model of the embodiment is most consistent with the true segmentation area. The visualization results further verify that the model of the embodiment has good segmentation effect and positioning ability in the application of near-infrared road weather segmentation. The model of the embodiment also achieves good classification and segmentation effect of road weather on simulation, cement, and asphalt road surfaces. In summary, the improved algorithm YOLOv8-BRFA of the embodiment has better segmentation performance.

[0112] 4. SUMMARY

[0113] To solve the classification and segmentation problem of near-infrared pavement weather state under non-parallel light source, under the premise of solving the problem of specular reflection and uneven illumination, aiming at the surface characteristics of pavement weather (including water accumulation, icing, wetness, dryness) and the problems of overexposure and underexposure of light intensity, an improved YOLOv8-BRFA segmentation model is proposed. The specific improvements include:

[0114] The dynamic sparse attention mechanism (C2f-biformer) is used to replace the traditional feature fusion module in the YOLOv8 backbone to reduce the parameter quantity and complexity and improve the average precision. The hardware perception neural network (RepFPN) is introduced in the neck to enhance the ability to capture the surface characteristics of weather states such as water accumulation and icing under weak light environment and improve the average precision of model segmentation. At the same time, the adaptive decoupling detection head (AsDDet) is introduced. Through task decoupling and DFL design, the interference of non-parallel light source and weak light environment is reduced, and the calculation complexity and parameter quantity are reduced through the depth separable convolution DWConv unique to the AsDDet detection head.

[0115] The experimental results show that compared with the benchmark model YOLov8, the two average precisions of the YOLOv8-BRFA model are improved by 12.79% and 1.3% respectively. The parameter quantity and calculation quantity are reduced by 5.3% and 6.7% respectively, fully proving the effectiveness of the improved algorithm. At the same time, the precision P and recall R of the YOLOv8-BRFA model are improved by 1.2% and 6.4% respectively. The YOLOv8-BRFA model reaches the optimal value in the algorithm model through the comparison of various mainstream algorithms.

[0116] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described by referring to the preferred embodiments of the present application, those skilled in the art should understand that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined in the appended claims.

Claims

1. A method for segmenting road surface meteorological state images based on the YOLOv8-BRFA model, characterized in that, Includes the following steps: Collect near-infrared road meteorological image dataset; Based on the YOLOv8 network architecture, a dynamic sparse attention mechanism and a hardware-aware neural network were designed, and combined with an adaptive decoupled detection head, to construct a road surface meteorological state segmentation model. The road meteorological state segmentation model was trained using a near-infrared road meteorological image dataset to obtain the trained road meteorological state segmentation model. The road surface meteorological image to be detected is acquired, and then input into the trained road surface meteorological state segmentation model to perform road surface meteorological state recognition and segmentation, thereby obtaining the localization and segmentation results of the road surface meteorological image to be detected.

2. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 1, characterized in that, Before training the road surface meteorological state segmentation model, the process also includes: removing specular reflections from the near-infrared road surface meteorological image dataset using mathematical morphology algorithms, and then performing illumination homogenization based on Retinex theory, combined with gamma transform and image difference.

3. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 2, characterized in that, The road surface meteorological state segmentation model includes a backbone network that integrates a dynamic sparse attention mechanism, a neck network that integrates a hardware-sensing neural network, and an adaptively decoupled detection head network. The trained road weather condition segmentation model takes the preprocessed near-infrared road weather image dataset as input to the backbone network, extracts deep feature maps through a dynamic sparse attention mechanism, and then uses the hardware-perceptual neural network of the neck network to perform multi-scale feature enhancement and fusion on the deep feature maps to generate multi-scale feature maps, which are then input to the adaptively decoupled detection head network. Classification and localization are performed through task decoupling and distributed focus loss, and finally the localization and segmentation results of the road weather condition are obtained, which are used as the output of the road weather condition segmentation model.

4. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 3, characterized in that, The backbone network includes sequentially connected convolutional layers, four dynamic attention feature extraction modules, and an SPPF module; the dynamic attention feature extraction module includes cascaded convolutional layers and a C2f_biformer module, wherein the C2f_biformer module includes cascaded depthwise separable convolutional layers, a first normalization layer, a two-layer routing attention module, a second normalization layer, and an MLP layer; In the C2f_biformer module, after feature extraction from the input features through a depthwise separable convolutional layer, the feature is concatenated with the input of the depthwise separable convolutional layer and used as the input of the first normalization layer. The output of the first normalization layer is then used as the input of the dual-layer routing attention module for multi-level feature extraction, and concatenated with the input of the first normalization layer to serve as the input of the second normalization layer. The output of the second normalization layer is then used as the input of the MLP module. Finally, the output of the MLP module is concatenated with the input of the second normalization layer to serve as the final output of the C2f_biformer module.

5. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 4, characterized in that, The processing procedure of the dual-layer routing attention module specifically includes: First, the feature map is divided into S×S attention regions, and a query matrix, key matrix, and value matrix are generated through linear transformation. Next, a two-layer routing mechanism is used to dynamically sparsify and filter the attention regions. Average pooling is performed on the query and key matrices of each region to obtain region-level query and key representations, and an affinity adjacency matrix between regions is constructed. Top-k filtering is used to retain the k most relevant target regions for each region, forming a routing index matrix. Subsequently, the corresponding key-value pairs of the target regions are aggregated based on this routing index matrix, transforming it into a dense matrix form. Fine-grained token-level attention calculation is then performed between the aggregated key-value pairs and the original query to generate a sparse attention weight matrix. Depthwise separable convolution is introduced to further enhance local contextual information. Finally, the sparse attention weight matrix and the aggregated values ​​are weighted and fused, and the output features are restored to their original spatial dimensions, resulting in output features with structural sparsity.

6. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 5, characterized in that, The neck network includes two reparameterized transpose fusion modules and two reparameterized stitching modules connected in sequence; the reparameterized transpose fusion module includes a cascaded SimConv layer, a Transpose layer, a connection layer, and a reparameterization module; the reparameterized stitching module includes a cascaded SimConv layer, a connection layer, and a reparameterization module; the reparameterization module is composed of multiple cascaded RepConv modules. The output of the backbone network serves as the input to the first reparameterized transpose fusion module. The outputs of the second and third dynamic attention feature extraction modules in the backbone network are transmitted to the connection layers in the first and second reparameterized transpose fusion modules in the neck network for splicing and fusion operations, respectively. The output of the SimConv layer in the first reparameterized transpose fusion module in the neck network is also transmitted to the connection layer in the second reparameterized splicing module for splicing and fusion transport. The output of the SimConv layer in the second reparameterized transpose fusion module is transmitted to the connection layer in the first reparameterized splicing module for splicing and fusion transport.

7. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 6, characterized in that, The detection head network includes three parallel AsDDet detection heads; each AsDDet detection head includes a parallel classification branch and a regression branch; the classification branch includes a cascaded 3×3 depthwise separable convolutional layer, a 3×3 convolutional layer, a 1×1 convolutional layer, and a sigmoid activation function; the regression branch includes a first 3×3 convolutional layer, a channel rearrangement unit, a second 3×3 convolutional layer, a 1×1 convolutional layer, a distributed focus loss module, and a bounding box decoding function. The output of the second reparameterized transpose fusion module of the neck network and the outputs of the two reparameterized splicing modules are used as inputs to the three AsDDet detectors in the detector head network.

8. The method for road surface meteorological state image segmentation based on the YOLOv8-BRFA model according to claim 7, characterized in that, The processing procedure of the AsDDet detection head specifically includes: In the AsDDet detection head, the input feature map is processed in parallel through decoupled classification and regression branches. In the classification branch, the input feature map is processed through a 3×3 depthwise separable convolutional layer to extract local detailed features, and then further fused with semantic information through another 3×3 convolutional layer. Subsequently, a 1×1 convolutional layer maps the number of channels to the category dimension, and finally, the classification probability corresponding to each position is output through the Sigmoid activation function. In the regression branch, the input feature map is first processed through a first 3×3 convolutional layer to extract localization features, and then the feature interaction capability is enhanced through channel shuffling. Then, the features are refined and the channels are adjusted through a second 3×3 convolution and a 1×1 convolution. Next, the probability distribution is modeled using a distributed focus loss module, and finally, the probability distribution is transformed and output to obtain the bounding box coordinates through a bounding box decoding function. The classification probability and the bounding box coordinates are merged to finally output the localization and segmentation results of the road meteorological image as the output of the road meteorological state segmentation model.

Images