A rain and snow weather road surface real-time monitoring and early warning method based on deep learning

Abstract

This invention discloses a deep learning-based method for real-time monitoring and early warning of road surfaces in rain and snow weather, comprising the following steps: acquiring road images to construct labeled and unlabeled datasets; training a teacher model to form a model set; generating pseudo-labels for unlabeled images and selecting stable images and pixels; training a student model to generate preliminary segmentation; extracting scene features to generate real observations and intrinsic road semantics; structural analysis to identify rain and snow disturbance areas and generate improved pseudo-labels; jointly training and updating the student model based on stable pixels and improved pseudo-labels; iteratively incorporating unselected images to optimize the model, and outputting rain and snow distribution and early warning results. This invention achieves high accuracy, high reliability, and real-time monitoring performance for road rain and snow distribution identification and early warning models.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and road monitoring technology, and in particular to a method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning. Background Technology

[0002] In the field of road safety monitoring, water and snow accumulation on roads caused by rain and snow significantly reduce the road surface friction coefficient, which is one of the main causes of traffic accidents. Semantic segmentation technology is needed to achieve pixel-level accurate detection and early warning. However, existing semantic segmentation technologies face two major bottlenecks in application to road rain and snow scenarios: The cost of annotation is increasing exponentially: Rain and snow areas have irregular shapes and blurred edges, and are easily superimposed with road stains and shadows. Annotators need to distinguish the boundary and background pixel by pixel, which is not only inefficient, but also makes it difficult to guarantee the accuracy of manual annotation. In semi-supervised learning, pseudo-label quality control is difficult: To reduce labeling costs, existing solutions often use pseudo-label strategies to utilize unlabeled data. However, the visual complexity of road rain and snow scenes makes pseudo-labels generated by a single model prone to noise, resulting in problems such as missing small areas of rain and snow or mislabeling non-rain and snow areas, which in turn exacerbates model training bias.

[0003] Therefore, how to provide a real-time monitoring and early warning method for road surfaces in rain and snow based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a real-time monitoring and early warning method for road surfaces in rain and snow based on deep learning. This invention fully utilizes the SegFormer semantic segmentation model, multi-level pseudo-label filtering, and pixel confidence weighted training algorithm. It describes in detail the implementation process of extracting pixel-level rain and snow information from road images, generating rain and snow distribution maps, and inferring early warning areas. It has the advantages of high segmentation accuracy, strong pseudo-label reliability, and fast road early warning response.

[0005] A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to an embodiment of the present invention includes the following steps: Acquire road image data and construct labeled and unlabeled datasets; The SegFormer semantic segmentation network is trained based on a labeled dataset, and models from multiple training stages are saved to form a teacher model set. Unlabeled data is input into the teacher model set to generate pseudo-labels. The overall semantic consistency of the predictions of the same image by multiple teacher models is compared to select stable images. The consistency of each pixel in the multi-model prediction category is statistically analyzed to select pixel-level stable pixels to form pseudo-labels. The student model is initialized and trained using pseudo-labels to generate preliminary segmentation results. The road image is input into the SegFormer student model to extract scene features, and the real observation semantic representation and the road intrinsic semantic representation are generated based on the scene features respectively. Structural parsing is performed on the semantic representation, including analysis of regional distribution, boundary continuity, and local feature stability. By comparing the structural differences between the real observation semantic representation and the road intrinsic semantic representation, rain and snow disturbance areas are identified, and improved pseudo-labels and segmentation results are generated. Based on stable images, stable pixels, and improved segmentation results, the SegFormer student model parameters are updated by assigning corresponding confidence weights to stable and unstable pixels using pixel prediction confidence. During training, previously unfiltered images are included, pseudo-labels and pixel weights are updated periodically, and a pavement monitoring and early warning model for rain and snow weather is obtained through iterative optimization. The model outputs pavement rain and snow distribution results and early warning results.

[0006] Optionally, acquiring road image data specifically includes: Set up a road image acquisition device to capture road scene images at a vehicle-mounted or roadside location; The acquired images are filtered, and samples with image blur exceeding a threshold are removed to form a preliminary image set; In the initial image set, rain and snow areas of some images are manually labeled, and rain, snow and background are distinguished pixel by pixel to generate a labeled dataset. Unlabeled images are saved as an unlabeled dataset. Perform basic normalization on unlabeled images to unify image resolution, color channels, and data format; The images were classified and labeled according to the time of acquisition, road type, and weather conditions. Indexes and storage structures are created for labeled and unlabeled datasets respectively, forming labeled and unlabeled datasets.

[0007] Optionally, training the SegFormer semantic segmentation network based on the labeled dataset specifically includes: The labeled dataset is input into the encoder of the SegFormer semantic segmentation network to extract multi-scale scene features, and the decoder generates initial pixel-level semantic predictions. In each training iteration, the network output and the corresponding labeled data are cross-entropy loss is calculated, and backpropagation is performed to update the SegFormer network parameters based on the loss value. During training, each time a preset number of rounds is completed or a training phase node is reached, the current network parameters are saved as a model checkpoint, and the network state, training rounds, and corresponding loss information are recorded. Repeat the training until the preset total number of rounds is completed, and save all training stage checkpoints to form a set of teacher models.

[0008] Optionally, generating the preliminary segmentation result specifically includes: Unlabeled images are input batch by batch into each checkpoint model of the teacher model set, and a corresponding pixel-level predicted probability distribution is generated for each image. The consistency of the overall semantics of the same image output by different teacher models is compared, the average intersection-union ratio is calculated, and images with consistency higher than a preset threshold are classified as stable images. For each pixel in a stable image, the consistency of the predicted categories of multiple teacher models is statistically analyzed to calculate pixel-level stability. Pixels whose consistency meets the threshold are classified as stable pixels and pseudo-labels are generated. The stable image and the pseudo-labels corresponding to the pixels are input into the initialized SegFormer student model, and cross-entropy loss is used for training. Backpropagation is performed to update the network parameters based on the error between pixel prediction and pseudo-label. During the training process, after each preset number of rounds, the current student model parameters are saved as student model checkpoints; Repeat the training until the preset number of rounds is completed to generate preliminary segmentation results; During training, stable images and pixel-level stable pixel information are preserved.

[0009] Optionally, the SegFormer student model specifically includes: Based on the weather labels in the labeled dataset, the road images are divided into a rain and snow image set and a sunny or cloudy image set; Rain and snow images are input into the SegFormer student model encoder in batches to extract multi-scale features, and then the decoder generates a real observation semantic representation. The corresponding sunny or cloudy day images are input into the same SegFormer student model encoder in batches to extract multi-scale features, and the road intrinsic semantic representation is generated by the decoder. During the generation process, the multi-scale feature maps of each image are channel normalized and upsampled to the original image resolution, while retaining the class probability and spatial location of each pixel; Output the true observation semantic representation and the road intrinsic semantic representation, including the predicted probability of each pixel in all categories.

[0010] Optionally, the execution structure parsing specifically includes: The student model outputs a pixel-by-pixel aligned and compared the true observation semantic representation with the road intrinsic semantic representation of the corresponding scene, and the difference in the predicted probability of each pixel across all categories is calculated. Based on the difference value, pixels that are continuous and have a probability difference exceeding a preset threshold are aggregated to form candidate rain and snow disturbance regions; For each candidate region, calculate boundary continuity metrics, including boundary length, curvature, and neighborhood pixel class consistency, to identify isolated pixels and broken boundaries; Local stability analysis is performed on the class probabilities of pixels within the candidate region, and isolated pixels are corrected by weighted averaging of the probabilities of neighboring pixels. Improved pseudo-labels are generated for the corrected candidate regions, and the improved pseudo-labels are input into the student model to form the final segmentation result matrix; Output the improved pseudo-label and segmentation result matrix, including the predicted probability of each pixel in all categories and its corresponding spatial coordinates.

[0011] Optionally, the joint training to update the SegFormer student model parameters specifically includes: Input the stable image and its corresponding stable pixels into the SegFormer student model, extract multi-scale features and generate pixel-level predictions through the decoder; For each pixel, a weight coefficient is calculated based on the stable pixel identifier and the predicted probability. Stable pixels are assigned a fixed weight, while unstable pixels are assigned a weight related to the predicted probability. The pixel weights are combined with the cross-entropy loss function to calculate the weighted loss for each batch of images; The weighted loss is backpropagated to update the parameters of the SegFormer student model. During the training process, after each preset round, the current model parameters are saved as a training checkpoint, and the batch index, loss value and weight distribution information are recorded. Repeat the training until the total number of rounds is completed, maintaining stable images and indexes of stable pixel information; In each training round, the pixel prediction probabilities after updating the student model parameters are used for the next round of weighted loss calculation, forming a joint training loop; Output the trained SegFormer student model parameters and pixel-level prediction matrix, and generate the final improved pseudo-labels and segmentation results.

[0012] Optionally, the iterative optimization specifically includes: Iterative training is performed on the stable images and stable pixels obtained during the training process of the student model; During the training iterations, previously unfiltered unstable images are included in the training dataset; In each training iteration, the stability and prediction confidence of the pixels are recalculated based on the pixel prediction results of the current student model. Generating or updating pseudo-labels and weight matrices based on updated stability and confidence; The student model is jointly trained using the updated pseudo-labels and pixel weights; Repeat the training steps iteratively until the predetermined number of training rounds is completed.

[0013] Optionally, the output pavement rain and snow distribution results and early warning results specifically include: The segmentation result matrix generated by the trained SegFormer student model is converted into road rain and snow distribution results, where the category label of each pixel represents the corresponding rain and snow state. Regional clustering is performed on road rain and snow distribution results to identify continuous rain and snow disturbance areas; Based on the area of ​​rain and snow, potential risk areas are identified, and road rain and snow warning results are generated. The output includes a monitoring and early warning model that includes road rain and snow distribution results and early warning results.

[0014] The beneficial effects of this invention are: (1) Reduced labeling cost: By making full use of unlabeled road images through semi-supervised learning, only a small number of manually labeled samples are needed to train the high-precision SegFormer student model, which effectively reduces the workload of pixel-level labeling and improves data utilization efficiency.

[0015] (2) Improved pseudo-label quality and segmentation accuracy: The multi-level view filtering strategy combined with pixel confidence weighted training significantly improved the recognition reliability of stable images and pixels, enhanced the segmentation accuracy of rain and snow boundaries and small-area rain and snow targets, and improved the average intersection-union ratio (MIoU) and small target detection rate.

[0016] (3) Enhanced real-time road rain and snow monitoring and early warning capabilities: Based on the segmentation results, a road rain and snow distribution map is generated, and risk areas are identified through regional analysis, boundary continuity and local feature stability assessment, so as to achieve rapid and accurate road rain and snow early warning and provide reliable decision support for traffic safety management. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a real-time monitoring and early warning method for road surfaces in rain and snow weather based on deep learning proposed in this invention; Figure 2 This is an architecture diagram of a pseudo-label generation and filtering method for a real-time monitoring and early warning method for road surfaces in rain and snow weather based on deep learning proposed in this invention. Figure 3 This is a flowchart illustrating the iterative optimization and rain / snow warning output process of a deep learning-based real-time pavement monitoring and early warning method for rain and snow weather proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figures 1-3 A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning includes the following steps: Acquire road image data and construct labeled and unlabeled datasets; The SegFormer semantic segmentation network is trained based on a labeled dataset, and models from multiple training stages are saved to form a teacher model set. Unlabeled data is input into the teacher model set to generate pseudo-labels. The overall semantic consistency of the predictions of the same image by multiple teacher models is compared to select stable images. The consistency of each pixel in the multi-model prediction category is statistically analyzed to select pixel-level stable pixels to form pseudo-labels. The student model is initialized and trained using pseudo-labels to generate preliminary segmentation results. The road image is input into the SegFormer student model to extract scene features, and the real observation semantic representation and the road intrinsic semantic representation are generated based on the scene features respectively. Structural parsing is performed on the semantic representation, including analysis of regional distribution, boundary continuity, and local feature stability. By comparing the structural differences between the real observation semantic representation and the road intrinsic semantic representation, rain and snow disturbance areas are identified, and improved pseudo-labels and segmentation results are generated. Based on stable images, stable pixels, and improved segmentation results, the SegFormer student model parameters are updated by assigning corresponding confidence weights to stable and unstable pixels using pixel prediction confidence. During training, previously unfiltered images are included, pseudo-labels and pixel weights are updated periodically, and a pavement monitoring and early warning model for rain and snow weather is obtained through iterative optimization. The model outputs pavement rain and snow distribution results and early warning results.

[0020] In this embodiment, acquiring road image data specifically includes: Set up a road image acquisition device to capture road scene images at a vehicle-mounted or roadside location; The acquired images are filtered, and samples with image blur exceeding a threshold are removed to form a preliminary image set; the Tenengrad function is used to calculate the image blur. In the initial image set, rain and snow areas of some images were manually labeled, and rain, snow and background were distinguished pixel by pixel to generate a labeled dataset. Unlabeled images were saved as an unlabeled dataset. The number of samples in the unlabeled dataset was much larger than the number of samples in the labeled dataset. Perform basic normalization on unlabeled images to unify image resolution, color channels, and data format; The images were classified and labeled according to the time of acquisition, road type, and weather conditions. Indexes and storage structures are created for labeled and unlabeled datasets respectively, forming labeled and unlabeled datasets.

[0021] Define the dataset, assuming there is a labeled dataset for the road rain and snow scene. ,in Input images of roads in rain or snow. Pixel-level annotations; unlabeled datasets are denoted as... , In addition, the number of categories is denoted as Semantic segmentation model is denoted as The parameters are .but Represented as pixels Predicted as the first The probability of a class.

[0022] In this embodiment, training the SegFormer semantic segmentation network based on a labeled dataset specifically includes: The labeled dataset is input into the encoder of the SegFormer semantic segmentation network to extract multi-scale scene features, and the decoder generates initial pixel-level semantic predictions. The encoder adopts the MiT-B5 structure; the decoder adopts a lightweight MLP head to fuse multi-scale features and upsample them to the original image resolution. In each training iteration, the network output and the corresponding labeled data are cross-entropy loss is calculated, and backpropagation is performed to update the SegFormer network parameters based on the loss value. During training, at each preset number of rounds or when a training phase node is reached, the current network parameters are saved as model checkpoints, recording the network state, training rounds, and corresponding loss information; each checkpoint saves the encoder weights, decoder weights, optimizer state, and cumulative training loss. Repeat the training until the preset total number of rounds is completed, and save all training stage checkpoints to form a set of teacher models.

[0023] In this embodiment, generating the preliminary segmentation result specifically includes: Unlabeled images are input batch by batch into each checkpoint model of the teacher model set, and a corresponding pixel-level predicted probability distribution is generated for each image. The consistency of the overall semantics of the same image output by different teacher models is compared, the average intersection-union ratio is calculated, and images with consistency higher than a preset threshold are classified as stable images. For the global hierarchical view, during the fully supervised training phase, the teacher model is trained based on labeled data and saved. Teacher model checkpoints For the same unlabeled image generate A fake mask ; with the final model Generated pseudo mask As a reference, the average intersection-over-union ratio (AUC) measures the consistency of predictions at each stage, thus defining the image-level global stability score: ; according to Sort all unlabeled images from highest to lowest resolution, and set a ratio threshold (such as selecting the top in this invention). (images) as a globally stable image set The rest are considered as a globally unstable image set. In model training, during the initial training phase, only... On; For each pixel in a stable image, the consistency of the predicted categories of multiple teacher models is statistically analyzed to calculate pixel-level stability. Pixels whose consistency meets the threshold are classified as stable pixels and pseudo-labels are generated. In a local hierarchy view, considering the same image The pixel difficulty difference between the foreground target and the blurred boundary and background noise, for each pixel collect The predicted probability distribution of each model: ; in, Indicates the first Each teacher model predicts pixels Let be the probability of class c. The hard-predicted class under different views can be represented as... .like If all views predict the same category, then the pixel can be considered semantically stable across multiple view levels. The pixel-level stability score is defined as follows: ; Where sp∈{0,1} is the pixel-level stability score function, where sp=1 indicates that pixel p is a stable pixel, and 0 indicates that it is an unstable pixel. For stable pixels, the final pseudo-label is further generated using the average probability of multi-level views. ; The set of stable pixels is denoted as In the first stage of unsupervised training, only stable pixels are used in the loss calculation to filter out noise interference from unstable pixels: ; in, For indicator functions, when hour By filtering multi-level views with image-level and pixel-level stability, pseudo-label noise caused by light spots, water vapor, and suspended particles in rain and snow scenes is reduced, improving the training signal-to-noise ratio of the model in early iterations and effectively preserving visual information of blurred boundary regions and small targets.

[0024] The stable image and the pseudo-labels corresponding to the pixels are input into the initialized SegFormer student model, and cross-entropy loss is used for training. Backpropagation is performed to update the network parameters based on the error between pixel prediction and pseudo-label. During the training process, after each preset number of rounds, the current student model parameters are saved as student model checkpoints; Repeat the training until the preset number of rounds is completed to generate preliminary segmentation results; During training, stable images and pixel-level stable pixel information are preserved.

[0025] In this embodiment, the SegFormer student model specifically includes: Based on the weather labels in the labeled dataset, the road images are divided into a rain and snow image set and a sunny or cloudy image set; Rain and snow images are input into the SegFormer student model encoder in batches to extract multi-scale features, and then the decoder generates a real observation semantic representation. The corresponding sunny or cloudy day images are input into the same SegFormer student model encoder in batches to extract multi-scale features, and the decoder generates the intrinsic semantic representation of the road; the input images are matched with rain and snow images according to the scene to ensure that the same road location corresponds to a sunny or cloudy day image; the encoder and decoder structures are consistent with the real observed semantic path. During the generation process, the multi-scale feature maps of each image are channel normalized and upsampled to the original image resolution, while retaining the class probability and spatial location of each pixel; Output the true observation semantic representation and the road intrinsic semantic representation, including the predicted probability of each pixel in all categories.

[0026] In this embodiment, the specific steps of performing structure parsing include: The student model outputs a pixel-by-pixel aligned and compared the true observation semantic representation with the road intrinsic semantic representation of the corresponding scene, and the difference in the predicted probability of each pixel across all categories is calculated. Based on the difference value, consecutive pixels with probability differences exceeding a preset threshold are aggregated to form candidate rain and snow disturbance regions; a threshold of 0.4 is set, indicating that if the probability difference of pixel categories exceeds this range, it is judged as abnormal; 8-connected clustering is performed on pixels that meet the threshold, and consecutive pixels are aggregated to form candidate rain and snow disturbance regions; the regions are filtered by minimum area, and regions with an area ≥ 16 pixels are retained as valid candidates. For each candidate region, a boundary continuity index is calculated, including boundary length, curvature, and neighborhood pixel class consistency, to identify isolated pixels and broken boundaries. Canny edge detection is used to obtain the boundaries of candidate regions, and the boundary length and local curvature are calculated. The curvature is calculated using the three-point method to estimate the angle change of continuous pixels. The class consistency of boundary pixels in a 3×3 neighborhood is statistically analyzed. If the consistency is less than 0.6, it is determined to be an isolated pixel or a broken boundary. Isolated pixels are marked as temporary exclusion points. Local stability analysis is performed on the class probabilities of pixels within the candidate region, and isolated pixels are corrected by weighted averaging of neighboring pixel probabilities. The class probabilities of each pixel and its 3×3 and 5×5 neighborhoods are weighted and averaged, with the weights distributed according to the reciprocal of the Euclidean distance between the neighboring pixels and the center pixel. The corrected class probabilities are used to replace the original isolated pixel predictions. The correction process retains more than 50% of the original probability contribution of the center pixel to prevent over-smoothing. Improved pseudo-labels are generated for the corrected candidate regions. The improved pseudo-labels are input into the student model to form the final segmentation result matrix. For each corrected pixel, the category corresponding to the maximum category probability is selected as the improved pseudo-label. The improved pseudo-labels are used as supervised input to the student model to perform one iteration of training and update the pixel-level prediction probabilities. The entire image is batch processed to generate the final segmentation result matrix with the same size as the input image. Output the improved pseudo-label and segmentation result matrix, including the predicted probability of each pixel in all categories and its corresponding spatial coordinates.

[0027] In this embodiment, the joint training and updating of the SegFormer student model parameters specifically includes: Input the stable image and its corresponding stable pixels into the SegFormer student model, extract multi-scale features and generate pixel-level predictions through the decoder; Student model obtained by retraining on stable pixels For all unlabeled images Perform prediction to obtain each pixel Predicted probability distribution Define pixel-level prediction confidence as the maximum class probability: ; The value ranges from [0,1]. The larger the value, the more reliable the student model's prediction of the pixel category. For each pixel, a weight coefficient is calculated based on the stable pixel identifier and the predicted probability. Stable pixels are assigned a fixed weight, while unstable pixels are assigned a weight related to the predicted probability. Combined with stability indicator function Construct a weight matrix Differentiated training weights are assigned to pixels with different levels of reliability: ; For multi-level vision Figure 1 For stable pixels with high-confidence pseudo-labels, the training weight is set to 1. For unstable pixels, the training weight is determined by the prediction confidence of the student model. The higher the confidence, the greater the weight, so as to strengthen the supervision of high-confidence unstable pixels and weaken the interference of low-confidence pixels. The pixel weights are combined with the cross-entropy loss function to calculate the weighted loss for each batch of images; To alleviate the pixel imbalance between rain / snow and background categories in road rain / snow scenes, a category reweighting term is introduced. Let... The first pseudo tag The number of pixels in the class, defining the class weight as ,in To balance the performance (default value 0.99), higher weights are assigned to rain and snow categories with fewer pixels, thus enhancing feature learning for long-tail categories. A weight matrix is ​​incorporated into the unsupervised loss for unlabeled data. With category weight Component confidence-weighted cross-entropy loss: ; In addition to unsupervised loss, this invention introduces standard pixel-level cross-entropy as a supervised loss: ; The total loss of the model is the weighted sum of the supervised loss and the unsupervised loss: ; Wherein, λ is the unsupervised loss weight (which increases linearly from 0.1 to 1.0 with each training epoch), balancing the supervisory effects of labeled and unlabeled data; The weighted loss is backpropagated to update the parameters of the SegFormer student model. During the training process, after each preset round, the current model parameters are saved as a training checkpoint, and the batch index, loss value and weight distribution information are recorded. Repeat the training until the total number of rounds is completed, maintaining stable images and indexes of stable pixel information; In each training round, the pixel prediction probabilities after updating the student model parameters are used for the next round of weighted loss calculation, forming a joint training loop; Output the trained SegFormer student model parameters and pixel-level prediction matrix, and generate the final improved pseudo-labels and segmentation results.

[0028] In this embodiment, iterative optimization specifically includes: Iterative training is performed on the stable images and stable pixels obtained during the training process of the student model; During the training iterations, previously unfiltered unstable images are included in the training dataset; as the training iterations proceed (let the total number of rounds be...),... ), when round At / 2, the globally unstable image set Incorporate it into training to expand the scope of pseudo-label generation; In each training iteration, the stability and prediction confidence of the pixels are recalculated based on the pixel prediction results of the current student model. Generating or updating pseudo-labels and weight matrices based on updated stability and confidence; Each interval Wheel (default) =5), recalculate the stability score for all pixels. Confidence level and category weights Update pseudo tags With weight matrix ; The student model is jointly trained using the updated pseudo-labels and pixel weights; Repeat the training steps iteratively until the predetermined number of training rounds is completed.

[0029] In this embodiment, the output of pavement rain and snow distribution results and early warning results specifically include: The segmentation result matrix generated by the trained SegFormer student model is converted into road rain and snow distribution results, where the category label of each pixel represents the corresponding rain and snow state. Regional clustering is performed on road rain and snow distribution results to identify continuous rain and snow disturbance areas; Based on the area of ​​rain and snow regions, potential risk areas are determined, and road rain and snow warning results are generated. In this embodiment, the percentage of pixels that are determined to be rain and snow regions is used as a threshold condition, and areas that are greater than the threshold are determined to be potential risk areas. The output includes a monitoring and early warning model that includes road rain and snow distribution results and early warning results.

[0030] Example 1: To verify the feasibility of this invention in practice, it was applied to a road rain and snow monitoring and early warning scenario. In this scenario, the road environment is complex, with interference factors such as rain, snow accumulation, shadows, water vapor, and light spots, making it difficult for traditional semantic segmentation methods to accurately distinguish rain and snow areas from the background. Labeling costs are high, and pseudo-labels are prone to noise. To solve these problems, this invention utilizes the SegFormer semantic segmentation network combined with multi-level pseudo-label filtering and pixel confidence-weighted training to achieve high-precision, low-cost rain and snow distribution identification and real-time early warning.

[0031] During implementation, a large number of road images were acquired, and a small number of images were manually labeled with rain and snow areas to construct a labeled dataset. Unlabeled images were saved as an unlabeled dataset. The labeled data was then input into the SegFormer training model to train the teacher model, and model checkpoints from multiple stages were saved during training to form a teacher model set. For unlabeled images, pseudo-labels were generated from the teacher model set. The consistency of the overall semantics of the same image output by different teacher models was compared to select image-level stable samples. In stable images, the consistency of each pixel across multiple model prediction categories was statistically analyzed to select pixel-level stable pixels to form pseudo-labels. The stable images and pixel pseudo-labels were then input into the initialized SegFormer student model for training to obtain preliminary segmentation results. By extracting scene features to generate real-observation semantic representations and road intrinsic semantic representations, regional distribution, boundary continuity, and local feature stability analyses were performed. Reasoning was applied to the differences between the real-observation semantic representations and the road intrinsic semantic representations to identify rain and snow disturbance areas, generating improved pseudo-labels and segmentation results. Finally, based on stable images, stable pixels, and improved segmentation results, stable and unstable pixels are jointly trained using pixel prediction confidence scores, with corresponding weights assigned. The student model parameters are continuously updated. During training, unfiltered images are gradually incorporated, and pseudo-labels and pixel weights are periodically updated. Through iterative optimization, the final rain and snow monitoring and early warning model is obtained, outputting road rain and snow distribution results and early warning information.

[0032] Table 1: Comparison of Road Rain and Snow Segmentation Detection Performance Based on Different Model Methods

[0033] The method of this invention demonstrates advantages in complex road rain and snow scenarios. The average intersection-union ratio reaches 83.5%, an improvement of approximately 5.2 percentage points compared to DeepLabv3+. The detection rate for small-area rain and snow reaches 72.2%, an improvement of 6.8 percentage points, indicating that this invention can effectively capture small-area rain and snow targets on roads and reduce missed detections. The pseudo-label accuracy reaches 78.9%, an improvement of more than 20% compared to existing semi-supervised methods such as Mean Teacher, further enhancing the reliability of model training. Notably, this invention can complete training using only 30% manually labeled data, reducing the amount of labeling by approximately 70% compared to traditional fully supervised methods, significantly reducing labor costs. The model convergence rounds are 150, about 25% fewer than conventional methods, accelerating training speed and ensuring stable model performance after convergence.

[0034] In summary, the method of this invention makes full use of unlabeled images and effectively solves the problems of high labeling costs, pseudo-label noise interference, and small target recognition in rain and snow scenarios by using multi-level view filtering, pixel confidence weighted training, and iterative optimization strategies. It improves segmentation accuracy, stability, and training efficiency, and provides a reliable technical means for road rain and snow monitoring and early warning.

[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning, characterized in that, Includes the following steps: Acquire road image data and construct labeled and unlabeled datasets; The SegFormer semantic segmentation network is trained based on a labeled dataset, and models from multiple training stages are saved to form a teacher model set. Unlabeled data is input into the teacher model set to generate pseudo-labels. The overall semantic consistency of the predictions of the same image by multiple teacher models is compared to select stable images. The consistency of each pixel in the multi-model prediction category is statistically analyzed to select pixel-level stable pixels to form pseudo-labels. The student model is initialized and trained using pseudo-labels to generate preliminary segmentation results. The road image is input into the SegFormer student model to extract scene features, and the real observation semantic representation and the road intrinsic semantic representation are generated based on the scene features respectively. Structural parsing is performed on the semantic representation, including analysis of regional distribution, boundary continuity, and local feature stability. By comparing the structural differences between the real observation semantic representation and the road intrinsic semantic representation, rain and snow disturbance areas are identified, and improved pseudo-labels and segmentation results are generated. Based on stable images, stable pixels, and improved segmentation results, the SegFormer student model parameters are updated by assigning corresponding confidence weights to stable and unstable pixels using pixel prediction confidence. During training, previously unfiltered images are included, pseudo-labels and pixel weights are updated periodically, and a pavement monitoring and early warning model for rain and snow weather is obtained through iterative optimization. The model outputs pavement rain and snow distribution results and early warning results.

2. The method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 1, characterized in that, The acquisition of road image data specifically includes: Set up a road image acquisition device to capture road scene images at a vehicle-mounted or roadside location; The acquired images are filtered, and samples with image blur exceeding a threshold are removed to form a preliminary image set; In the initial image set, rain and snow areas of some images are manually labeled, and rain, snow and background are distinguished pixel by pixel to generate a labeled dataset. Unlabeled images are saved as an unlabeled dataset. Perform basic normalization on unlabeled images to unify image resolution, color channels, and data format; The images were classified and labeled according to the time of acquisition, road type, and weather conditions. Indexes and storage structures are created for labeled and unlabeled datasets respectively, forming labeled and unlabeled datasets.

3. The method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 2, characterized in that, The training of the SegFormer semantic segmentation network based on the labeled dataset specifically includes: The labeled dataset is input into the encoder of the SegFormer semantic segmentation network to extract multi-scale scene features, and the decoder generates initial pixel-level semantic predictions. In each training iteration, the network output and the corresponding labeled data are cross-entropy loss is calculated, and backpropagation is performed to update the SegFormer network parameters based on the loss value. During training, each time a preset number of rounds is completed or a training phase node is reached, the current network parameters are saved as a model checkpoint, and the network state, training rounds, and corresponding loss information are recorded. Repeat the training until the preset total number of rounds is completed, and save all training stage checkpoints to form a set of teacher models.

4. The method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 3, characterized in that, The generation of preliminary segmentation results specifically includes: Unlabeled images are input batch by batch into each checkpoint model of the teacher model set, and a corresponding pixel-level predicted probability distribution is generated for each image. The consistency of the overall semantics of the same image output by different teacher models is compared, the average intersection-union ratio is calculated, and images with consistency higher than a preset threshold are classified as stable images. For each pixel in a stable image, the consistency of the predicted categories of multiple teacher models is statistically analyzed to calculate pixel-level stability. Pixels whose consistency meets the threshold are classified as stable pixels and pseudo-labels are generated. The stable image and the pseudo-labels corresponding to the pixels are input into the initialized SegFormer student model, and cross-entropy loss is used for training. Backpropagation is performed to update the network parameters based on the error between pixel prediction and pseudo-label. During the training process, after each preset number of rounds, the current student model parameters are saved as student model checkpoints; Repeat the training until the preset number of rounds is completed to generate preliminary segmentation results; During training, stable images and pixel-level stable pixel information are preserved.

5. A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning, as described in claim 4, is characterized in that... The SegFormer student model specifically includes: Based on the weather labels in the labeled dataset, the road images are divided into a rain and snow image set and a sunny or cloudy image set; Rain and snow images are input into the SegFormer student model encoder in batches to extract multi-scale features, and then the decoder generates a real observation semantic representation. The corresponding sunny or cloudy day images are input into the same SegFormer student model encoder in batches to extract multi-scale features, and the road intrinsic semantic representation is generated by the decoder. During the generation process, the multi-scale feature maps of each image are channel normalized and upsampled to the original image resolution, while retaining the class probability and spatial location of each pixel; Output the true observation semantic representation and the road intrinsic semantic representation, including the predicted probability of each pixel in all categories.

6. The method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 5, characterized in that, The execution structure parsing specifically includes: The student model outputs a pixel-by-pixel aligned and compared the true observation semantic representation with the road intrinsic semantic representation of the corresponding scene, and the difference in the predicted probability of each pixel across all categories is calculated. Based on the difference value, pixels that are continuous and have a probability difference exceeding a preset threshold are aggregated to form candidate rain and snow disturbance regions; For each candidate region, calculate boundary continuity metrics, including boundary length, curvature, and neighborhood pixel class consistency, to identify isolated pixels and broken boundaries; Local stability analysis is performed on the class probabilities of pixels within the candidate region, and isolated pixels are corrected by weighted averaging of the probabilities of neighboring pixels. Improved pseudo-labels are generated for the corrected candidate regions, and the improved pseudo-labels are input into the student model to form the final segmentation result matrix; Output the improved pseudo-label and segmentation result matrix, including the predicted probability of each pixel in all categories and its corresponding spatial coordinates.

7. The method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 6, characterized in that, The joint training update of the SegFormer student model parameters specifically includes: Input the stable image and its corresponding stable pixels into the SegFormer student model, extract multi-scale features and generate pixel-level predictions through the decoder; For each pixel, a weight coefficient is calculated based on the stable pixel identifier and the predicted probability. Stable pixels are assigned a fixed weight, while unstable pixels are assigned a weight related to the predicted probability. The pixel weights are combined with the cross-entropy loss function to calculate the weighted loss for each batch of images; The weighted loss is backpropagated to update the parameters of the SegFormer student model. During the training process, after each preset round, the current model parameters are saved as a training checkpoint, and the batch index, loss value and weight distribution information are recorded. Repeat the training until the total number of rounds is completed, maintaining stable images and indexes of stable pixel information; In each training round, the pixel prediction probabilities after updating the student model parameters are used for the next round of weighted loss calculation, forming a joint training loop; Output the trained SegFormer student model parameters and pixel-level prediction matrix, and generate the final improved pseudo-labels and segmentation results.

8. A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning according to claim 7, characterized in that, The iterative optimization specifically includes: Iterative training is performed on the stable images and stable pixels obtained during the training process of the student model; During the training iterations, previously unfiltered unstable images are included in the training dataset; In each training iteration, the stability and prediction confidence of the pixels are recalculated based on the pixel prediction results of the current student model. Generating or updating pseudo-labels and weight matrices based on updated stability and confidence; The student model is jointly trained using the updated pseudo-labels and pixel weights; Repeat the training steps iteratively until the predetermined number of training rounds is completed.

9. A method for real-time monitoring and early warning of road surfaces in rain and snow weather based on deep learning, as described in claim 8, is characterized in that, The output pavement rain and snow distribution results and early warning results specifically include: The segmentation result matrix generated by the trained SegFormer student model is converted into road rain and snow distribution results, where the category label of each pixel represents the corresponding rain and snow state. Regional clustering is performed on road rain and snow distribution results to identify continuous rain and snow disturbance areas; Based on the area of ​​rain and snow, potential risk areas are identified, and road rain and snow warning results are generated. The output includes a monitoring and early warning model that includes road rain and snow distribution results and early warning results.

Images