Semantic segmentation method, device and equipment for road snow scene, and storage medium

Abstract

The present disclosure provides a road rain and snow scene semantic segmentation method, system, device and storage medium. The method comprises: acquiring an unlabeled data set under a road rain and snow scene; based on a preset first pixel-level labeling model, performing multi-level pixel-level classification label prediction on each road rain and snow image in the unlabeled data set to obtain a multi-level first pseudo label prediction result corresponding to each road rain and snow image; based on the multi-level first pseudo label labeling result corresponding to each road rain and snow image, determining the global stability and the pixel stability of each road rain and snow image; and using the classification label of the stable pixels in the globally stable road rain and snow image to train a road rain and snow scene semantic segmentation model. The present disclosure fuses the global stability and the pixel-level stability of the road rain and snow image, screens and constructs high-confidence pseudo labels, and then trains a high-precision road rain and snow scene semantic segmentation model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and in particular to a semantic segmentation method, apparatus, device and storage medium for road rain and snow scenes. Background Technology

[0002] In road safety monitoring, slippery roads caused by rain and snow are one of the main causes of traffic accidents. Achieving pixel-level accurate detection through semantic segmentation is the key to ensuring driving safety.

[0003] Existing general semantic segmentation methods mainly employ supervised or semi-supervised learning in road rain and snow scenarios. These two types of algorithms have the following drawbacks: First, rain and snow have irregular shapes and blurred edges. Supervised learning requires manual pixel-level annotation, which is time-consuming, labor-intensive, and inefficient, severely restricting the large-scale application of the model. Second, semi-supervised methods usually rely on a single model to generate pseudo-labels, which is difficult to cope with the confusion between rain and snow and complex backgrounds such as road stains and shadows, resulting in frequent missed and mislabeled phenomena, introducing a large amount of noise interference, and seriously affecting the stability of model training and segmentation accuracy.

[0004] Therefore, how to reduce the cost of manual annotation while effectively improving the quality of pseudo-label generation and achieving high-precision semantic segmentation of road rain and snow scenes is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, this disclosure proposes a semantic segmentation method, apparatus, device and storage medium for road rain and snow scenes, which can effectively improve the generation quality of pseudo-tags and achieve high-precision semantic segmentation of road rain and snow scenes.

[0006] According to a first aspect of this disclosure, a semantic segmentation method for road rain and snow scenes is provided, comprising: A semantic segmentation model for road rain and snow scenes is used to predict pixel-level classification labels for the latest acquired road rain and snow images, in order to segment the rain and snow areas on the road from the latest road rain and snow images. The training process of the semantic segmentation model for the road rain and snow scene is as follows: Obtain an unlabeled dataset of road rain and snow scenes, wherein the unlabeled dataset includes only multiple unlabeled road rain and snow images; Based on the preset first pixel-level labeling model, multi-level pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the multi-level first pseudo-label prediction results corresponding to each of the road rain and snow images. Based on the multi-level first pseudo-label annotation results corresponding to each of the road rain and snow images, the global stability and pixel stability of each of the road rain and snow images are determined. A semantic segmentation model for the road rain and snow scene is obtained by training the classification labels of stable pixels in globally stable road rain and snow images.

[0007] In one possible implementation, when performing multi-level pixel-level classification label prediction on each of the road rain and snow images in the unlabeled dataset based on the first pixel-level annotation model to obtain the multi-level first pseudo-label prediction result corresponding to each of the road rain and snow images, the process includes: During the training process of the first pixel-level labeled model, multiple model checkpoints are saved; Using each of the model checkpoints, pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the pseudo-label prediction results of each road rain and snow image under multiple model checkpoints; The pseudo-label prediction results of each road rain and snow image at multiple model checkpoints are used as the multi-level first pseudo-label prediction results corresponding to each road rain and snow image.

[0008] In one possible implementation, determining the global stability and pixel stability of each road rain and snow image based on the multi-level first pseudo-label annotation results corresponding to each of the road rain and snow images includes: The global stability score of each road rain and snow image is calculated based on the multi-level first pseudo-label annotation results, and the global stability of each road rain and snow image is determined by the global stability score. Pixel-level stability scores are calculated for each pixel in each of the road rain and snow images based on the multi-level first pseudo-label annotation results, and pixel stability is determined for each pixel in each of the road rain and snow images based on the pixel-level stability scores.

[0009] In one possible implementation, when training a semantic segmentation model for the road rain and snow scene using classification labels of stable pixels in a globally stable road rain and snow image, the following steps are included: A second-pixel-level annotation model is obtained by training the classification labels of stable pixels in globally stable road rain and snow images; Based on the second pixel-level labeling model, pixel-level classification label prediction is performed again on each of the road rain and snow images in the unlabeled dataset to obtain the second pseudo-label prediction result corresponding to each of the road rain and snow images; Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight and stability weight of each pixel in each of the road rain and snow images are calculated. Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight of each pixel and the stability weight of each pixel, a third pixel-level annotation model is trained, and the third pixel-level annotation model is used as the semantic segmentation model for road rain and snow scenes.

[0010] In one possible implementation, when calculating the classification weight of each pixel in each of the road rain and snow images based on the second pseudo-label prediction result corresponding to each of the road rain and snow images, the calculation includes: Based on the prediction results of the second pseudo-labels corresponding to each of the road rain and snow images, the number of category samples corresponding to each classification label is counted. The category weight of each category label is determined based on the number of category samples corresponding to each category label; In one possible implementation, when calculating the stability weight of each pixel in each of the road rain and snow images based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the calculation includes: Obtain the prediction probability distribution of each pixel in the second pseudo-label prediction result corresponding to each of the road rain and snow images; The prediction confidence level of the corresponding pixel is determined based on the prediction probability distribution of each pixel. The stability weight of each pixel is determined based on the prediction confidence of each pixel.

[0011] In one possible implementation, when training the third pixel-level annotation model based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight of each pixel, and the stability weight of each pixel, the following steps are included: Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight of each pixel and the stability weight of each pixel, the loss value corresponding to each of the road rain and snow images is calculated. The third pixel-level annotation model is trained based on the loss values ​​corresponding to the rain and snow images of each road.

[0012] According to a second aspect of this disclosure, a semantic segmentation apparatus for road rain and snow scenes is provided, including... Semantic segmentation module: Employs a semantic segmentation model for road rain and snow scenes to predict pixel-level classification labels for the latest acquired road rain and snow images, thereby segmenting the rain and snow areas on the road from the latest road rain and snow images. Model training module: Obtain an unlabeled dataset of road rain and snow scenes, wherein the unlabeled dataset only includes multiple unlabeled road rain and snow images; Based on the preset first pixel-level labeling model, multi-level pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the multi-level first pseudo-label prediction results corresponding to each of the road rain and snow images. Based on the multi-level first pseudo-label annotation results corresponding to each of the road rain and snow images, the global stability and pixel stability of each of the road rain and snow images are determined. A semantic segmentation model for the road rain and snow scene is obtained by training the classification labels of stable pixels in globally stable road rain and snow images.

[0013] According to a third aspect of this disclosure, a semantic segmentation device for road rain and snow scenes is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in the first aspect of this disclosure.

[0014] According to a fourth aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the method described in the first aspect of this disclosure.

[0015] This disclosure provides a semantic segmentation method, apparatus, device, and storage medium for road rain and snow scenes. The method includes: acquiring an unlabeled dataset of road rain and snow scenes; performing multi-level pixel-level classification label prediction on each road rain and snow image in the unlabeled dataset based on a preset first pixel-level labeling model to obtain multi-level first pseudo-label prediction results for each road rain and snow image; determining the global stability and pixel stability of each road rain and snow image based on the multi-level first pseudo-label labeling results; and training a semantic segmentation model for the road rain and snow scene using the classification labels of stable pixels in the globally stable road rain and snow image. This disclosure achieves high-precision semantic segmentation of road rain and snow scenes by fusing the global stability and pixel-level stability of road rain and snow images, selecting and constructing high-confidence pseudo-labels, and then training the model.

[0016] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0018] Figure 1 A flowchart illustrating a semantic segmentation method for a road rain and snow scene according to an embodiment of the present disclosure is shown; Figure 2 This diagram illustrates the method architecture of a semantic segmentation method for a road rain and snow scene according to an embodiment of the present disclosure; Figure 3A schematic block diagram of a semantic segmentation apparatus for a road rain and snow scene according to an embodiment of the present disclosure is shown; Figure 4 A schematic block diagram of a semantic segmentation device for a road rain and snow scene according to an embodiment of the present disclosure is shown. Detailed Implementation

[0019] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0020] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0021] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0022] <Method Implementation> Figure 1 A flowchart illustrating a page management method based on a task manager according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes steps S1100-S1500: S1100: Obtain an unlabeled dataset of road rain and snow scenes. The unlabeled dataset only includes multiple unlabeled road rain and snow images.

[0023] Specifically, road images under different weather conditions (including rain, snow, sleet, and clear skies) are collected using vehicle-mounted cameras or roadside monitoring equipment (i.e., unlabeled road images in rain and snow) to form an unlabeled dataset for road rain and snow scenarios. This unlabeled dataset is denoted as... ,in The images are rain and snow images of various roads in an unlabeled dataset. This represents the total number of images in the dataset.

[0024] S1200 performs multi-level pixel-level category label prediction on each road rain and snow image in the unlabeled dataset based on the preset first pixel-level annotation model, and obtains the multi-level first pseudo-label prediction results corresponding to each road rain and snow image.

[0025] First, it should be noted that the first pixel-level annotation model is obtained by fully supervised training of a pre-defined semantic segmentation network (such as deeplabv3+, UNet, etc.) on a labeled dataset (including a small number of road rain and snow images, and each road rain and snow image is labeled with a pixel-level category label).

[0026] Meanwhile, during the training of the first pixel-level annotation model, multiple model checkpoints are saved. Each model checkpoint is used to perform pixel-level classification label prediction on each of the road rain and snow images in the unlabeled dataset, so as to obtain the pseudo-label prediction results of each road rain and snow image under multiple model checkpoints.

[0027] In one possible implementation, during the fully supervised training phase of the first pixel-level annotation model, the data is saved... Model checkpoints (Right now, Figure 2 The teacher model shown is T_1...Teacher model T_R). After the first pixel-level labeled model is trained, the saved... Each model checkpoint for the same unlabeled image Labeling is performed to obtain the checkpoints of each model for the unlabeled image. Label prediction probability distribution for each pixel Here, model checkpoints refer to saving the model parameters in the current training state at preset time intervals during the model training process. From the saved model parameters, a complete model with corresponding reasoning capabilities can be directly reconstructed.

[0028] Among them, the label prediction probability distribution , Indicates the first Each model checkpoint predicts pixels For the first The probability of the class label is calculated, and then the label prediction probability distribution is calculated based on the output of each model checkpoint. Determine the first Each model checkpoint predicts pixels Category ,in, Finally, checkpoints for each model. For adjacent pixels with the same prediction result (i.e., Connectivity aggregation is performed on adjacent pixels with the same value to generate the corresponding pseudo-mask. .

[0029] Furthermore, based on the label prediction probability distribution output by each model checkpoint... Determine pixels pseudo-category labels ,in, That is, for The predicted probabilities of each model checkpoint are taken as an arithmetic mean, and the class with the highest average predicted probability is selected as the pseudo-class label for that pixel.

[0030] The probability distribution of label prediction for each pixel in each unlabeled road rain and snow image is calculated based on the checkpoints of each model. Pseudomas generated for rain and snow images of unlabeled roads and the final predicted pseudo-class label This serves as the first pseudo-label annotation result for the multi-level system.

[0031] S1300 determines the global stability and pixel stability of each road rain and snow image based on the multi-level first pseudo-label annotation results corresponding to each road rain and snow image.

[0032] First, the global stability score of each unlabeled road rain and snow image is calculated based on the multi-level first pseudo-label annotation results, and the global stability of each road rain and snow image is determined by the global stability score.

[0033] In one possible implementation, for each unlabeled road rain / snow image, the first pixel-level annotation model after training is used. Generated pseudo mask As a reference mask, calculate the checkpoints for each model. Output pseudomask With reference mask The average intersection-union ratio between them, and take this The average of the average crossover ratios is used as the global stability score for each unlabeled road rain and snow image.

[0034] Specifically, for unlabeled road rain and snow images Image stability score The calculation formula is as follows: in, Assign a score to the global stability score; This represents the number of checkpoints in the model. Model checkpoints for the first pixel-level annotation model For unlabeled image The generated pseudo-mask; The first pixel-level labeled model after training For unlabeled image The generated pseudo-mask; The average intersection-union ratio formula is used to measure the pseudomask. With pseudo-mask The degree of overlap between them.

[0035] Image stability score The range of values ​​is , The higher the value, the higher the image quality, the more stable the prediction of the first pixel-level annotation model, and the less affected by noise.

[0036] Furthermore, based on image stability scores The unlabeled images are sorted from highest to lowest quality, and the top-ranked unlabeled images are identified as globally stable road rain and snow images (i.e., Figure 2 The unlabeled image corresponding to the stable mask shown is used; the remaining unlabeled images are determined to be globally unstable road rain and snow images (i.e., Figure 2 (The unlabeled image corresponding to the unstable mask shown). For example, an image stability ratio threshold can be preset. As a criterion for global stability (i.e., ranking images with higher stability scores), Unlabeled images are classified as globally stable road rain / snow images. The image stability ratio threshold is used as an example. The threshold can be flexibly adjusted according to the complexity of rainy / snowy road scenes: when the scene complexity is high, the image stability ratio threshold can be appropriately lowered to expand the selection range of stable samples; when the scene complexity is low, the image stability ratio threshold can be appropriately increased to improve the quality of stable samples. Image stability threshold The preferred value range can be set to [30%, 70%].

[0037] Secondly, based on the multi-level first pseudo-label annotation results, the pixel-level stability score of each pixel in the rain and snow images of each road is calculated, and the pixel stability of each pixel in the rain and snow images of each road is determined by the pixel-level stability score.

[0038] In one possible implementation, each model checkpoint For each pixel Predict the pseudo-category label Calculate the pixel-level stability score for each pixel. ,in The specific calculation formula is as follows: in, Pixel-level stability score; For each model checkpoint For pixels The predicted pseudo-class label. This is only true if each model checkpoint is pixel-wise. When all predicted categories are consistent, In other cases .

[0039] Furthermore, pixel-level stability scores Pixels with a value of 1 are considered stable pixels, and pixel-level stability scores are calculated. Pixels with a value of 0 are considered unstable pixels.

[0040] S1400 uses pseudo-class labels of stable pixels in globally stable road rain and snow images to train a semantic segmentation model for road rain and snow scenes.

[0041] First, a second pixel-level annotation model is trained using the globally stable road rain and snow image obtained in step S1300 and the pseudo-class labels of stable pixels (i.e., Figure 2 The student model shown is used. Specifically, the second pixel-level annotation model is obtained by fully supervised training of a pre-defined semantic segmentation network (such as Deeplabv3+, UNet, etc.) based on globally stable road rain and snow images and pseudo-class labels of stable pixels (that is, the training data of the second pixel-level annotation model is stable pixels on globally stable road rain and snow images, and the data labels are pseudo-class labels of stable pixels).

[0042] Based on the second pixel-level annotation model, pixel-level category label prediction is performed again on the rain and snow images of each road in the unlabeled dataset to obtain the second pseudo-label prediction results corresponding to each unlabeled rain and snow image.

[0043] In one possible implementation, a second-pixel-level annotation model is used to annotate the same unlabeled road rain and snow image. Labeling is performed to obtain a second-pixel level labeling model for each pixel in each unlabeled road rain and snow image. Predicted pseudo-category labels .

[0044] Specifically, the second pixel-level annotation model targets each pixel. Calculate the label prediction probability distribution ,in, , This indicates the pixel predicted by the second-pixel level annotation model. The probability of class c is then determined. Subsequently, the class corresponding to the maximum probability in the label prediction probability distribution for each pixel is identified as the class of the second pixel-level labeling model for that pixel. Predicted pseudo-category labels That is, the class with the highest predicted probability in the second-pixel level labeling model is selected as the predicted pseudo-class label for that pixel. The label prediction probability distribution is then used. and pseudo-category labels for each pixel This serves as the second pseudo-label prediction result.

[0045] Secondly, based on the second pseudo-label prediction results corresponding to the rain and snow images of each road, the category weight and stability weight of each pixel in the rain and snow images of each road are calculated.

[0046] In one possible implementation, the calculation process for the class weight of each pixel is as follows: First, based on the second pseudo-label prediction results of each road rain and snow image, count the number of pixels corresponding to each pseudo-category label. (i.e., the number of category samples); subsequently, based on the number of pixels. The category weight of each pseudo-category label is determined by a preset weight calculation strategy. Finally, for each pixel in the rain and snow images of each road, the category weight corresponding to its pseudo-category label is calculated. The value is assigned to the pixel as its class weight.

[0047] Category weight The specific calculation formula is as follows: in, Category weights; The pseudo-class label predicted by the second-pixel level annotation model is The number of pixels; This is a balancing coefficient used to control the degree to which the number of samples in each category affects the category weights. Typically, The settings can be adjusted based on the class imbalance in the road rain and snow dataset. When the pixel proportion of target categories such as rain and snow is low and the class distribution is severely imbalanced, the settings can be adjusted accordingly. Set a value closer to 1 to enhance weight compensation for a minority of categories; when the number of pixels in each category is relatively balanced, the value can be appropriately reduced. This avoids the instability of training caused by excessively large weights for a few categories. The preferred value is 0.99.

[0048] It's important to note that in road rain and snow scenarios, not only are there few samples for rain and snow categories, but their visual features are also highly confused with fog, glare, and slippery road surfaces. This results in generated pseudo-labels inherently possessing both "low confidence" and "category sparsity." Calculating category weights... The purpose is to compensate for signal loss by assigning higher loss weights to scarce categories, so that the model can fully learn the feature representations of long-tail categories such as rain and snow during training, and avoid underfitting caused by the gradient dominance of the dominant category.

[0049] In one possible implementation, the calculation process for the stability weights of each pixel is as follows: First, based on the second pseudo-label prediction results of each road rain and snow image, the prediction confidence of each pixel is determined. Its value is taken from the second pixel-level annotation model for pixels. Output label prediction probability distribution The maximum value in the range is used to characterize the confidence level of the prediction for that pixel category; subsequently, the prediction confidence levels of each pixel are combined. and the pixel-level stability score of each pixel determined in step S1300. Calculate the stability weights for each pixel.

[0050] Stability weight The specific calculation formula is as follows: in, This is the stability weight, and its range is... ; To predict confidence levels; Pixel-level stability score.

[0051] For stable pixels (i.e., The second pseudo-label prediction result has high reliability, therefore it is given the maximum training weight (i.e., pixels) during the training process of the semantic segmentation model for road rain and snow scenes. ), to fully utilize high-quality pseudo-labels to enhance the model's feature learning for deterministic regions; for unstable pixels (i.e., (pixels), due to the uncertainty of the second pseudo-label prediction result, to avoid noise interference introduced by erroneous labels, a prediction confidence level is used. Assign lower training weights to unstable pixels ( This can effectively reduce the chances of the model learning incorrect knowledge.

[0052] Finally, based on the second pseudo-label prediction results corresponding to each road rain and snow image, the classification weight of each pixel, and the stability weight of each pixel, the loss value corresponding to each road rain and snow image is calculated; based on the loss value corresponding to each road rain and snow image, the third pixel-level annotation model is trained.

[0053] In one possible implementation, a confidence-weighted cross-entropy loss function is constructed based on the second pseudo-label prediction result, the classification weight of each pixel, and the stability weight of each pixel.

[0054] Confidence-weighted cross-entropy loss function The specific calculation formula is as follows: in, This is a dataset of unlabeled road images in rain and snow. This represents the total number of images in the dataset. This represents the set of pixels in a single image that are used in model training. This indicates the total number of pixels in the set; That is, the pixel-level stability weights calculated above (when pixel When stabilizing pixels When pixel When for unstable pixels ); The pseudo-category labels (such as road, vehicle, pedestrian, rain, snow, etc.) for semantic segmentation determined by the second pseudo-label prediction results. The total number of categories for semantic segmentation; The category weights calculated earlier assign greater loss weights to long-tail categories with scarce samples (such as rain and snow) and smaller weights to dominant categories (such as roads). For indicator functions, that is, only when the pixel The pseudo-tag category is When the condition is met, the indicator function takes a value of 1; otherwise, it takes a value of 0, ensuring that the log loss is calculated only for the probability term corresponding to the predicted category. This represents the semantic segmentation model to be trained (with parameters). For the input image medium pixel The predicted pseudo-category label is The probability output, This is the log-likelihood term of the standard cross-entropy.

[0055] Another point worth noting is that the innovation of the confidence-weighted cross-entropy loss function lies in the introduction of pixel-level stability weights. and category weights The loss value is calculated, and the remainder of the confidence-weighted cross-entropy loss function is the standard pixel-level cross-entropy loss function. Pixel-level stability weights are introduced. and category weights The goal is to enable the trained semantic segmentation model to achieve synergistic optimization of "noise resistance" and "equalization" on unlabeled road rain and snow images. On the one hand, through... Pixel-level confidence assessment of pseudo-labels ensures the model acquires strong supervision signals only from high-confidence regions, while automatically reducing learning intensity for low-confidence regions. This effectively avoids the contamination of model parameters by erroneous pseudo-labels caused by interference factors such as rain and snow occlusion and blurring. Furthermore, class weights... The model is forced to focus on long-tail categories such as rain and snow, which have scarce samples but are critical to safety, to avoid underfitting of long-tail categories caused by dominant categories such as roads and background having an absolute advantage in the loss.

[0056] The loss value corresponding to each road rain and snow image is calculated based on the confidence-weighted cross-entropy loss function, and then the third pixel-level annotation model is trained based on the loss value.

[0057] In one possible implementation, the training of the third-pixel-level annotation model directly employs a single-stage unsupervised learning strategy. Specifically, it utilizes the pseudo-class labels of each pixel output by the second-pixel-level annotation model. As pseudo-labels, and combined with the confidence-weighted cross-entropy loss function mentioned above. Calculate the loss value corresponding to each road image in rain and snow. This process utilizes pixel-level stability weights. With category weight The synergistic effect of these technologies allows for end-to-end training of the third-pixel-level annotation model directly on unlabeled data, enabling it to learn semantic features in road rain and snow scenarios. This approach is suitable for scenarios where a high-quality second-pixel-level annotation model already exists, computational resources are limited, and rapid deployment is required.

[0058] In another possible implementation, to further improve the model's robustness in complex rain and snow environments, the training process of the third model is divided into two progressive phases: Phase 1: Pseudo-class labels for each pixel output by the first pixel-level annotation model. As the first pseudo-label, a standard pixel-level cross-entropy loss function is constructed as the loss function for the first round of training (i.e., ...). Figure 2 The stable pixel unsupervised loss is shown. Since the first pixel-level labeling model focuses on basic semantic recognition, its output pseudo-labels, although potentially noisy in detail, provide global semantic priors. Preliminary training using standard cross-entropy loss allows the third model to quickly converge to a better parameter space, establishing a basic perception capability for dominant categories such as roads and vehicles.

[0059] Phase Two: Building upon Phase One, switch to using the pseudo-class labels for each pixel output by the second-pixel-level annotation model. As the second pseudo-label, the confidence-weighted cross-entropy loss function described above is constructed. This serves as the loss function for the second round of training. At this point, the model possesses basic semantic understanding capabilities, and stability weights are introduced. It can effectively filter out local noise caused by rain and snow interference in the second round of pseudo-labeling, while also adjusting the category weights. The guided model focuses on correcting prediction biases for long-tail categories such as rain and snow, achieving a performance leap from "rough perception" to "precise segmentation".

[0060] Secondly, as a preferred implementation, in order to make full use of the limited labeled data resources and maximize the value of unlabeled data, the labeled dataset used when training the first pixel-level labeled model is introduced, a supervised loss function is constructed, and the above two-stage unsupervised training is integrated with it to form a joint training framework of "supervised + two-stage unsupervised".

[0061] Specifically, the total loss function of the third model Loss due to supervision Unsupervised loss The weighted summation consists of: in, and Losses are categorized as supervised and unsupervised losses, respectively. This is the unsupervised loss weight, used to balance the supervisory effects of labeled and unlabeled data. For example, It can linearly increase from 0.1 to 1.0 with each training round; that is, let the total number of training rounds be... The current round is Then the current round It can be done (t) represents, .

[0062] Phase 1 (i.e., the current training round) Before): Based on the first pseudo-label The standard pixel-level cross-entropy loss is constructed. In this stage, the supervised loss provides accurate gradient guidance to prevent the model from overfitting prematurely on low-quality pseudo-labels; the unsupervised loss uses a large amount of unlabeled data to expand the generalization boundary of the model, and the two complement each other to complete the alignment of basic features.

[0063] The second phase (i.e., the current training round is in) arrive ): Switch to using the second pseudo-tag Constructed confidence-weighted cross-entropy loss function As model performance improves, supervised loss continuously anchors the accuracy of key samples, while the confidence-weighted cross-entropy loss function can suppress noise while specifically enhancing the learning of difficult-to-distinguish samples such as rain and snow.

[0064] This preferred approach effectively mitigates the risk of pseudo-label error accumulation during pure unsupervised training by providing supervised signals throughout the process. At the same time, the progressive design of the two-stage unsupervised loss enables the model to gradually extract semantic information from massive amounts of unlabeled data, significantly improving the detection accuracy and stability of the final semantic segmentation model in real road rain and snow scenarios.

[0065] S1500 uses a semantic segmentation model for road rain and snow scenes to predict pixel-level classification labels for the latest road rain and snow images, and the semantic segmentation model for road rain and snow scenes segments the rain and snow areas on the road from the latest road rain and snow images.

[0066] The trained third-pixel-level annotation model is designated as the semantic segmentation model for road rain and snow scenes, and this model is used to infer the latest road rain and snow images. Specifically, the latest road rain and snow image is input into the semantic segmentation model, which outputs the classification label prediction results for each pixel in the image. Based on the classification label prediction results for each pixel, the rain and snow areas on the road are segmented from the latest road rain and snow image, thereby achieving pixel-level localization and recognition of the rain and snow coverage area in the actual road scene.

[0067] <Device Embodiment> Figure 3 A schematic block diagram of a semantic segmentation apparatus for a road rain and snow scene according to an embodiment of the present disclosure is shown. Figure 3 As shown, the device 100 includes: Semantic segmentation module 110: Employs a semantic segmentation model for road rain and snow scenes to predict pixel-level classification labels for the latest acquired road rain and snow images, thereby segmenting the rain and snow areas on the road from the latest road rain and snow images. Model training module 120: Obtain an unlabeled dataset of road rain and snow scenes, wherein the unlabeled dataset only includes multiple unlabeled road rain and snow images; Based on the preset first pixel-level annotation model, multi-level pixel-level classification label prediction is performed on each road rain and snow image in the unlabeled dataset to obtain the multi-level first pseudo-label prediction results corresponding to each road rain and snow image. Based on the multi-level first pseudo-label annotation results corresponding to the rain and snow images of each road, the global stability and pixel stability of the rain and snow images of each road are determined. A semantic segmentation model for road rain and snow scenes is trained using the classification labels of stable pixels in globally stable road rain and snow images.

[0068] <Equipment Example> Figure 4 A schematic block diagram of a semantic segmentation device for a road rain and snow scene according to an embodiment of the present disclosure is shown. Figure 4As shown, the page management device 200 includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the page snapshot management methods described above when executing the executable instructions.

[0069] It should be noted here that the number of processors 210 can be one or more. Furthermore, the semantic segmentation device 200 for a road rain and snow scene according to an embodiment of this disclosure may also include an input device 230 and an output device 240. The processors 210, memory 220, input device 230, and output device 240 can be connected via a bus or other means, without specific limitations here.

[0070] The memory 220, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the semantic segmentation method for a road rain and snow scene according to an embodiment of this disclosure. The processor 210 executes various functional applications and data processing of the page management device 200 by running the software program or module stored in the memory 220.

[0071] Input device 230 can be used to receive input digital numbers or signals. These signals may include key signals related to user settings and function control of the device / terminal / server. Output device 240 may include a display device such as a screen.

[0072] <Storage Medium Examples> According to a fourth aspect of this disclosure, a non-volatile computer-readable storage medium is also provided, on which computer program instructions are stored, which, when executed by processor 210, implement any of the page snapshot management methods described above.

[0073] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A semantic segmentation method for road rain and snow scenes, characterized in that, include: A semantic segmentation model for road rain and snow scenes is used to predict pixel-level classification labels for the latest acquired road rain and snow images, in order to segment the rain and snow areas on the road from the latest road rain and snow images. The training process of the semantic segmentation model for the road rain and snow scene is as follows: Obtain an unlabeled dataset of road rain and snow scenes, wherein the unlabeled dataset includes only multiple unlabeled road rain and snow images; Based on the preset first pixel-level labeling model, multi-level pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the multi-level first pseudo-label prediction results corresponding to each of the road rain and snow images. Based on the multi-level first pseudo-label annotation results corresponding to each of the road rain and snow images, the global stability and pixel stability of each of the road rain and snow images are determined. A semantic segmentation model for the road rain and snow scene is obtained by training the classification labels of stable pixels in globally stable road rain and snow images.

2. The method according to claim 1, characterized in that, When performing multi-level pixel-level classification label prediction on each of the road rain and snow images in the unlabeled dataset based on the first pixel-level annotation model to obtain the multi-level first pseudo-label prediction result corresponding to each of the road rain and snow images, the process includes: During the training process of the first pixel-level labeled model, multiple model checkpoints are saved; Using each of the model checkpoints, pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the pseudo-label prediction results of each road rain and snow image under multiple model checkpoints; The pseudo-label prediction results of each road rain and snow image at multiple model checkpoints are used as the multi-level first pseudo-label prediction results corresponding to each road rain and snow image.

3. The method according to claim 1, characterized in that, When determining the global stability and pixel stability of each road rain and snow image based on the multi-level first pseudo-label annotation results corresponding to each of the aforementioned road rain and snow images, the following steps are included: The global stability score of each road rain and snow image is calculated based on the multi-level first pseudo-label annotation results, and the global stability of each road rain and snow image is determined by the global stability score. Pixel-level stability scores are calculated for each pixel in each of the road rain and snow images based on the multi-level first pseudo-label annotation results, and pixel stability is determined for each pixel in each of the road rain and snow images based on the pixel-level stability scores.

4. The method according to claim 1, characterized in that, When training a semantic segmentation model for the road rain and snow scene using classification labels of stable pixels in globally stable road rain and snow images, the following steps are included: A second-pixel-level annotation model is obtained by training the classification labels of stable pixels in globally stable road rain and snow images; Based on the second pixel-level labeling model, pixel-level classification label prediction is performed again on each of the road rain and snow images in the unlabeled dataset to obtain the second pseudo-label prediction result corresponding to each of the road rain and snow images; Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight and stability weight of each pixel in each of the road rain and snow images are calculated. Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight of each pixel and the stability weight of each pixel, a third pixel-level annotation model is trained, and the third pixel-level annotation model is used as the semantic segmentation model for road rain and snow scenes.

5. The method according to claim 4, characterized in that, When calculating the classification weight of each pixel in each of the road rain and snow images based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the calculation includes: Based on the prediction results of the second pseudo-labels corresponding to each of the road rain and snow images, the number of category samples corresponding to each classification label is counted. The category weight of each category label is determined based on the number of category samples corresponding to each category label; The category weight of each pixel in the road rain and snow images is determined as the classification weight of the corresponding pixel.

6. The method according to claim 4, characterized in that, When calculating the stability weight of each pixel in each of the road rain and snow images based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the calculation includes: Obtain the prediction probability distribution of each pixel in the second pseudo-label prediction result corresponding to each of the road rain and snow images; The prediction confidence level of the corresponding pixel is determined based on the prediction probability distribution of each pixel. The stability weight of each pixel is determined based on the prediction confidence of each pixel.

7. The method according to claim 4, characterized in that, When training a third pixel-level annotation model based on the second pseudo-label prediction results corresponding to each of the aforementioned road rain and snow images, the classification weight of each pixel, and the stability weight of each pixel, the training includes: Based on the second pseudo-label prediction results corresponding to each of the road rain and snow images, the classification weight of each pixel and the stability weight of each pixel, the loss value corresponding to each of the road rain and snow images is calculated. The third pixel-level annotation model is trained based on the loss values ​​corresponding to the rain and snow images of each road.

8. A semantic segmentation device for road rain and snow scenes, characterized in that, include: Semantic segmentation module: Employs a semantic segmentation model for road rain and snow scenes to predict pixel-level classification labels for the latest acquired road rain and snow images, thereby segmenting the rain and snow areas on the road from the latest road rain and snow images. Model training module: Obtain an unlabeled dataset of road rain and snow scenes, wherein the unlabeled dataset only includes multiple unlabeled road rain and snow images; Based on the preset first pixel-level labeling model, multi-level pixel-level classification label prediction is performed on each of the road rain and snow images in the unlabeled dataset to obtain the multi-level first pseudo-label prediction results corresponding to each of the road rain and snow images. Based on the multi-level first pseudo-label annotation results corresponding to each of the road rain and snow images, the global stability and pixel stability of each of the road rain and snow images are determined. A semantic segmentation model for the road rain and snow scene is obtained by training the classification labels of stable pixels in globally stable road rain and snow images.

9. A semantic segmentation device for road rain and snow scenes, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 7 when executing the executable instructions.

10. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images