Road scene semantic segmentation method based on unsupervised traffic element alignment and related device

Abstract

The application discloses a kind of unsupervised traffic element alignment road scene semantic segmentation method and related device, comprising the following steps: obtaining the traffic scene image to be segmented;The traffic scene image to be segmented is input to calculate using deep learning model, and the road scene semantic segmentation result is obtained;The deep learning model is obtained by training convolutional neural network using total loss function;Total loss function is the combination of first cross-entropy loss function, second cross-entropy loss function, dynamic traffic element alignment loss and static traffic element alignment loss.The application determines total loss function by innovatively aligning dynamic and static traffic elements, divides the entire image into dynamic traffic elements and static traffic elements according to the characteristics of elements by processing, and designs dynamic and static element alignment loss function, so that the unsupervised semantic segmentation method is significantly improved in accuracy and does not need to be labeled;The segmentation capability is improved, so as to realize the accurate identification of road scene.

Description

Technical Field

[0001] This invention belongs to the fields of image processing, computer vision and pattern recognition, and specifically relates to a road scene semantic segmentation method and related apparatus with unsupervised traffic element alignment. Background Technology

[0002] In recent years, autonomous driving technology has made continuous breakthroughs, with computer vision research playing a crucial role. Semantic segmentation is an intensive pixel-level classification task that requires predicting the class label for all pixels. Compared to classification or detection tasks, it is more challenging. As a pixel-level classification task, manual annotation is very costly. Therefore, domain adaptation techniques can transfer models trained on synthetic datasets to real-world traffic scenarios.

[0003] Common unsupervised domain-adaptive semantic segmentation methods include domain adaptation based on generative adversarial networks and domain adaptation based on pseudo-label training. Generative adversarial network (GAN)-based methods, such as the one proposed by Tsai YH et al. (Tsai YH, Hung WC, Schulter S, et al. Learning to Adapt Structured OutputSpace for Semantic Segmentation[C]. Proceeding of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018:7472-7481.), use independent class-wise adversarial learning strategies instead of a global adversarial learning strategy. However, due to the inherent difficulty in training adversarial learning, this can lead to unstable model training. Pseudo-label-based training methods, such as ProCA (refer to Jiang Z, Li Y, Yang C, et al. Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation[C]. Proceeding of the European Conference on Computer Vision(ECCV), 2022:36-54.), add adaptive schemes such as inter-class alignment and prototype comparison, and use class-aware thresholds to update pseudo-labels online. However, this method does not distinguish between background classes and instance classes, and using the same alignment strategy is not a suitable choice, resulting in insufficient accuracy in road scene segmentation. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention aims to provide a road scene semantic segmentation method and related apparatus with unsupervised traffic element alignment.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A road scene semantic segmentation method with unsupervised traffic element alignment includes the following steps: Obtain the traffic scene image to be segmented; The traffic scene image to be segmented is input into a deep learning model to obtain the semantic segmentation result of the road scene; The deep learning model is obtained by training a convolutional neural network using a total loss function; the total loss function is a combination of the first cross-entropy loss function, the second cross-entropy loss function, the dynamic traffic element alignment loss, and the static traffic element alignment loss.

[0006] Furthermore, the first cross-entropy loss function is: (1) in, and These are the source domain images. Category of corresponding pixels Labels, and the predicted probability; These represent the height and width of the image, respectively. Indicates the total number of categories. The category of the pixel.

[0007] Furthermore, the second cross-entropy loss function is: (2) in, and Representing the target domain image respectively The pseudo-labels and predicted probabilities corresponding to the middle pixels.

[0008] Furthermore, the loss of alignment of dynamic traffic elements for: (3) in, This is the similarity matrix after threshold filtering. It is a set of dynamic traffic element categories. It is the first category in the dynamic traffic element category. It is the second category in the dynamic traffic element category. This is the nth category in the dynamic traffic element categories. As dynamic traffic elements, This represents the total number of categories of dynamic traffic elements.

[0009] Furthermore, the alignment loss of static traffic elements for: (4) in It is the cosine similarity metric formula. It is a set of static traffic element categories. For the category of pixels, For category indexing, For category Corresponding to the global prototype of the source domain. For the target domain prototype, For category Corresponding source domain prototype.

[0010] Furthermore, the total loss function is: (5) in, and These are the weights of the second cross-entropy loss function and the weights of the alignment loss of dynamic and static traffic elements, respectively.

[0011] Furthermore, the deep learning model is obtained by training a convolutional neural network using a total loss function, including the following steps: training the convolutional neural network on the training set using the total loss function, employing backpropagation and stochastic gradient descent, and continuously updating and iterating; adding pseudo-labels to the training process until the total loss function value of the forward propagation tends to converge, at which point training is stopped, and the deep learning model is obtained.

[0012] Furthermore, pseudo-tags are obtained through the following process: The Cityscapes dataset, representing real-world scenarios, was used as the target domain data. The GTA5 game dataset and the SYTHNIA synthetic dataset were used as source domain data. High-precision segmentation models are obtained by using source domain data for supervised training. The high-precision segmentation model parameters are shared with the target domain model, and the target domain model parameters are initialized. The target domain data is used as input to the target domain model to obtain the prediction results; The prediction results are filtered, and those prediction results that are greater than a specified category threshold are used as pseudo-labels.

[0013] A road scene semantic segmentation system with unsupervised traffic element alignment includes: The image acquisition module is used to acquire traffic scene images to be segmented; The calculation module is used to input the traffic scene image to be segmented into a deep learning model to obtain the semantic segmentation result of the road scene; The deep learning model is obtained by training a convolutional neural network using a total loss function; the total loss function is a combination of the first cross-entropy loss function, the second cross-entropy loss function, the dynamic traffic element alignment loss, and the static traffic element alignment loss.

[0014] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the road scene semantic segmentation method for unsupervised traffic element alignment.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention innovatively aligns dynamic and static traffic elements and determines the total loss function. It divides the entire image into dynamic and static traffic elements based on the characteristics of the elements and designs a dynamic-static element alignment loss function. This significantly improves the accuracy of unsupervised semantic segmentation methods without requiring annotation. The deep learning model of this invention is easy to train. Although it involves many processes, the training concept is simple. The generated pseudo-labels are automatically added to the next round of training to improve segmentation ability, thereby achieving accurate recognition of road scenes. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the entire model training framework; Figure 2 A flowchart illustrating the alignment of dynamic and static traffic elements; Figure 3 A flowchart illustrating the alignment of dynamic traffic elements; Figure 4 A flowchart illustrating the alignment of static traffic elements; Figure 5 In GTA5 Visualization results of semantic segmentation in the Cityscapes task, where (a) is the original image, (b) is the ground truth label, (c) is the source domain training only, (d) is the CPSL method and (e) is the experimental results of this invention; Figure 6 This is a schematic diagram of the road scene semantic segmentation system for unsupervised traffic element alignment according to the present invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] See Figure 1 The present invention provides a road scene semantic segmentation method with unsupervised traffic element alignment, comprising the following steps: Step 1: Use the real-world Cityscapes dataset as the target domain data (no manual annotation required); The GTA5 game dataset and the SYTHNIA synthetic dataset were used as source domain data. The tasks are divided into two domain adaptation tasks: GTA5 game dataset to Cityscapes and SYNTHIA synthetic dataset to Cityscapes.

[0019] This step utilizes the game dataset GTA5 and the synthetic dataset SYNTHIA, which avoids the problem of high cost of annotating real-world data.

[0020] Step 2: Warm-up Phase Model Training: This stage utilizes source domain data for supervised training to obtain a high-precision segmentation model; the specific process is as follows: To further improve the performance and robustness of the model, three image enhancement techniques—Gaussian blur, color dithering, and horizontal mirroring—were used to augment the source domain dataset. Then, supervised learning was performed using data from the augmented source domain dataset, and the first cross-entropy loss function was used for optimization to obtain a high-precision segmentation model.

[0021] The first cross-entropy loss function is: (1) in, and These are the source domain images. The category label (c) of the corresponding pixel, and the predicted probability; These represent the height and width of the image, respectively. Indicates the total number of categories. The category of the pixel.

[0022] Step 3: Share the high-precision segmentation model parameters to the target domain model and initialize the target domain model parameters.

[0023] The target domain data is used as input to the target domain model to obtain the prediction results; The pseudo-label generation strategy is designed, which adopts a fixed threshold filtering method. By setting a category threshold, the prediction results are filtered, and the prediction results that are greater than the specified category threshold are used as pseudo-labels. Pseudo-labels are used as target domain labels, and the second cross-entropy loss function is used for semi-supervised training. The second cross-entropy loss function is: (2) in, and Representing the target domain image respectively The pseudo-labels and predicted probabilities corresponding to the middle pixels.

[0024] The pseudo-label is calculated using the following formula. (3) in, It is the predicted category. It is the probability of the corresponding category. It's pixel-by-pixel multiplication. It is a pixel-by-pixel indicator function; it only applies when the predicted class probability is greater than a threshold. The value is 1 in the specified case and 0 in other cases. It is a pseudo-label. For the category of pixels, It is a set of categories for pixels.

[0025] In other words, among all pixels, the threshold Only probability Greater than the threshold The predicted category corresponding to the pixel It will be added to pseudo tags In the middle, the remaining pixels will be ignored.

[0026] Step 4: Design the alignment of dynamic and static traffic elements. Based on the differences in visual appearance of different traffic element types, select different alignment strategies to perform targeted cross-domain alignment by category.

[0027] The entire alignment of dynamic and static traffic elements can be divided into three steps: semantic matching, traffic element feature separation, and traffic element alignment.

[0028] See Figure 2 Semantic matching is performed using the feature maps obtained from the encoder in the network structure. Corrected labels obtained after downsampling with pseudo-labels Generate semantic feature maps of the source and target domains. After obtaining the correction tags for the source and target domains... Then, the semantic feature map of the source domain can be defined as , For the feature map of the source domain, As the corrected label for the source domain, the semantic feature map of the target domain can be defined as follows: , For the feature map of the target domain, The corrected label is used to define the semantic category of each pixel in the semantic feature map.

[0029] Traffic element feature separation: Extract all semantic feature vectors corresponding to each semantic category on the semantic feature maps of the source and target domains, and divide them into dynamic semantic feature vectors and static semantic feature vectors.

[0030] When separating traffic elements, instance classes such as vehicles and pedestrians, which have significant individual differences, are generally classified as dynamic traffic elements, while background classes such as roads, sky, and trees, which have less visual difference, are classified as static traffic elements. Alignment includes both dynamic and static traffic element alignment.

[0031] See Figure 3 The specific process of aligning the dynamic traffic elements is as follows: The source domain feature vectors are arranged into a queue, and the queue is transposed to obtain the source domain queue feature vectors. First, the dynamic semantic feature vectors are flattened and normalized to obtain the dynamic source domain feature vectors and the dynamic target domain feature vectors. Next, the dynamic source domain feature vectors are formed into a queue, and the queue is transposed to obtain the dynamic source domain queue feature vectors. The cosine similarity matrix between the dynamic target domain feature vectors and the dynamic source domain queue feature vectors is calculated. Secondly, a threshold is applied to the cosine similarity matrix, and cosine similarity matrices greater than the threshold are retained. Finally, the dynamic traffic element alignment loss is defined using a cosine similarity maximization alignment strategy. The dynamic traffic element alignment loss is defined as follows: (4) in, This is the similarity matrix after threshold filtering. It is a set of dynamic traffic element categories. It is the first category in the dynamic traffic element category. It is the second category in the dynamic traffic element category. This is the nth category in the dynamic traffic element categories. As dynamic traffic elements, This represents the total number of categories of dynamic traffic elements.

[0032] The target domain model is trained using dynamic traffic element alignment loss to achieve dynamic traffic element alignment.

[0033] See Figure 4 The specific process for aligning the static traffic elements is as follows: First, the static semantic feature vectors need to be flattened to obtain the static source domain feature vectors and the static target domain feature vectors. Based on the static source domain feature vector and the static target domain feature vector, the global semantic features of the source domain and the target domain are calculated. Here, they are defined as prototypes. The calculation method is to add the features of each class and divide by the number of corresponding classes, which is the average value of each class of features, thus obtaining the global semantic features.

[0034] To preserve more historical information in the source domain prototype, this invention also maintains a global prototype for the source domain. The calculation method is to use the exponential moving average method to calculate the global prototype of the source domain. Update.

[0035] The prototype of the target domain is obtained by summing the features of the target domain and dividing by the number of categories in the target domain. .

[0036] The next step is to prototype the target domain. and source domain global prototype Inspired by contrastive learning, alignment is achieved by bringing global prototypes of the target and source domains of the same category closer together while distancing them as much as possible from the global prototypes of the source domains of different categories. Therefore, the static traffic element alignment loss function is defined as: (5) in It is the cosine similarity metric formula. This is a set of static traffic element categories; For category indexing, For category Corresponding to the global prototype of the source domain. For the target domain prototype, For category Corresponding source domain prototype.

[0037] The target domain model is trained using static traffic element alignment loss to achieve dynamic traffic element alignment.

[0038] Step 5: Combine the first cross-entropy loss function in Step 2, the second cross-entropy loss function in Step 3, and the dynamic traffic element alignment loss and static traffic element alignment loss in Step 4 to obtain the total loss function; The total loss function is: (6) in, and These are the weights of the second cross-entropy loss function and the weights of the dynamic and static traffic element alignment loss function, respectively.

[0039] The convolutional neural network is trained on the training set using the total loss function, employing backpropagation and stochastic gradient descent to continuously update and iterate. Higher-quality pseudo-labels are generated and added to the training process until the forward propagation loss function value tends to converge, at which point training stops, resulting in a deep learning model.

[0040] The training parameters and details are set as follows: DeepLab v2 based on ResNet-101 is used as the backbone network, and the learning rate is [value missing]. The weight decays to The SGD optimizer has a momentum of 0.9, a batch size of 1, and performs 5000 iterations per round for a total of 200K iterations. To prevent overfitting, an early stopping strategy is implemented after 100K iterations. The hyperparameters are the weights of the second cross-entropy loss function. Weights of the loss function aligned with dynamic and static elements Set them to 1 and 0.1 respectively. Step 6: Use a deep learning model to extract images from the test set, classify each image pixel by pixel, and use a color palette to distinguish between different classes to obtain the segmented images of the test set and the segmentation results of the entire test set.

[0041] This invention first collects a dataset of real traffic road scenes, defining it as the target domain data; simultaneously, it uses a computer-generated dataset as the source domain data. Secondly, the entire training phase is divided into a warm-up phase and a self-training phase. In the warm-up phase, supervised training is performed using labeled source domain data to obtain a high-precision warm-up model. In the self-training phase, the target domain data is used as input to the warm-up model, and the prediction results are filtered out using a set threshold to select high-confidence predictions as corresponding pseudo-labels. Then, semantic matching is performed, using feature maps and corrected labels to generate semantic feature maps for the corresponding domain. The obtained feature maps are then used to separate traffic elements, and based on the semantic information in the semantic feature maps, the corresponding dynamic / static traffic element feature maps are separated. For the characteristics of dynamic and static traffic elements, alignment processes and loss functions are designed separately to improve the model's segmentation performance.

[0042] In this invention, a deep learning model is used to achieve semantic segmentation of traffic scene images to be segmented.

[0043] See Figure 6 The present invention also provides a road scene semantic segmentation system with unsupervised traffic element alignment, comprising: The image acquisition module is used to acquire traffic scene images to be segmented; The calculation module is used to input the traffic scene image to be segmented into a deep learning model to obtain the semantic segmentation result of the road scene; The deep learning model is obtained by training a convolutional neural network using a total loss function; the total loss function is a combination of the first cross-entropy loss function, the second cross-entropy loss function, the dynamic traffic element alignment loss, and the static traffic element alignment loss.

[0044] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the road scene semantic segmentation method for unsupervised traffic element alignment.

[0045] The following is a specific embodiment.

[0046] The neural network of this invention's algorithm is implemented using the PyTorch framework, and the workstation used for training is equipped with an RTX 3090 GPU for computational acceleration. Experiments use DeepLab v2 based on ResNet-101 as the backbone network, with a learning rate of... The weight decays to The SGD optimizer with momentum of 0.9 is used, the batch size is set to 1, each round has 5000 iterations, and a total of 200K iterations are performed. To prevent overfitting, an early stopping strategy is executed when the iteration reaches 100K.

[0047] The experiment included two tasks, namely GTA5. Cityscapes Domain Adaptation Experiment and SYNTHIA Cityscapes Domain Adaptation Experiment. The GTA5 dataset is a dataset collected from the game Grand Theft Auto, which includes 24,966 finely annotated synthetic images, with a size of [missing information]. The Cityscapes dataset is a collection of real-world daytime street scene images from 50 European cities, including 5000 fully annotated images, with a size of [missing information - likely a unit of measurement]. The dataset consists of 2975 training images, 500 validation images, and 1525 test images. The SYNTHIA dataset contains 9400 photorealistic image frames rendered from a virtual city, with fine pixel-level annotations. The image size is [missing information - likely a unit of measurement]. The algorithm of this invention is compared with four neural network models: AdaptSegNet, SIM, ProDA, and CPSL. The quantitative evaluation standard is the mean intersection-union ratio (mIoU). Experimental results are shown in Tables 1 and 2. Figure 5 As shown in (a)-(e): Table 1 GTA5 Quantitative assessment of the Cityscapes task

[0048] Table 2 SYNTHIA Quantitative assessment of the Cityscapes task

[0049] As shown in Table 1, in the domain adaptation task from GTA5 to Cityscapes, the method of this invention significantly outperforms the more advanced domain adaptation methods CPSL and ProDA of the past two years in terms of the evaluation index mIoU, verifying the effectiveness of the method of this invention. At the same time, compared with the similar method SIM, the method of this invention outperforms SIM in the evaluation index mIoU in the vast majority of categories, further verifying the effectiveness of the dynamic and static traffic element alignment proposed in this invention.

[0050] As shown in Table 2, for the domain adaptation experiment from SYNTHIA to Cityscapes, the cross-domain adaptation of the Cityscapes dataset on the SYNTHIA dataset is more challenging because the image realism of the SYNTHIA dataset is inferior to that of the GTA5 dataset. However, the method of this invention still performs well. Compared with the SIM method, the evaluation metric mIoU of this invention is improved by 5.5, leading in multiple categories, further verifying that the method of this invention also outperforms the SIM method on this task.

[0051] Figure 5 As can be seen from the figures, the model trained using only source domain data in column (c) performs poorly in the target domain, especially in road areas. Column (d) shows the experimental visualization results of the CPSL method, demonstrating a significant improvement in segmentation performance compared to the source domain-only method in column (c). This further verifies the effectiveness of the category-by-category alignment method in complex traffic scenarios. However, compared to the method proposed in this invention, for the parts marked in the dashed box, in rows 3, 4, 7, and 8 of column (d), the CPSL method results are blurry. In rows 5 and 6 of column (e), it can be seen that the traffic signs predicted by the method of this invention are more complete, better verifying the effectiveness of the dynamic and static traffic element alignment method proposed in this invention.

[0052] The above description is only of the preferred embodiment of the present invention and should not be construed as limiting the scope of the claims. The present invention is not limited to the above embodiments, and variations in its specific structure are permitted. All variations made within the scope of the independent claims of the present invention are also within the scope of protection of the present invention.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

Claims

1. A road scene semantic segmentation method with unsupervised traffic element alignment, characterized in that, Includes the following steps: Obtain the traffic scene image to be segmented; The traffic scene image to be segmented is input into a deep learning model to calculate the semantic segmentation result of the road scene; The deep learning model is obtained by training a convolutional neural network using a total loss function; the total loss function is a combination of the first cross-entropy loss function, the second cross-entropy loss function, the dynamic traffic element alignment loss, and the static traffic element alignment loss. The design aligns dynamic and static traffic elements, and selects different alignment strategies to perform targeted cross-domain alignment by category based on the differences in visual appearance of different traffic element types. The alignment of dynamic and static traffic elements consists of three steps: semantic matching, traffic element feature separation, and traffic element alignment. Semantic matching is performed by utilizing the feature maps obtained from the encoder in the network structure. Corrected labels obtained after downsampling with pseudo-labels Generate semantic feature maps of the source and target domains. ; After obtaining the correction tags for the source and target domains Then, the semantic feature map of the source domain is defined as , For the feature map of the source domain, The source domain's corrected label is defined as the semantic feature map of the target domain. , For the feature map of the target domain, Correction labels for the target domain; In the semantic feature map, the semantic category corresponding to each pixel on the feature map is determined by correcting the label; Traffic element feature separation: Extract all semantic feature vectors corresponding to each semantic category on the semantic feature maps of the source and target domains, and divide them into dynamic semantic feature vectors and static semantic feature vectors; Alignment includes dynamic traffic element alignment and static traffic element alignment; The specific process of aligning dynamic traffic elements is as follows: The source domain feature vectors are arranged into a queue, and the queue is transposed to obtain the source domain queue feature vectors. The dynamic semantic feature vectors are flattened and normalized to obtain dynamic source domain feature vectors and dynamic target domain feature vectors. Next, the dynamic source domain feature vectors are formed into a queue, and the queue is transposed to obtain the dynamic source domain queue feature vectors. The cosine similarity matrix between the dynamic target domain feature vectors and the dynamic source domain queue feature vectors is calculated. The cosine similarity matrix is ​​filtered by a threshold, and cosine similarity matrices greater than the threshold are retained; The dynamic traffic element alignment loss is defined using a cosine similarity maximization alignment strategy, as follows: （4） in, This is the similarity matrix after threshold filtering. It is a set of dynamic traffic element categories. It is the first category in the dynamic traffic element category. It is the second category in the dynamic traffic element category. The first in the category of dynamic traffic elements Categories As dynamic traffic elements, This represents the total number of categories of dynamic traffic elements. The target domain model is trained using dynamic traffic element alignment loss to achieve dynamic traffic element alignment. The specific process for aligning static traffic elements is as follows: The static semantic feature vectors are flattened to obtain the static source domain feature vector and the static target domain feature vector. Based on the static source domain feature vector and the static target domain feature vector, calculate the global semantic features of the source domain and the target domain, define them as prototypes, and calculate them by adding the features of each class and dividing by the number of the corresponding classes to obtain the global semantic features. Using the exponential moving average method, the global prototype of the source domain is analyzed. Update; The prototype of the target domain is obtained by summing the features of the target domain and dividing by the number of categories in the target domain. ; The static traffic element alignment loss function is defined as follows: （5） in It is the cosine similarity metric formula. This is a set of static traffic element categories; For category indexing, For category Corresponding to the global prototype of the source domain. For the target domain prototype, For category Corresponding source domain prototype; The static traffic element alignment loss is used to train the target domain model to achieve static traffic element alignment. The first cross-entropy loss function is: （1） in, and These are the source domain images. Category of corresponding pixels Labels, and the predicted probability; These represent the height and width of the image, respectively. Indicates the total number of categories. The category of the pixel; The second cross-entropy loss function is: （2） in, and Representing the target domain image respectively The pseudo-labels and predicted probabilities corresponding to the middle pixels.

2. The road scene semantic segmentation method with unsupervised traffic element alignment according to claim 1, characterized in that, The total loss function is: （5） in, and These are the weights of the second cross-entropy loss function and the weights of the alignment loss of dynamic and static traffic elements, respectively.

3. The road scene semantic segmentation method with unsupervised traffic element alignment according to claim 1, characterized in that, The deep learning model is obtained by training a convolutional neural network using a total loss function, including the following steps: training the convolutional neural network on the training set using the total loss function, employing backpropagation and stochastic gradient descent, and continuously updating and iterating; adding pseudo-labels to the training process until the total loss function value of forward propagation tends to converge, at which point training is stopped, and the deep learning model is obtained.

4. The road scene semantic segmentation method with unsupervised traffic element alignment according to claim 1, characterized in that, Pseudo-tags are obtained through the following process: The Cityscapes dataset, representing real-world scenarios, was used as the target domain data. The GTA5 game dataset and the SYTHNIA synthetic dataset were used as source domain data. High-precision segmentation models are obtained by using source domain data for supervised training. The high-precision segmentation model parameters are shared with the target domain model, and the target domain model parameters are initialized. The target domain data is used as input to the target domain model to obtain the prediction results; The prediction results are filtered, and those prediction results that are greater than a specified category threshold are used as pseudo-labels.

5. A road scene semantic segmentation system with unsupervised traffic element alignment, characterized in that, include: The image acquisition module is used to acquire traffic scene images to be segmented; The calculation module is used to input the traffic scene image to be segmented into a deep learning model to obtain the semantic segmentation result of the road scene; The deep learning model is obtained by training a convolutional neural network using a total loss function; the total loss function is a combination of the first cross-entropy loss function, the second cross-entropy loss function, the dynamic traffic element alignment loss, and the static traffic element alignment loss. The design aligns dynamic and static traffic elements, and selects different alignment strategies to perform targeted cross-domain alignment by category based on the differences in visual appearance of different traffic element types. The alignment of dynamic and static traffic elements consists of three steps: semantic matching, traffic element feature separation, and traffic element alignment. Semantic matching is performed by utilizing the feature maps obtained from the encoder in the network structure. Corrected labels obtained after downsampling with pseudo-labels Generate semantic feature maps of the source and target domains. ; After obtaining the correction tags for the source and target domains Then, the semantic feature map of the source domain is defined as , For the feature map of the source domain, The source domain's corrected label is defined as the semantic feature map of the target domain. , For the feature map of the target domain, Correction labels for the target domain; In the semantic feature map, the semantic category corresponding to each pixel on the feature map is determined by correcting the label; Traffic element feature separation: Extract all semantic feature vectors corresponding to each semantic category on the semantic feature maps of the source and target domains, and divide them into dynamic semantic feature vectors and static semantic feature vectors; Alignment includes dynamic traffic element alignment and static traffic element alignment; The specific process of aligning dynamic traffic elements is as follows: The source domain feature vectors are arranged into a queue, and the queue is transposed to obtain the source domain queue feature vectors. The dynamic semantic feature vectors are flattened and normalized to obtain dynamic source domain feature vectors and dynamic target domain feature vectors. Next, the dynamic source domain feature vectors are formed into a queue, and the queue is transposed to obtain the dynamic source domain queue feature vectors. The cosine similarity matrix between the dynamic target domain feature vectors and the dynamic source domain queue feature vectors is calculated. The cosine similarity matrix is ​​filtered by a threshold, and cosine similarity matrices greater than the threshold are retained; The dynamic traffic element alignment loss is defined using a cosine similarity maximization alignment strategy, as follows: （4） in, This is the similarity matrix after threshold filtering. It is a set of dynamic traffic element categories. It is the first category in the dynamic traffic element category. It is the second category in the dynamic traffic element category. The first in the category of dynamic traffic elements Categories As dynamic traffic elements, This represents the total number of categories of dynamic traffic elements. The target domain model is trained using dynamic traffic element alignment loss to achieve dynamic traffic element alignment. The specific process for aligning static traffic elements is as follows: The static semantic feature vectors are flattened to obtain the static source domain feature vector and the static target domain feature vector. Based on the static source domain feature vector and the static target domain feature vector, calculate the global semantic features of the source domain and the target domain, define them as prototypes, and calculate them by adding the features of each class and dividing by the number of the corresponding classes to obtain the global semantic features. Using the exponential moving average method, the global prototype of the source domain is analyzed. Update; The prototype of the target domain is obtained by summing the features of the target domain and dividing by the number of categories in the target domain. ; The static traffic element alignment loss function is defined as follows: （5） in It is the cosine similarity metric formula. This is a set of static traffic element categories; For category indexing, For category Corresponding to the global prototype of the source domain. For the target domain prototype, For category Corresponding source domain prototype; The static traffic element alignment loss is used to train the target domain model to achieve static traffic element alignment. The first cross-entropy loss function is: （1） in, and These are the source domain images. Category of corresponding pixels Labels, and the predicted probability; These represent the height and width of the image, respectively. Indicates the total number of categories. The category of the pixel; The second cross-entropy loss function is: （2） in, and Representing the target domain image respectively The pseudo-labels and predicted probabilities corresponding to the middle pixels.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the road scene semantic segmentation method for unsupervised traffic element alignment as described in any one of claims 1 to 4.

Images