Nighttime semantic segmentation method based on unsupervised learning and cross attention

Abstract

The application discloses a night semantic segmentation method based on unsupervised learning and cross attention, comprising the following steps: step 1: using an existing semantic segmentation model as a backbone network, based on daytime images containing semantic segmentation labels, daytime images not containing semantic segmentation labels and night images, calculating a mixed loss through a ClassMix strategy, processing and calculating a consistency regularization loss through consistency regularization, calculating a segmentation loss through a cross attention mechanism, and generating a static label and calculating a static loss; calculating a loss function according to the mixed loss, the consistency regularization loss, the segmentation loss and the static loss, training a night semantic segmentation model based on the loss function; step 2: inputting a night image to be processed into the trained night semantic segmentation model to obtain a semantic segmentation result output by the night semantic segmentation model. The application improves the accuracy of night semantic segmentation without a night semantic segmentation dataset.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a nighttime semantic segmentation method based on unsupervised learning and cross-attention. Background Technology

[0002] Semantic segmentation has become a key problem in computer vision and a crucial technology for scene understanding. As a core issue in computer vision, scene understanding is increasingly attracting attention because more and more real-world applications require inferring semantic meaning from images or image sequences. These applications include autonomous driving, image search engines, augmented reality, and human-computer interaction. In recent years, with the large-scale application of convolutional neural networks in semantic segmentation, the accuracy of semantic segmentation models in daytime scenes has significantly improved. However, since the vast majority of scenes in existing semantic segmentation datasets are daytime scenes, the trained models are only suitable for daytime scenes, and their accuracy drops considerably in nighttime scenes. Furthermore, the lack of large-scale nighttime semantic segmentation datasets makes it difficult to directly train semantic segmentation models for nighttime scenes. Therefore, adapting daytime scene segmentation models to nighttime scenes without directly using nighttime scene segmentation datasets becomes crucial. Summary of the Invention

[0003] The purpose of this invention is to provide a nighttime semantic segmentation method based on unsupervised learning and cross-attention, in order to improve the accuracy of nighttime semantic segmentation in the absence of a nighttime semantic segmentation dataset.

[0004] To solve the above technical problems, the technical solution of the present invention is: a nighttime semantic segmentation method based on unsupervised learning and cross-attention, comprising:

[0005] Step 1: Train the nighttime semantic segmentation model;

[0006] Step 101: Obtain sample images, including daytime images with semantic segmentation labels, daytime images without semantic segmentation labels, and nighttime images;

[0007] Step 102: Calculate the mixture loss based on the daytime image with semantic segmentation labels and the nighttime image without semantic segmentation labels using the ClassMix strategy;

[0008] Step 103: Perform consistency regularization on nighttime images without semantic segmentation labels and calculate the consistency regularization loss;

[0009] Step 104: Calculate the segmentation loss based on the daytime image with semantic segmentation labels and the semantic segmentation labels using a cross-attention mechanism;

[0010] Step 105: Generate static labels for daytime images without semantic segmentation labels and supervise nighttime images without semantic segmentation labels based on the static labels, and calculate static loss;

[0011] Step 106: Calculate the loss function based on the hybrid loss, consistency regularization loss, segmentation loss, and static loss, and train the nighttime semantic segmentation model based on the loss function;

[0012] Step 2: Obtain the nighttime image to be processed, input the nighttime image to be processed into the trained nighttime semantic segmentation model, and obtain the semantic segmentation result output by the nighttime semantic segmentation model.

[0013] Further, step 101 specifically includes:

[0014] Step 1011: Define a source domain S and two target domains T d and T n Among them, S and T d and T n Representing daytime, daytime, and nighttime respectively, only the source domain S has semantic segmentation labels during training; from the source domain S and the target domain T d T n Daytime image A and daytime image B are extracted respectively. d and nighttime image B n The daytime image A has a semantic segmentation label L. A Image B d B n It does not have semantic segmentation tags;

[0015] Step 1012: Enhance the illumination and segment the image:

[0016] Image A is processed by illumination enhancement network g to generate R. A =g(A),R A After segmentation network f θ Generate segmentation map S A =f θ (R A );

[0017] Image B d Generated after passing through the illumination enhancement network g Then input the segmentation network f θ Segmentation map generated later

[0018] Image B n Generated after passing through the illumination enhancement network g Then input the segmentation network f θ Segmentation map generated later

[0019] Furthermore, in step 102, the ClassMix strategy is based on the ClassMix module, and step 102 specifically includes:

[0020] The ClassMix module uses semantic segmentation tags L A All categories C A After removing easily separable categories, the remaining categories are: exist Randomly select half of the categories and set the value of the pixels in these categories to 1 in the binary mask M, while setting the value of all other pixels to 0, to generate the binary mask M;

[0021] Then, images A and B are blended using a mask. n To enhance image X A In the middle, enhance the image X A Includes pixels from A with a mask of 1 and B with a mask of 0. n The pixels; according to image B d Predicted semantic segmentation graph To correct Errors in the code lead to the generation of pseudo-tags.

[0022] Using a binary mask M, the semantic segmentation label L A and pseudo-tags The images are blended to produce an enhanced image X. A Hybrid pseudo-label Y A ;

[0023] The hybrid loss function of the ClassMix module is: L mix =l(f θ (X A ),Y A ); where X A and Y A These are the hybrid enhanced image and its hybrid pseudo-label, generated by the ClassMix module. The input image A is randomly sampled from the source domain S, while B... n This is from the target domain T n The result is obtained from random sampling; l represents the cross-entropy loss, defined as: Where W and H are the width and height of the image, S(i,j,c) and Y(i,j,c) are the probabilities that the pixel at coordinates i,j belongs to class c, S is the predicted semantic segmentation map, and Y is the label.

[0024] Furthermore, in step 103, the consistency regularization processing uses a consistency regularization module; step 103 specifically includes:

[0025] The consistency regularization module generates a strongly enhanced image for each unlabeled image, then generates the class probability distribution of the strongly enhanced image, and then uses it for KL divergence; wherein, the image is subjected to gray-level shift and contrast shift to obtain a weakly enhanced image; the image is subjected to gray-level shift and contrast shift, and random Gaussian noise is superimposed to obtain a strongly enhanced image;

[0026] To obtain the class probability distribution, given an unlabeled image B n Weakly enhanced image B nw Calculate the predicted class distribution of the model under the following conditions: P w =f θ (B nw ), in P w The gradient backpropagation is terminated at the point where Y is generated. w Then, according to B n Strongly enhanced image B ns Also calculate a predicted class distribution P s =f θ (B ns Finally, the KL divergence is used as the distance between the two class probability distributions to obtain the consistency regularization loss:

[0027] Furthermore, step 104 specifically includes:

[0028] Step 1041:

[0029] X n Nighttime image B n The output features X after passing through the feature extraction network n ∈R C×H×W ;

[0030] X d Nighttime image B n Corresponding daytime image B d The output features X after passing through the feature extraction network d ∈R C×H×W ;

[0031] X n After a 1×1 convolution, a feature map Q is generated;

[0032] X d After a 1×1 convolution, a feature map K is generated; Q, K∈R C×H×W ;

[0033] After transposing Q, perform matrix multiplication with K, and then use a softmax layer to compute the attention map S∈R. N×N :

[0034]

[0035] Where N = H × W is the number of pixels in the image, and s ji This represents the influence of the i-th position in the daytime image feature map on the j-th position in the nighttime image feature map. The more similar the feature representations of the two positions are, the greater the correlation between them.

[0036] Step 1042: Transfer feature X n Input a 1×1 convolutional layer to generate a new feature map V∈R C×H×W And change its shape to R C×N Multiply the transposed matrices of V and S; then multiply the product by a scaling parameter α to form an attention-weighted feature map, which is added to the original feature map X. n In this context, this process can be represented as: Initialize α to 0 and set α as a learnable weight to be adaptively adjusted during training;

[0037] Step 1043: Propose a weighting strategy for the predicted class likelihood graph: For each class k∈C, first define a weight: w K =1 / (a k ); where a k It is the proportion of all valid pixels labeled as category k in the source domain, a k The smaller the value of w K The larger the value, the better; weighted cross-entropy loss is used to train semantic image segmentation in the source domain, thereby calculating the segmentation loss L. seg :

[0038]

[0039] Where P s (k) The prediction P is from the source image. s The kth channel GT (k) It is the one-hot encoding of the label of the k-th category.

[0040] Furthermore, step 105 specifically includes:

[0041] Given a segmentation prediction P td ∈R H×W×C and P tn ∈R H×W×C This loss is calculated by considering only the channels corresponding to the static categories; C S If we express this as the total number of static object categories, then we have and

[0042] Static loss L static Defined as:

[0043] Where N is the total number of valid pixels in the segmentation pseudo-labels, γ is the parameter of focal loss, and p is the likelihood map of the correct class, calculated at each pixel i in the 3×3 local region of class c: Where 'o' is a pseudo tag. The one-hot encoding, j represents each position of the 3×3 region centered at i.

[0044] Furthermore, in step 6, the loss function is: L total =L seg +L static +β1L mix +β2L kl

[0045] Among them, L seg L represents the segmentation loss. static L represents the static loss. mix L represents the mixed loss. kl β1 and β2 represent the consistency regularization loss; β1 and β2 are hyperparameters.

[0046] The present invention has the following beneficial effects:

[0047] I. This invention utilizes the ClassMix module to fuse supervised daytime scenes and unsupervised nighttime scenes, making full use of the supervised information in daytime scenes and the unique texture information of nighttime scenes; it also enables the model to adapt to both daytime and nighttime scenes, enhancing the robustness of the semantic segmentation network; by training a nighttime semantic segmentation model, this invention can perform semantic segmentation on input nighttime images and output segmented images, improving the accuracy and quality of the nighttime semantic segmentation model.

[0048] Second, this invention proposes a cross-attention mechanism, which allows the nighttime semantic segmentation model to pay more attention to the parts of the nighttime scene that are similar to the daytime scene, so that the model can make full use of the high-level semantic information of the daytime scene when training in the nighttime scene.

[0049] Third, this invention introduces consistency regularization, which enables the segmentation model to adapt to complex and varied nighttime scene textures and lighting, while also resisting additional noise caused by low-light enhancement. Attached Figure Description

[0050] Figure 1 This is an overall flowchart of the method of the present invention;

[0051] Figure 2 This is a schematic diagram of the nighttime semantic segmentation model framework based on semi-supervised learning and cross-attention mechanism of the present invention.

[0052] Figure 3This is a schematic diagram of the ClassMix module in this embodiment;

[0053] Figure 4 This is a schematic diagram of consistency regularization in this embodiment;

[0054] Figure 5 This is a schematic diagram of the cross-attention module in this embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Please refer to Figures 1 to 5 This invention is a nighttime semantic segmentation method based on unsupervised learning and cross-attention, comprising:

[0057] Step 1: Train the nighttime semantic segmentation model;

[0058] Step 101: Obtain sample images, including daytime images with semantic segmentation labels, daytime images without semantic segmentation labels, and nighttime images;

[0059] Step 102: Calculate the mixture loss based on the daytime image with semantic segmentation labels and the nighttime image without semantic segmentation labels using the ClassMix strategy;

[0060] Step 103: Perform consistency regularization on nighttime images without semantic segmentation labels and calculate the consistency regularization loss;

[0061] Step 104: Calculate the segmentation loss based on the daytime image with semantic segmentation labels and the semantic segmentation labels using a cross-attention mechanism;

[0062] Step 105: Generate static labels for daytime images without semantic segmentation labels and supervise nighttime images without semantic segmentation labels based on the static labels, and calculate static loss;

[0063] Step 106: Calculate the loss function based on the hybrid loss, consistency regularization loss, segmentation loss, and static loss, and train the nighttime semantic segmentation model based on the loss function;

[0064] Step 2: Obtain the nighttime image to be processed, input the nighttime image to be processed into the trained nighttime semantic segmentation model, and obtain the semantic segmentation result output by the nighttime semantic segmentation model.

[0065] Step 101 specifically includes:

[0066] Step 1011: Define a source domain S and two target domains T d and T nAmong them, S and T d and T n These represent Cityscapes (daytime), Dark Zurich-D (daytime), and Dark Zurich-N (nighttime), respectively, with only the source domain S having semantic segmentation labels during training; from the source domain S and the target domain T... d T n Daytime image A and daytime image B are extracted respectively. d and nighttime image B n The daytime image A has a semantic segmentation label L. A Image B d B n It does not have semantic segmentation tags;

[0067] Step 1012: Enhance the illumination and segment the image: as follows Figure 2 As shown,

[0068] Image A is processed by illumination enhancement network g to generate R. A =g(A),R A After segmentation network f θ Generate segmentation map S A =f θ (R A );

[0069] Image B d Generated after passing through the illumination enhancement network g Then input the segmentation network f θ Segmentation map generated later

[0070] Image B n Generated after passing through the illumination enhancement network g Then input the segmentation network f θ Segmentation map generated later

[0071] See Figure 3 In step 102, the ClassMix strategy is based on the ClassMix module. Step 102 specifically includes:

[0072] The ClassMix module uses semantic segmentation tags L A All categories C A After removing easily segmentable categories such as sky, road, and forest, the remaining categories are: exist Randomly select half of the categories and set the value of the pixels in these categories to 1 in the binary mask M, while setting the value of all other pixels to 0, to generate the binary mask M;

[0073] Then use this mask to blend images A and B.n To enhance image X A In the middle, enhance the image X A Includes pixels from A with a mask of 1 and B with a mask of 0. n The pixels; due to B n Corresponding daytime scene image B d With B n Both images were taken in the same scene, and the positions of static objects (objects that cannot move freely, such as traffic lights, telephone poles, etc.) are relatively close in the images (due to the nature of the Dark Zurich dataset, the positions are not one-to-one). Based on image B... d Predicted semantic segmentation graph To correct Errors in the text are corrected to generate more accurate pseudo-tags. Using a binary mask M, the semantic segmentation label L A and pseudo-tags The images are blended to produce an enhanced image X. A Hybrid pseudo-label Y A ;

[0074] The hybrid loss function of the ClassMix module is: L mix =l(f θ (X A ),Y A ); where X A and Y A These are the hybrid enhanced image and its hybrid pseudo-label, generated by the ClassMix module. The input image A is randomly sampled from the source domain S, while B... n This is from the target domain T n The result is obtained from random sampling; l represents the cross-entropy loss, defined as: Where W and H are the width and height of the image, S(i,j,c) and Y(i,j,c) are the probabilities that the pixel at coordinates i,j belongs to class c, S is the predicted semantic segmentation map, and Y is the label.

[0075] See Figure 4 In step 103, the consistency regularization process uses the consistency regularization module; step 103 specifically includes:

[0076] The consistency regularization module generates a strongly enhanced image for each unlabeled image, then generates the class probability distribution of the strongly enhanced image, and then uses it for KL divergence; wherein, the image is subjected to gray-level shift and contrast shift to obtain a weakly enhanced image; the image is subjected to gray-level shift and contrast shift, and random Gaussian noise is superimposed to obtain a strongly enhanced image;

[0077] To obtain the class probability distribution, given an unlabeled image Bn Weakly enhanced image B nw Calculate the predicted class distribution of the model under the following conditions: P w =f θ (B nw ), in P w The gradient backpropagation is terminated at the point where Y is generated. w Then, according to B n Strongly enhanced image B ns Also calculate a predicted class distribution P s =f θ (B ns Finally, the KL divergence is used as the distance between the two class probability distributions to obtain the consistency regularization loss:

[0078] See Figure 5 Step 104 is implemented through a cross-attention mechanism, specifically including:

[0079] Step 1041:

[0080] X n Nighttime image B n The output features X after passing through the feature extraction network n ∈R C×H×W ;

[0081] X d Nighttime image B n Corresponding daytime image B d The output features X after passing through the feature extraction network d ∈R C×H×W ;

[0082] X n After a 1×1 convolution, a feature map Q is generated;

[0083] X d After a 1×1 convolution, a feature map K is generated; Q, K∈R C×H×W ;

[0084] After transposing Q, perform matrix multiplication with K, and then use a softmax layer to compute the attention map S∈R. N×N :

[0085]

[0086] Where N = H × W is the number of pixels in the image, and s ji This represents the influence of the i-th position in the daytime image feature map on the j-th position in the nighttime image feature map. The more similar the feature representations of the two positions are, the greater the correlation between them.

[0087] Step 1042: Transfer feature Xn Input a 1×1 convolutional layer to generate a new feature map V∈R C×H×W And change its shape to R C×N Multiply the transposed matrices of V and S; then multiply the product by a scaling parameter α to form an attention-weighted feature map, which is added to the original feature map X. n In this context, this process can be represented as: Initialize α to 0 and set α as a learnable weight to be adaptively adjusted during training;

[0088] Step 1043: Propose a weighting strategy for the predicted class likelihood graph. Specifically, for each class k∈C, first define a weight: w K =1 / (a k ); where a k It is the proportion of all valid pixels labeled as category k in the source domain, a k The smaller the value of w K The larger the value, the better; weighted cross-entropy loss is used to train semantic image segmentation in the source domain, thereby calculating the segmentation loss L. seg :

[0089]

[0090] Where P s (k) The prediction P is from the source image. s The kth channel GT (k) It is the one-hot encoding of the label of the k-th category.

[0091] Step 105 specifically includes:

[0092] If only static object categories are considered, then daytime images and their corresponding nighttime images have a rough similarity. Here, a static loss is introduced to provide pixel-level pseudo-supervision for static object categories (such as roads, sidewalks, walls, vegetation, and sky) in nighttime images. Given a segmentation prediction P... td ∈R H×W×C and P tn ∈R H×W×C This loss is calculated by considering only the channels corresponding to the static categories; C S If we express this as the total number of static object categories, then we have and

[0093] Static loss L static Defined as:

[0094] Where N is the total number of valid pixels in the segmented pseudo-labels, γ is the parameter of focal loss, and p is the likelihood map of the correct class. Unlike focal loss, this invention calculates p at each pixel i in the 3×3 local region of class c: Where 'o' is a pseudo tag. The one-hot encoding, j represents each position of the 3×3 region centered at i.

[0095] In step 6, the total loss function is: L total =L seg +L static +β1L mix +β2L kl ; among which, L seg L represents the segmentation loss. static L represents the static loss. mix L represents the mixed loss. kl β1 and β2 represent the consistency regularization loss; β1 and β2 are hyperparameters.

[0096] In this invention, image A is processed by illumination enhancement network g to generate R. A =g(A),R A After segmentation network f θ Generate segmentation map S A =f θ (R A According to the predicted segmentation map S A and label L A Calculate the segmentation loss L seg Image B d Generated after passing through the illumination enhancement network g After inputting into the segmentation network, a segmentation map is generated. Since there are no labels, the segmentation loss cannot be directly calculated. Image B n Generated after passing through the illumination enhancement network g After inputting into the segmentation network, a segmentation map is generated. Also without labels. Due to nighttime image B n Since there are no labels, this paper generates static labels based on daytime images for supervision, and the static loss function is L. static At the same time, R A L A , and The input is processed by the ClassMix module, mixed, and the mixing loss L is finally calculated. mix While the clarity of nighttime images improved after illumination enhancement, the noise was also amplified. To enhance the robustness of the segmentation model and minimize the impact of noise, image B... nInput the consistency regularization module and calculate the consistency regularization loss L. kl The invention calculates the total loss function based on the mixture loss, consistency regularization loss, segmentation loss, and static loss. It utilizes existing semantic segmentation models (such as U-Net, FCN, DeepLab v1 / v2 / v3 / v3) as the backbone network, trains the segmentation model using the loss function, and obtains a nighttime semantic segmentation model. This model can be directly used as input to nighttime images to obtain segmented images. This invention effectively improves the accuracy of nighttime semantic segmentation, achieving better results than existing algorithms, thus obtaining higher-quality segmentation results.

[0097] All parts not covered in this invention are the same as or implemented using existing technologies.

[0098] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A nighttime semantic segmentation method based on unsupervised learning and cross-attention, characterized by: include Step 1: Train the nighttime semantic segmentation model; Step 101: Obtain sample images, including daytime images with semantic segmentation labels, daytime images without semantic segmentation labels, and nighttime images; Step 102: Calculate the mixture loss based on the daytime image with semantic segmentation labels and the nighttime image without semantic segmentation labels using the ClassMix strategy; Step 103: Perform consistency regularization on nighttime images without semantic segmentation labels and calculate the consistency regularization loss; Step 104: Calculate the segmentation loss based on the daytime image with semantic segmentation labels and the semantic segmentation labels using a cross-attention mechanism; Step 105: Generate static labels for daytime images without semantic segmentation labels and supervise nighttime images without semantic segmentation labels based on the static labels, and calculate static loss; Step 106: Calculate the loss function based on the hybrid loss, consistency regularization loss, segmentation loss, and static loss, and train the nighttime semantic segmentation model based on the loss function; Step 2: Obtain the nighttime image to be processed, input the nighttime image to be processed into the trained nighttime semantic segmentation model, and obtain the semantic segmentation result output by the nighttime semantic segmentation model; In step 103, the consistency regularization process uses the consistency regularization module; Step 103 specifically includes: The consistency regularization module generates a strongly enhanced image for each unlabeled image, then generates the class probability distribution of the strongly enhanced image, and then uses it for KL divergence; wherein, the image is subjected to gray-level shift and contrast shift to obtain a weakly enhanced image; the image is subjected to gray-level shift and contrast shift, and random Gaussian noise is superimposed to obtain a strongly enhanced image; To obtain the class probability distribution, given an unlabeled image B n Weakly enhanced image Calculate the predicted class distribution of the model under the following conditions: ,exist Terminating gradient backpropagation at the point of origin, generating Then, according to B n Strongly enhanced image Also calculate a predicted class distribution Finally, the KL divergence is used as the distance between the two class probability distributions to obtain the consistency regularization loss: ; Step 104 specifically includes: Step 1041: Nighttime image B n The output features after passing through the feature extraction network ; Nighttime image B n Corresponding daytime image The output features after passing through the feature extraction network ; Feature maps are generated after 1 × 1 convolution. ; Feature maps are generated after 1 × 1 convolution. ; ; After transposing Q, perform matrix multiplication with K, and then use a softmax layer to compute the attention map. : ； in It is the number of pixels in the image, where This represents the influence of the i-th position in the feature map of the daytime image on the j-th position in the feature map of the nighttime image. The more similar the feature representations of the two positions are, the greater the correlation between them. Step 1042: Add features Input a 1 × 1 convolutional layer to generate new feature maps. and change its shape Multiply the transposed matrices of V and S; then multiply the product by a scaling parameter α to form an attention-weighted feature map, which is then added to the original feature map. In this context, the process is represented as: Initialize α to 0 and set α as a learnable weight to adaptively adjust during training; Step 1043: Propose a weighting strategy for the predicted class likelihood plot: for each class First, define a weight: ;in It is the proportion of all valid pixels labeled as category k in the source domain. The smaller the value, The larger the value, the better; weighted cross-entropy loss is used to train semantic image segmentation in the source domain, thereby calculating the segmentation loss. : ； in It is a prediction from the source image. The kth channel It is the one-hot encoding of the label of the k-th category.

2. The nighttime semantic segmentation method based on unsupervised learning and cross-attention as described in claim 1, characterized in that: Step 101 specifically includes: Step 1011: Define a source domain S and two target domains T. d and T n Among them, S and T d and T n Representing daytime, daytime, and nighttime respectively, only the source domain S has semantic segmentation labels during training; from the source domain S and the target domain T d T n Daytime image A and daytime image B are extracted respectively. d and nighttime image B n The daytime image A has a semantic segmentation label L. A Image B d B n It does not have semantic segmentation tags; Step 1012: Enhance the illumination and segment the image: Image A is generated after passing through the illumination enhancement network g. , After segmentation network Generate segmentation map ; Image B d After being processed by the illumination enhancement network g, the following is generated. Then input the segmentation network Segmentation map generated later ; image After being processed by the illumination enhancement network g, the following is generated. Then input the segmentation network Segmentation map generated later .

3. The nighttime semantic segmentation method based on unsupervised learning and cross-attention as described in claim 1, characterized in that: In step 102, the ClassMix strategy is based on the ClassMix module. Step 102 specifically includes: The ClassMix module uses semantic segmentation tags L A All categories After removing easily separable categories, the remaining categories are: ,exist Randomly select half of the categories and set the value of the pixels in these categories to 1 in the binary mask M, while setting the value of all other pixels to 0, to generate the binary mask M; Then use a mask to blend images A and B. n To enhance images In the middle, enhance the image Includes pixels from A with a mask of 1 and B with a mask of 0. n The pixels; according to image B d Predicted semantic segmentation graph To correct Errors in the code lead to the generation of pseudo-tags. ; Using a binary mask M, the semantic segmentation label L A and pseudo-tags Blending is performed to produce an enhanced image. Hybrid pseudo-tags ; The hybrid loss function of the ClassMix module is: ;in and These are the hybrid enhanced image and its hybrid pseudo-label, generated by the ClassMix module. The input image A is randomly sampled from the source domain S, while B... n This is from the target domain T n The value is obtained from random sampling; ℓ represents the cross-entropy loss, defined as: Where W and H are the width and height of the image, S(i, j, c) and Y(i, j, c) are the probabilities that the pixel at coordinates i, j belongs to class c, S is the predicted semantic segmentation map, and Y is the label.

4. The nighttime semantic segmentation method based on unsupervised learning and cross-attention as described in claim 1, characterized in that: Step 105 specifically includes: Given segmentation prediction and This loss is calculated by considering only the channels corresponding to the static categories; If we express this as the total number of static object categories, then we have and ; static loss Defined as: ; Where N is the total number of valid pixels in the segmentation pseudo-labels, γ is the parameter of the focal loss, and p is the likelihood map of the correct class, calculated at each pixel i in the 3 × 3 local region of class c: ; where o is a pseudo tag The one-hot encoding, j represents each position of a 3 × 3 region centered at i.

5. The nighttime semantic segmentation method based on unsupervised learning and cross-attention according to claim 1, characterized in that: In step 106, the loss function is: ; in, Indicates the loss from partitioning. Represents static loss. Indicates mixed loss, This represents the consistency regularization loss; and This is a hyperparameter.

Images