Night semantic segmentation method based on low illumination enhancement and edge optimization

Abstract

The invention relates to the technical field of semantic segmentation, in particular to a night semantic segmentation method based on low illumination enhancement and edge optimization, and the method comprises the steps: inputting an image into a low-light enhancement repair network based on the Retinex theory, local contrast enhancement and adaptive feature fusion, obtaining a denoised and enhanced intermediate image, and carrying out the edge optimization of the intermediate image; inputting the intermediate image into a semantic segmentation network to obtain a category distribution diagram of each pixel; inputting a discriminator embedded with a channel attention module according to the category distribution diagram of each pixel, and optimizing a generator composed of a low light enhancement repair network and a semantic segmentation network through a multi-task joint optimization loss function; and inputting a night image to be detected and segmented into the optimized generator, and outputting a segmentation result. By adopting the method, the low-light enhancement repair network is combined with a local contrast enhancement and channel feature fusion mechanism, so that the overall brightness of the image is improved, the details and edge structures of the image are reserved, and the perception capability and robustness of the model are improved.

Description

A Nighttime Semantic Segmentation Method Based on Low-Light Enhancement and Edge Optimization Technical Field

[0001] This invention relates to the field of semantic segmentation technology, and in particular to a nighttime semantic segmentation method based on low-light enhancement and edge optimization. Background Technology

[0002] Semantic segmentation, a crucial task in computer vision, aims to accurately label the category of each pixel in an image and is widely applied in fields such as autonomous driving, industrial vision, remote sensing, robot navigation, and virtual reality. While deep learning has achieved significant results in daytime semantic segmentation, model performance degrades considerably at night and in adverse weather conditions due to insufficient lighting and visual degradation. This poses a major challenge to applications like autonomous driving that require all-weather operation, as these systems demand high robustness in various complex and changing environments. However, most current semantic segmentation models are trained on daytime datasets, making it difficult to adapt to the unique challenges of nighttime driving environments, such as varying lighting, exposure imbalances, edge degradation, and motion blur. These issues cause uneven brightness variations in the appearance and texture of objects, increasing the difficulty of boundary recognition and thus limiting the model's performance in nighttime scenes.

[0003] Existing nighttime datasets provide a certain number of labeled nighttime images, but they are still insufficient in terms of quantity and diversity to fully cover the complexity and diversity of nighttime scenes. In order to overcome the technical problems of nighttime data, existing technologies use techniques such as data transformation and data processing of daytime data to obtain more data, which increases the computational cost. Existing image processing methods are effective in improving brightness and contrast, but they have the following problems: (1) They lack protection for edge details and are prone to introducing blur or over-smoothing.

[0004] (2) The enhancement process may result in color distortion or structural distortion.

[0005] (3) The enhanced image is only used as preprocessing and is not deeply coupled with the segmentation task, lacking end-to-end optimization. Summary of the Invention

[0006] The purpose of this invention is to provide a nighttime semantic segmentation method based on low-light enhancement and edge optimization to solve the above-mentioned technical problems.

[0007] To achieve the above objectives, this invention provides a nighttime semantic segmentation method based on low-light enhancement and edge optimization, with the following specific steps: Step S1: Prepare a training dataset and input the images in the training dataset into a low-light enhancement and inpainting network based on Retinex theory, local contrast enhancement, and adaptive feature fusion to obtain intermediate images after noise reduction and enhancement processing; Step S2: Input the intermediate images into a semantic segmentation network, which outputs a category distribution map for each pixel; Step S3: Input the category distribution map for each pixel into a discriminator embedded with a channel attention module, and perform end-to-end optimization of the generator composed of the low-light enhancement and inpainting network and the semantic segmentation network by jointly optimizing the loss function through multiple tasks including image reconstruction loss, semantic segmentation loss, static loss, and adversarial loss; Step S4: Input the nighttime image to be segmented into the optimized generator and output the segmentation result.

[0008] Preferably, in step S1, a source domain image and a target domain image are obtained, which correspond to daytime images and nighttime images, respectively. The obtained images are then normalized after being standardized to obtain a training dataset.

[0009] Preferably, the low-light enhancement and restoration network includes a Retinex-based illumination estimation module, a local contrast enhancement module, and an adaptive feature fusion module arranged sequentially.

[0010] Preferably, the Retinex-based illumination estimation module includes a first convolutional layer and an illumination map generator connected in sequence, and the images in the training dataset generate feature maps by passing through the first convolutional layer and the illumination map generator.

[0011] Preferably, the feature map enters the local contrast enhancement module, which includes two second convolutional layers, so that the context mask is calculated while the spatial dimension remains unchanged. The context mask is normalized by Softmax and applied to the original input through element-wise multiplication. The results are summed in the spatial dimension to generate the final context representation.

[0012] Preferably, the output of the local contrast enhancement module and the feature map are fed into the adaptive feature fusion module through downsampling and upsampling. Global context information is extracted through adaptive average pooling, and the number of channels of the feature map is compressed by dimensionality reduction convolution to generate feature representation. The fully connected layer remaps the features back to the original number of channels to generate attention map. The attention map is normalized by the Softmax layer. During the fusion process, adaptive weighting is performed according to the importance of features at each scale. The output of the adaptive feature fusion module is input into the third convolutional layer. The output of the third convolutional layer is added to the output of the Retinex-based illumination estimation module and then input into the fourth convolutional layer. At the same time, the images in the training dataset are input into the fourth convolutional layer to output intermediate images.

[0013] Preferably, in step S2, the semantic segmentation network adopts an improved semantic segmentation model, which includes a ResNet-101 backbone network. A polarization self-attention module is embedded in the residual module of the ResNet-101 backbone network. The convolutional features are dynamically calibrated in two dimensions, channel and space, before the residual connection. The key regions are focused through the weight matrix, and the global structural information transmitted by the original skip connections is preserved.

[0014] Preferably, the polarization self-attention module includes a channel attention submodule and a spatial attention submodule set in parallel. The channel attention submodule and the spatial attention submodule calculate the attention weights respectively, and finally the results of the two are fused to achieve dynamic feature calibration.

[0015] Preferably, the discriminator of the channel attention module includes a fully convolutional network with a multi-layer convolutional structure, and the channel attention module is embedded in the fully convolutional network.

[0016] Preferably, the multi-task joint optimization loss function includes image reconstruction loss, semantic segmentation loss, static loss, and adversarial loss; the image reconstruction loss includes basic pixel loss. Chabonnier's loss And edge loss; the edge loss is as follows: ;in, For edge loss, For the original image The original image edge feature map obtained based on the Laplacian operator To generate an edge feature map of the generated image based on the Laplacian operator, The Euclidean norm is used to calculate the pixel-level differences between the edge feature maps of the original and generated images. It is a constant.

[0017] Therefore, the present invention adopts the above-mentioned nighttime semantic segmentation method based on low-light enhancement and edge optimization, which has the following beneficial effects: (1) The low-light enhancement and repair network, combined with the local contrast enhancement and channel feature fusion mechanism, not only improves the overall brightness of the image, but also effectively preserves the image details and edge structure. The enhanced image is more suitable for subsequent segmentation tasks, and improves the model's perception ability and robustness in low-light environments.

[0018] (2) By introducing edge optimization loss, structural consistency is effectively maintained, and the consistency constraint between the enhanced image and the original image in terms of edge structure is achieved. When used in conjunction with traditional pixel loss, it can significantly reduce image over-smoothing and structural deformation, improve the accuracy of target boundary recognition, and is particularly suitable for blurred images at night.

[0019] (3) Introducing a channel attention mechanism into the discriminator enhances the response to key feature channels, effectively improves the discriminator's ability to distinguish between source and target domain images, thereby improving the adversarial training effect and stability, enabling more accurate capture of domain differences, and promoting domain alignment of the generator.

[0020] (4) A generator composed of a low-light enhancement and restoration network and a semantic segmentation network, combined with a discriminator with an embedded channel attention module, realizes an end-to-end semantic segmentation training framework, which integrates and optimizes image enhancement, semantic segmentation, domain discrimination and multiple loss functions. No additional stage training or external label support is required, realizing unsupervised transfer from the daytime domain to the nighttime domain, and has stronger automation and adaptability.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] Figure 1 is a flowchart of a nighttime semantic segmentation method based on low-light enhancement and edge optimization according to the present invention; Figure 2 is a schematic diagram of the low-light enhancement and repair network structure of the present invention; Figure 3 is a schematic diagram of the local contrast enhancement module structure of the present invention; Figure 4 is a schematic diagram of the semantic segmentation principle of the present invention; Figure 5 is a schematic diagram of the semantic segmentation network structure of the present invention; Figure 6 is a visualization result of semantic segmentation using different algorithms of the present invention. Detailed Implementation

[0023] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] As shown in Figure 1, a nighttime semantic segmentation method based on low-light enhancement and edge optimization has the following specific steps: Step S1: Prepare a training dataset and input the images in the training dataset into a low-light enhancement and insulation network based on Retinex theory, local contrast enhancement and adaptive feature fusion to obtain intermediate images after noise reduction and enhancement processing.

[0026] Preparation of training dataset: Obtain source domain images and target domain images, which correspond to daytime images and nighttime images, respectively. After unifying the size of the obtained images, normalize them to obtain the training dataset.

[0027] As shown in Figure 2, the low-light enhancement and restoration network comprises a Retinex-based illumination estimation module, a local contrast enhancement module, and an adaptive feature fusion module, arranged sequentially. The core modules of the low-light enhancement and restoration module (illumination generator, local contrast enhancement module, and adaptive feature fusion module) all revolve around the decomposition and optimization of illumination-reflection components under Retinex theory: the illumination generator is responsible for decomposition, the local contrast enhancement module is responsible for optimizing the details of the reflection components, and the adaptive feature fusion module is responsible for fusing the multi-scale features after decomposition. The discrepancy lies in contrast enhancement and channel feature fusion. Contrast enhancement only corresponds to the local function of the local contrast enhancement module, and channel feature fusion is not the essence of the adaptive feature fusion module (it dynamically adjusts weights based on the global context, rather than simply fusing channel dimensions).

[0028] The Retinex-based illumination estimation module consists of a first convolutional layer and an illumination map generator connected in sequence, using images from the training dataset ( Corresponding image in the source domain, and The image (corresponding to the target domain) is processed through the first convolutional layer and a lighting map generator to generate a feature map. The input image is decomposed into illumination and reflection components via convolution, thus extracting key features and generating the lighting map. A 1×1 convolution expands the feature dimension to achieve fine-grained spatial information extraction without changing the spatial resolution. A 5×5 deep convolution enhances channel-specific feature learning, while simultaneously fusing global average channel and local features of the input image. This balances model complexity and feature extraction capability to generate a robust lighting map, effectively enhancing image clarity under complex lighting conditions and thus improving image clarity under different lighting conditions.

[0029] The feature map enters the local contrast enhancement module, as shown in Figure 3. This module includes two second convolutional layers, which compute the context mask while maintaining spatial dimension invariance. The context mask is then normalized using Softmax and applied to the original input element-wise. The results are summed spatially to generate the final context representation. This approach captures long-range dependencies, improves understanding of the overall structure, and effectively preserves key textures and edge details, thereby reducing information loss and enhancing feature representation capabilities.

[0030] The output and feature maps of the local contrast enhancement module are fed into the adaptive feature fusion module through downsampling and upsampling. Global contextual information is extracted using adaptive average pooling, and the number of channels in the feature maps is compressed using dimensionality reduction convolution to generate compact feature representations. Fully connected layers remap the features back to the original number of channels, generating attention maps. These attention maps are then normalized using a softmax layer. During the fusion process, features are adaptively weighted according to their importance at each scale. The adaptive feature fusion module can adjust weights based on the dynamic content of the input features, emphasizing information-rich features and suppressing unimportant ones, effectively overcoming the limitations of traditional fusion methods.

[0031] The output of the adaptive feature fusion module is input to the third convolutional layer. The output of the third convolutional layer is added to the output of the Retinex-based illumination estimation module and then input to the fourth convolutional layer. Simultaneously, images from the training dataset are input to the fourth convolutional layer, which outputs an intermediate image. , , By efficiently processing illumination components and combining optimized convolutional blocks to enhance context-sensitive features, it significantly improves image sharpness and color fidelity, addressing the noise, color distortion, and detail loss issues inherent in traditional network techniques in complex lighting environments.

[0032] Step S2: Input the intermediate image into the semantic segmentation network, as shown in Figure 4. The semantic segmentation network outputs the category distribution map of each pixel. , , The semantic segmentation network employs an improved semantic segmentation model, which includes a ResNet-101 backbone network. It utilizes a pyramid pooling module to aggregate global contextual information. However, its fixed-scale pooling operation is ill-suited to adapting to complex scenes such as low-light conditions at night and abrupt changes in target scale. Furthermore, deep features are prone to losing minute target edge details after multiple downsampling operations, as shown in Figure 5. To address this, a polarization self-attention module is embedded in the residual module of the ResNet-101 backbone network. This module performs channel-space dual-dimensional dynamic calibration on the convolutional features before the residual connections, focusing on key regions (such as low-light textures and small target contours) through a weight matrix. This preserves the global structural information passed from the original skip connections, improving the segmentation accuracy for nighttime scenes.

[0033] The polarization self-attention module includes a channel attention submodule and a spatial attention submodule set in parallel. The channel attention submodule and the spatial attention submodule calculate the attention weights respectively, and finally the results of the two are fused to achieve dynamic feature calibration.

[0034] Let the channel branch self-attention mechanism be... : ;in, , and This represents the learned convolutional kernel applied to the feature map. and It is a tensor reshaping operator used to adjust dimensions to perform matrix multiplication. It is the SoftMax operator, which performs normalized weighting in the spatial dimension. For feature maps, Input identifiers for query features. Input identifier for value feature, and They represent Output identifiers and The output result identifier.

[0035] "×" represents the matrix dot product operation, expressed as: ;remember Indicates channel multiplication. The total number of feature elements participating in the SoftMax operator. For the first Feature elements at each position, For the first Feature elements at each position.

[0036] Suppose a spatial branching self-attention mechanism ,definition: ;in, , and There are three tensor reshaping operators. It is the SoftMax operator. It is a global pool operator.

[0037] ;in, and These represent the height and width of the feature map, respectively. and Indices for height and width, respectively; [Note: The original text contains some formatting errors and inconsistencies. A Representing space multiplication, the output of the parallel PSA attention mechanism is: ; The output features of channel branch self-attention. For the output features of spatial branch self-attention, and These are the channel branch self-attention index and the spatial branch self-attention index, respectively.

[0038] By halving the channel dimension without changing the spatial resolution, the performance of fine-grained pixel-level tasks is optimized while maintaining low computational complexity. Therefore, the polarization self-attention module effectively improves the accuracy of image pixel-level segmentation tasks.

[0039] Step S3: Input the discriminator with embedded channel attention module according to the category distribution map of each pixel, and perform end-to-end optimization of the generator composed of low-light enhancement and inpainting network and semantic segmentation network by jointly optimizing the loss function through multiple tasks including image reconstruction loss, semantic segmentation loss, static loss and adversarial loss.

[0040] The generator, composed of a low-light enhancement and inpainting network and a semantic segmentation network, is optimized through a multi-task joint optimization loss function. This function includes image reconstruction loss, semantic segmentation loss, static loss, and adversarial loss. This approach aims to enhance detail edges and improve the model's generalization ability while preserving pixel-level accuracy. The image reconstruction loss includes a basic pixel loss. Chabonnier's loss And edge loss.

[0041] Image reconstruction loss includes basic pixel loss. And the loss of Chabonnier Used to constrain the pixel-level alignment of the enhanced image with the true value and suppress overall deviation, it is calculated as follows: Basic pixel loss Used to calculate the mean absolute error (MAE) between the enhanced and real images, suppressing overall bias: ; in, These are the model's predicted values. is the true value, and n is the number of samples.

[0042] Chabonnier loss The robustness to noise and outliers is enhanced by smoothing the square root form of the L1 norm, avoiding over-smoothing of the reconstructed image. The calculation is as follows: ;in, It is an enhanced image. It is a real image. It is a constant, usually set to .

[0043] The edge loss extracts edge features using the Laplacian operator, constraining the edge structure preservation ability of the reconstructed image, and is calculated as follows: ;in, For edge loss, For the original image The original image edge feature map obtained based on the Laplacian operator To generate an edge feature map of the generated image based on the Laplacian operator, The Euclidean norm is used to calculate the pixel-level differences between the edge feature maps of the original and generated images. It is a constant.

[0044] Combining the above three factors, image reconstruction loss Its weights can be adjusted as follows: ;in, control and Weight, .

[0045] The semantic segmentation loss uses a weighted cross-entropy loss to reweight class imbalance, and optimizes segmentation by improving the segmentation performance of small objects. ;in, It is the m-th channel of the segmentation result. These are the weights set. N is the number of valid pixels in the segmentation marker image. It is the number of labeled categories in the dataset. for Norm summation, This represents a one-hot encoding of the basic facts of the m-th category. To increase the weight of smaller objectives, the following reweighting strategy is adopted: ;in, It represents the proportion of pixels of category m in the source dataset, with each category... The weights are normalized as follows: Here, This is the scaling factor, which is set to 0.05 in the text. This is the average value. Standard deviation This is the original value for category m.

[0046] Static loss leverages the spatial consistency of static objects (such as roads and buildings) in day and night scenes, employing pseudo-labels to supervise the loss. Definition and We focus only on the static category prediction results for the target during the day and the target at night, respectively, at the channel dimension. This represents the number of categories of static objects. Subsequently, the daytime reweighted prediction result is obtained using equation (10). The static loss is then: Where N is the number of valid pixels in the corresponding segmentation marker map. A likelihood diagram representing the correct category is defined as follows: ;operate Indicates semantic pseudo-tags One-hot encoding, Therefore Each location within a 3×3 area centered on the center. These are category labels in semantic segmentation tasks.

[0047] Adversarial loss employs adversarial learning to align the output distributions between pixels in the source and target domains. A least-squares loss function is used to train two discriminators. and Adversarial loss is defined as: ; For source discriminator, The target label is the discriminator; the objective function of the two discriminators is determined by the following formula: ; ;in They are target field labels with the same output resolution. , , These are the semantic prediction results for the source domain, the target domain during the day, and the target domain at night, respectively.

[0048] Step S4: Input the nighttime image to be segmented into the optimized generator and output the segmentation result.

[0049] To verify the superiority of the technical solution in this embodiment, experimental verification was conducted on three datasets: Cityscapes, DarkZurich, and Nighttime Driving.

[0050] The Cityscapes dataset provides a wealth of city street view images with detailed annotations, offering valuable data resources for semantic segmentation and other computer vision tasks. The dataset contains 5000 street view images from 50 cities, with pixel-level annotations across 19 categories. These images include 2975 training images, 500 validation images, and 1525 publicly annotated test images, all with a resolution of 2048×1024.

[0051] The Dark Zurich dataset is specifically designed for nighttime segmentation tasks in autonomous driving scenarios. It contains city street view images categorized into daytime, dusk, and nighttime, with annotations identical to those in the Cityscapes category. This facilitates model transfer, adaptation, and comparison between the two datasets. The training set contains 3024 unannotated images for unsupervised or semi-supervised training, enhancing the model's generalization ability in nighttime scenes. The validation set contains 50 annotated images, and the test set contains 151 annotated images, all at a resolution of 1920×1080, used to evaluate the model's performance in nighttime scenes.

[0052] The Nighttime Driving dataset consists of various nighttime driving scenarios, including different weather conditions and lighting conditions. This dataset contains 50 annotated nighttime images. Similar to the Cityscapes and Dark Zurich datasets, the images in the Nighttime Driving dataset are also annotated into 19 categories. The images have a resolution of 1280×720. These images focus on nighttime driving scenarios, filling the gaps in daytime and nighttime driving data, and helping to improve the performance of models under nighttime conditions.

[0053] Evaluation was performed using the mean Intersection over Union (mIoU). All experiments were conducted on an Ubuntu 20.04 server with an NVIDIA GeForce GTX 3090 graphics card. The basic network was built using PyTorch 1.7.0. Due to limited GPU memory, the image size was adjusted to 512×512. The model was trained using a stochastic gradient descent optimizer, and the discriminator was trained using the Adam optimizer, with a momentum coefficient of 0.9, a weight decay factor of 5×10⁻⁴, an initial learning rate of 2.5×10⁻⁴, and a batch size of 2. Comparison data with several existing algorithms is shown in Table 1.

[0054] Table 1 Comparison data of multiple algorithms

[0055] Existing algorithms suffer from problems such as large model parameter count, blurred edge information, and multi-stage stitching. This embodiment preserves key texture details and reduces the number of parameters; at the same time, it uses a generative adversarial network to avoid the impact of multi-stage steps on segmentation. Experimental results show that the network architecture proposed in this paper outperforms traditional networks using domain adaptation methods on two authoritative datasets (Dark Zurich-test and Nighttime Drivingtest). It demonstrates good adaptability and the ability to highlight class boundaries to achieve more accurate semantic segmentation. Tests were conducted on the Nighttime Drivingtest dataset, and the semantic segmentation results of each algorithm are visualized in Figure 6. It can be seen that IA-Seg has the highest segmentation accuracy for trucks in the first row (dash box). In the second row, the segmentation of pedestrians and streetlights is clearer and closest to the label results. In the third and fourth rows, the recognition effect of this embodiment for small distant objects is more obvious compared to the other two methods, indicating that the technical solution of this embodiment improves the performance of semantic segmentation at night, especially in recognizing small objects.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A nighttime semantic segmentation method based on low-light enhancement and edge optimization, characterized in that, The specific steps are as follows: Step S1: Prepare a training dataset and input the images in the training dataset into a low-light enhancement and inpainting network based on Retinex theory, local contrast enhancement and adaptive feature fusion to obtain intermediate images after noise reduction and enhancement; Step S2: Input the intermediate images into a semantic segmentation network, which outputs a category distribution map for each pixel; Step S3: Input the category distribution map for each pixel into a discriminator with an embedded channel attention module, and perform end-to-end optimization of the generator composed of the low-light enhancement and inpainting network and the semantic segmentation network by jointly optimizing the loss function through multiple tasks including image reconstruction loss, semantic segmentation loss, static loss and adversarial loss; Step S4: Input the nighttime image to be segmented into the optimized generator and output the segmentation result.

2. The nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 1, characterized in that: In step S1, source domain images and target domain images are obtained, corresponding to daytime images and nighttime images, respectively. The obtained images are then normalized after being standardized to obtain the training dataset.

3. The nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 2, characterized in that: The low-light enhancement and restoration network consists of a Retinex-based illumination estimation module, a local contrast enhancement module, and an adaptive feature fusion module, arranged sequentially.

4. The nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 3, characterized in that: The Retinex-based illumination estimation module consists of a first convolutional layer and a illumination map generator connected in sequence. Images in the training dataset are processed by the first convolutional layer and the illumination map generator to generate feature maps.

5. A nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 4, characterized in that: The feature map enters the local contrast enhancement module, which includes two second convolutional layers. This allows the context mask to be calculated while maintaining the spatial dimension. The context mask is normalized by Softmax and applied to the original input through element-wise multiplication. The results are then summed in the spatial dimension to generate the final context representation.

6. The nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 5, characterized in that: The output and feature map of the local contrast enhancement module are fed into the adaptive feature fusion module through downsampling and upsampling. Global context information is extracted through adaptive average pooling, and the number of channels of the feature map is compressed by dimensionality reduction convolution to generate feature representation. The fully connected layer remaps the features back to the original number of channels to generate attention map. The attention map is normalized by the Softmax layer. During the fusion process, adaptive weighting is performed according to the importance of features at each scale. The output of the adaptive feature fusion module is input into the third convolutional layer. The output of the third convolutional layer is added to the output of the Retinex-based illumination estimation module and then input into the fourth convolutional layer. At the same time, the images in the training dataset are input into the fourth convolutional layer and the output is an intermediate image.

7. A nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 6, characterized in that: In step S2, the semantic segmentation network adopts an improved semantic segmentation model, which includes a ResNet-101 backbone network. A polarization self-attention module is embedded in the residual module of the ResNet-101 backbone network. The convolutional features are dynamically calibrated in two dimensions, channel and space, before the residual connection. The key regions are focused through the weight matrix, and the global structural information passed by the original skip connections is preserved.

8. A nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 7, characterized in that: The polarization self-attention module includes a channel attention submodule and a spatial attention submodule set in parallel. The channel attention submodule and the spatial attention submodule calculate the attention weights respectively, and finally the results of the two are fused to achieve dynamic feature calibration.

9. A nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 8, characterized in that: The discriminator of the channel attention module consists of a fully convolutional network with a multi-layer convolutional structure, and the channel attention module is embedded in the fully convolutional network.

10. A nighttime semantic segmentation method based on low-light enhancement and edge optimization according to claim 9, characterized in that: The multi-task joint optimization loss function includes image reconstruction loss, semantic segmentation loss, static loss, and adversarial loss; the image reconstruction loss includes basic pixel loss. Chabonnier's loss And edge loss; the edge loss is as follows: ;in, For edge loss, For the original image The original image edge feature map obtained based on the Laplacian operator To generate an edge feature map of the generated image based on the Laplacian operator, The Euclidean norm is used to calculate the pixel-level differences between the edge feature maps of the original and generated images. It is a constant.

Images