Aerial image semantic segmentation method for outdoor scenes in low-light environment

Abstract

The present invention relates to the field of image data processing technology, and discloses a method for semantic segmentation of aerial images for outdoor scenes in a low-light environment, comprising the steps of: decomposing an original image into a V-channel illumination map and an RGB image, inputting the V-channel illumination map into an illumination feature extraction network, and outputting a plurality of illumination guide maps; inputting the plurality of illumination guide maps into a first illumination feature fusion module to obtain an illumination guide fusion map; inputting the RGB image into an image feature extraction network, outputting a plurality of feature maps, and inputting each feature map into an illumination feature embedding module in a one-to-one correspondence with an illumination guide map for fusion, and outputting an illumination embedding map; inputting the illumination embedding map and the illumination guide fusion map into a second illumination feature fusion module to obtain an illumination embedding fusion map; and extracting a segmentation result from the illumination embedding fusion map. The present invention improves the processing efficiency and quality of images in outdoor low-light environments.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a semantic segmentation method of aerial images for outdoor scenes in a low-light environment. Background Art

[0002] In low-light environments, the loss of image details and color distortion pose great challenges to semantic segmentation tasks, especially in images taken by drones in wild scenes. To address these issues, deep learning models play a vital role in semantic segmentation under low-light conditions. The following are some deep learning technologies and research directions in this specific scenario:

[0003] Convolutional Neural Network (CNN): As the core architecture for image processing tasks, CNN has demonstrated strong capabilities in low-light semantic segmentation. By using classic architectures such as U-Net, the model can simultaneously capture local and global features of the image. Integrating data under low-light conditions in this architecture allows the model to better learn and adapt to image features in low-light environments.

[0004] Multi-scale feature extraction: Under low-light conditions, there may be varying degrees of detail loss and color deviation at different scales of the image. Through the multi-scale feature extraction method, the model can capture the information in the image more comprehensively and effectively improve the semantic segmentation performance under low-light conditions.

[0005] Attention mechanism: The attention mechanism can help the model better focus on areas with important semantic information in low-light environments. Since some parts of the image may lack obvious features in low-light conditions, the introduction of the attention mechanism allows the model to concentrate resources during segmentation and prioritize these key areas, thereby improving segmentation accuracy.

[0006] Data augmentation technology: Under low-light conditions, data augmentation technology can help the model better adapt to diverse lighting changes by introducing methods such as brightness adjustment, contrast enhancement, and color correction, thereby improving its performance under complex lighting conditions.

[0007] In addition, it also includes methods such as transfer learning, illumination normalization, and post-processing technology, which work together to enhance the model's image semantic segmentation capabilities in low-light environments. In order to further improve the image processing efficiency of drones in low-light environments in the wild, it is necessary to integrate and optimize these methods to achieve more efficient semantic segmentation. Summary of the invention

[0008] The purpose of the present invention is to improve the processing efficiency and quality of images in outdoor low-light environments, and to provide a semantic segmentation method for aerial images of outdoor scenes in low-light environments.

[0009] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0010] A method for semantic segmentation of aerial images for outdoor scenes in low-light environments is implemented through a segmentation model, wherein the segmentation model includes an illumination feature extraction network, an image feature extraction network, an illumination feature embedding module, a first illumination feature fusion module, and a second illumination feature fusion module;

[0011] The method comprises the following steps:

[0012] Step 1, decompose the original image into a V channel illumination map and an RGB image, input the V channel illumination map into the illumination feature extraction network, and output multiple illumination guide maps;

[0013] Step 2, after inputting the multiple illumination guidance maps into the first illumination feature fusion module, a illumination guidance fusion map is obtained;

[0014] Step 3: Input the RGB image into the image feature extraction network, output multiple feature maps, and compare each feature map with a light guide Figure 1 The corresponding input illumination feature embedding module is fused and the illumination embedding map is output;

[0015] Step 4, inputting the illumination embedding map and the illumination guided fusion map into the second illumination feature fusion module to obtain the illumination embedding fusion map;

[0016] Step 5: Extract the segmentation result from the illumination embedding fusion map.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] (1) Improved semantic segmentation accuracy under low-light conditions: This paper introduces an illumination feature embedding module to effectively fuse illumination information with semantic features, so that the segmentation model can better adapt to low-light environments in field scenes. By dynamically adjusting the weight of illumination features in each convolutional layer, the segmentation model can accurately capture key semantic information under low-light conditions, thereby significantly improving the accuracy of semantic segmentation.

[0019] (2) Accurate capture of detail information: ESTDC is used in combination with 3×3 convolution kernels and 7×7 convolution kernels to enhance the segmentation model's ability to capture detailed features in the image. The 3×3 convolution kernel helps to extract delicate local features such as edges and textures, while the 7×7 convolution kernel ensures the effective acquisition of global information. This combination of multi-scale convolution kernels makes the segmentation model more sensitive to small and important changes in the scene when processing complex scenes in low-light environments, improving the model's refined segmentation capabilities.

[0020] (3) Improved real-time performance and computational efficiency: The present invention adopts a lightweight ESTDC architecture in the illumination feature extraction network and the image feature extraction network, which significantly reduces the amount of computation and memory usage while maintaining efficient feature extraction. Through the short-term dense connection mechanism, ESTDC can achieve deep feature fusion in a shallow network structure, ensuring the application effect of the segmentation model in the real-time processing tasks of drones. This design improves the real-time performance of the segmentation model in low-light environments and is suitable for embedded systems with limited resources.

[0021] (4) Larger information capacity of feature maps: Through the combination of the illumination feature embedding map module and the first illumination feature fusion module and the second illumination feature fusion module, the illumination embedding fusion map generated by the present invention is more informative and contains multi-level illumination and semantic features. The convolution operation further optimizes the expression capability of the illumination embedding fusion map, enabling the segmentation model to process more feature information in complex low-light scenes, thereby improving the overall semantic segmentation performance and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is a schematic diagram of the network structure of the segmentation model of the present invention;

[0024] Figure 2 is a schematic diagram of the network structure of the illumination feature extraction network of the present invention, Figure 2 (a) is a schematic diagram of the network structure of ESTDC in the illumination feature extraction network. Figure 2 (b) is a schematic diagram of the network structure of the illumination feature extraction network;

[0025] Figure 3 is a schematic diagram of the network structure of the image feature extraction network of the present invention, Figure 3 (a) is a schematic diagram of the network structure of ESTDC in the image feature extraction network. Figure 3 (b) is a schematic diagram of the network structure of the image feature extraction network;

[0026] Figure 4 Schematic diagram of the network structure of the first illumination feature embedding unit in the illumination feature embedding module of the present invention;

[0027] Figure 5 Schematic diagram of the network structure of the first illumination feature fusion module of the present invention. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0029] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0030] Embodiment 1:

[0031] The present invention is achieved through the following technical solutions: Figure 1 As shown, a semantic segmentation method of aerial images for outdoor scenes in low-light environments is proposed, including the following steps:

[0032] Step 1: Decompose the original image into a V-channel illumination map and an RGB image, input the V-channel illumination map into the illumination feature extraction network, and output multiple illumination guidance maps.

[0033] The original images are a large amount of drone image data taken under low-light conditions, including farmland, open space, forest and other outdoor scenes. These drone image data cover a variety of complex scenes from cities to rural areas. In order to ensure that the method of this scheme can accurately identify and segment different object categories in the image, these drone image data are semantically annotated pixel by pixel to generate a high-quality training data set. The training data set is used as the input of the segmentation model, and the segmentation model is trained. The segmentation model outputs the segmentation result, that is, the semantic category of each pixel.

[0034] The segmentation model includes an illumination feature extraction network, an image feature extraction network, an illumination feature embedding module, a first illumination feature fusion module, and a second illumination feature fusion module. The original image is decomposed through image channels to obtain a V channel illumination map and an RGB image.

[0035] See also Figure 1 , input the V channel illumination map into the illumination feature extraction network. Specifically, the illumination feature extraction network includes an illumination feature pre-extraction unit, a first illumination feature extraction unit, a second illumination feature extraction unit, and a third illumination feature extraction unit. After the V channel illumination map is input into the illumination feature pre-extraction unit, an illumination guidance map Pre_LG1 is obtained; the illumination guidance map Pre_LG1 is input into the first illumination feature extraction unit to obtain an illumination guidance map LG1; the illumination guidance map LG1 is input into the second illumination feature extraction unit to obtain an illumination guidance map LG2; the illumination guidance map LG2 is input into the third illumination feature extraction unit to obtain an illumination guidance map LG3.

[0036] See also Figure 2 , the network structure of the illumination feature pre-extraction unit is equivalent to Figure 2 Stage 1 in (b) consists of the initial convolutional layer ConvX1, ConvX2, ReLU activation function, and maximum pooling layer. The size of the V channel illumination map input to the illumination feature pre-extraction unit is H×W×1 (H is the image height, W is the image width, and 1 represents the number of illumination information channels). After being processed by the 3×3 initial convolutional layer ConvX1, ConvX2 and ReLU activation function, the maximum pooling operation is performed to generate an illumination guidance map Pre_LG1 with a size of H / 4×W / 4×C1 (C1 represents the number of feature channels).

[0037] The first illumination feature extraction unit is equivalent to Figure 2 In (b), stage 2 consists of X feature enhancement blocks (ESTDC), namely ESTDC3.1, ..., ESTDC3.X, and each ESTDC has the same network structure. Figure 2 In (a), ESTDC includes a 1×1 convolution block (Block1), a 3×3 convolution block (Block2), a 3×3 convolution block (Block3), a 3×3 convolution block (Block4), a feature fusion layer, and a channel attention layer, and the feature maps output by each block are fused in the feature fusion layer through jump connections. Each block includes a convolution layer, a batch normalization layer (BN), and an activation function ReLU. ESTDC performs deep processing on the illumination guidance map Pre_LG1, further compresses the spatial size through specific pooling operations, and generates an illumination guidance map LG1 with a size of H / 8×W / 8×4C1.

[0038] The second illumination feature extraction unit is equivalent to Figure 2 Stage 3 in (b) consists of X feature enhancement blocks (ESTDC), namely ESTDC4.1, ..., ESTDC4.X, which have the same structure as the first illumination feature extraction unit and are not repeated here. The illumination guidance map LG1 is input into the second illumination feature extraction unit for processing, and an illumination guidance map LG2 with a size of H / 16×W / 16×8C1 is generated.

[0039] The third illumination feature extraction unit is equivalent to Figure 2 In stage 4 of (b), there are X feature enhancement blocks (ESTDC), namely ESTDC5.1, ..., ESTDC5.X, which have the same structure as the first illumination feature extraction unit and are not repeated here. After the illumination guidance map LG2 is input into the third illumination feature extraction unit for processing, an illumination guidance map LG3 with a size of H / 32×W / 32×16C1 is generated.

[0040] The illumination feature extraction network compresses the image spatial size through specific pooling operations, reducing the spatial resolution of the feature map block by block, but extracts multi-scale illumination distribution features, while enhancing feature expression by increasing the number of channels, and finally generates an illumination guidance map LG3.

[0041] Step 2: After inputting multiple illumination guidance maps into the first illumination feature fusion module, an illumination guidance fusion map is obtained.

[0042] The lighting guidance maps Pre_LG1, LG1, LG2, and LG3 are spatially aligned and resized respectively to make the lighting guidance maps of different scales consistent in spatial dimensions. Figure 5 As shown in FIG. 1 , the illumination guidance map LG3 is processed by a 1×1 convolution kernel to obtain the feature map y4, and the illumination guidance map LG3 is processed by another 1×1 convolution kernel to obtain the feature map y5; the feature map y4 is upsampled and restored to the same spatial size as the illumination guidance map LG2, and the illumination guidance map LG2 is processed by a 1×1 convolution kernel and pooling, and then weightedly fused with the feature map y4 to obtain the feature map y3; the feature map y3 is upsampled and restored to the same spatial size as the illumination guidance map LG1, and the illumination guidance map LG2 is weightedly fused with the feature map y4 to obtain the feature map y3. After G1 is processed by 1×1 convolution kernel and pooling, it is weightedly fused with feature map y3 to obtain feature map y2; the feature map y2 is upsampled and restored to the same spatial size as the illumination guidance map Pre_LG1. After the illumination guidance map Pre_LG1 is processed by 1×1 convolution kernel and pooling, it is weightedly fused with feature map y2 to obtain feature map y1; the feature maps y1, y2, y3, and y4 are spliced ​​and then weightedly fused with feature map y5 to finally obtain the illumination guidance fusion map P1.

[0043] In this step, during the processing of the first illumination feature fusion module, the number of channels is reduced by a 1×1 convolution kernel and aligned with the shallow feature map in terms of the number of channels to ensure that the feature maps of different scales are consistent in spatial dimensions. The feature maps of each layer are added using a layer-by-layer weighted fusion strategy, and nonlinear processing is performed using the ReLU activation function. The size of the final illumination-guided fusion map P1 is H / 32×W / 32×Ccombined, where Ccombined represents the number of feature channels after fusion. This fusion process not only retains the information of illumination features of different scales, but also enhances the adaptability of the image to low-light environments.

[0044] Step 3: Input the RGB image into the image feature extraction network, output multiple feature maps, and compare each feature map with a light guide Figure 1 The corresponding input illumination feature embedding modules are fused and the illumination embedding map is output.

[0045] For more details, see Figure 1 The image feature extraction network includes an image feature pre-extraction unit, a first image feature extraction unit, a second image feature extraction unit, and a third image feature extraction unit. The illumination feature embedding module includes a first illumination feature embedding unit, a second illumination feature embedding unit, and a third illumination feature embedding unit.

[0046] The RGB image is input into the image feature pre-extraction unit to obtain a feature map Pre_F1; the feature map Pre_F1 is input into the first image feature extraction unit to obtain a feature map F1; the feature map F1 and the illumination guidance map LG1 are input into the first illumination feature embedding unit together to obtain an illumination embedding map LGF1; the illumination embedding map LGF1 is input into the second image feature extraction unit to obtain a feature map F2; the feature map F2 and the illumination guidance map LG2 are input into the second illumination feature embedding unit together to obtain an illumination embedding map LGF2; the illumination embedding map LGF2 is input into the third image feature extraction unit to obtain a feature map F3; the feature map F2 and the illumination guidance map LG3 are input into the third illumination feature embedding unit together to obtain an illumination embedding map LGF3.

[0047] See also Figure 3 , the network structure of the image feature pre-extraction unit is equivalent to Figure 3 Stage 1 in (b) consists of the initial convolution layer ConvX1, ConvX2, ReLU activation function, and maximum pooling layer. The RGB image of the input image feature pre-extraction unit is processed by the 3×3 initial convolution layer ConvX1, ConvX2 and ReLU activation function, and then the maximum pooling operation is performed to generate the feature map Pre_F1.

[0048] The first image feature extraction unit is equivalent to Figure 3Stage 2 in (b) consists of X feature enhancement blocks (ESTDC), namely ESTDC3.1, ..., ESTDC3.X, and each ESTDC has the same network structure. Figure 3 In (a), ESTDC includes a 1×1 convolution block (Block1), a 7×7 convolution block (Block2), a 5×5 convolution block (Block3), a 3×3 convolution block (Block4), a feature fusion layer, and a channel attention layer, and the feature maps output by each Block are fused in the feature fusion layer through jump connections. Each Block includes a convolution layer, a batch normalization layer (BN), and an activation function ReLU. After the feature map Pre_F1 enters the first image feature extraction unit, the feature map F1 is obtained.

[0049] Input the feature map F1 and the illumination guidance map LG1 into the first illumination feature embedding unit together, see Figure 4 It is a schematic diagram of the network structure of the first illumination feature embedding unit. The illumination guidance map LG1 is sequentially subjected to linear normalization, convolution, and size processing. The feature map F1 is sequentially subjected to linear normalization, convolution, and size processing, so that the processed LG1 and the feature map F1 have the same scale. Then they are fused, and the obtained fused feature map R1 has the ability of illumination perception. The fused feature map R1 is fused again with the processed feature map F1 to obtain the fused feature map R2. The fused feature map R2 is fused again with the unprocessed feature map F1, and the obtained fused feature map is the illumination embedding map LGF1.

[0050] Then the illumination embedding map LGF1 is input into the second image feature extraction unit, which is equivalent to Figure 3 Stage 3 in (b) consists of X feature enhancement blocks (ESTDC), namely ESTDC4.1, ..., ESTDC4.X. The network structure of each ESTDC is the same as that in stage 2 and will not be repeated here. The illumination embedding map LGF1 is input into the second image feature extraction unit for processing to obtain the feature map F2.

[0051] The feature map F2 and the illumination guide map LG2 are input into the second illumination feature embedding unit to obtain the illumination embedding map LGF2. The network structure of the second illumination feature embedding unit is the same as that of the first illumination feature embedding unit, which will not be described here.

[0052] Then the illumination embedding map LGF2 is input into the third image feature extraction unit, and the third image feature extraction unit is equivalent to Figure 3Stage 4 in (b) consists of X feature enhancement blocks (ESTDC), namely ESTDC5.1, ..., ESTDC5.X. The network structure of each ESTDC is the same as that in stage 2 and will not be repeated here. The illumination embedding map LGF2 is input into the third image feature extraction unit for processing to obtain the feature map F3.

[0053] The feature map F3 and the illumination guide map LG3 are input into the third illumination feature embedding unit to obtain the illumination embedding map LGF3. The network structure of the third illumination feature embedding unit is the same as that of the second illumination feature embedding unit, which will not be described here.

[0054] ESTDC uses a short-term dense connection mechanism to connect the output of each convolution block directly to the input of the subsequent convolution block, forming a local structure similar to a dense network. This structure not only retains the feature information of the previous layer, but also allows the network to capture multi-scale features, which is suitable for processing complex image details. According to network requirements, ESTDC can use convolution kernels of different sizes to adapt to different feature extraction tasks. For example, using a larger convolution kernel (such as 7×7) can capture the global features of the image, which is suitable for processing low-frequency features such as illumination information; while using a smaller convolution kernel (such as 3×3) is more suitable for capturing detailed features and local changes.

[0055] The ESTDC architecture in the image feature extraction network uses a 7×7 convolution kernel design, focusing on extracting detailed semantic features in the image. With a smaller convolution kernel, ESTDC can accurately capture high-frequency information of the image, such as edges and textures, within the local receptive field. At each stage of the image feature extraction network, the ESTDC with a 7×7 convolution kernel enhances the network's ability to capture and express detailed features through a short-term dense connection mechanism, constructs a more refined semantic feature map layer by layer, and finally generates a comprehensive feature map for semantic segmentation.

[0056] In this step, the low-level semantic feature map F1 and the illumination guidance map LG1 with multi-scale illumination features are fused. First, F1 and LG1 are compressed by 1×1 convolution kernels to reduce the number of channels and enhance the concentration of information. Then, the processed F1 and LG1 are fused by 3×3 to ensure the close combination of illumination information and low-level semantic features. The key to this step is to extract the detailed information of the image through small convolution kernels while maintaining the adaptability of illumination features to low-light environments. When the illumination feature embedding module fuses each layer, it uses an adaptive weight mechanism to weightedly fuse illumination features and semantic features to highlight important illumination information and suppress noise interference under low-light conditions. After each fusion, the feature map will pass through the ReLU activation function to ensure the effective expression of nonlinear features. The ReLU activation function not only improves the expression ability of the segmentation model, but also effectively prevents the gradient disappearance problem, so that the feature map is gradually enhanced in the extraction of deep semantic information. During the processing of each layer, the illumination feature embedding module further deepens the semantic level of the feature map. Through this layer-by-layer illumination information guidance, the final generated illumination embedding map LGF3 has stronger resistance to low-light interference and more accurate semantic segmentation performance.

[0057] In order to adapt to the lighting changes in different scenes, an adaptive weight mechanism is introduced to dynamically adjust the weights of lighting features and semantic features to ensure that the performance of the lighting embedding map always remains optimal under different lighting conditions. The fused lighting embedding map is nonlinearly processed through the ReLU activation function to further improve the performance of the segmentation model in low-light environments.

[0058] Step 4: Input the illumination embedding map and the illumination guided fusion map into the second illumination feature fusion module to obtain the illumination embedding fusion map.

[0059] The illumination embedding map LGF3 and the illumination guided fusion map P1 are input into the second illumination feature fusion module for final feature fusion. A global weighted mechanism is used to generate the illumination embedding fusion map P2 through deep convolution operation on LG3 and P1, and P2 is upsampled through the decoder to restore the original spatial resolution of the image while maintaining the feature details and semantic integrity of the illumination embedding fusion map P2.

[0060] Step 5: Extract the segmentation result from the illumination embedding fusion map.

[0061] The size of the segmentation result Rseg finally extracted from the illumination embedding fusion map P2 is H×W×Cseg, where Cseg is the number of channels of the semantic category. The segmentation result can clearly identify complex semantic scenes in low-light environments.

[0062] After training, the segmentation model can accurately identify and segment different semantic objects in low-light environments, ensuring its practicality and reliability in practical applications. The loss function used by the segmentation model is an adaptive edge enhancement loss function, which consists of three parts:

[0063] 1. Weighted Cross Entropy Loss: It deals with the problem of class imbalance and gives higher weight to minority classes (detail pixels).

[0064]

[0065] Among them, Weighted Cross Entropy Loss (p, y) represents the weighted cross entropy loss function; p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel (0 or 1); Represents the category weight, corresponding to the category y to which pixel i belongs i .

[0066] Category weight for:

[0067]

[0068] Where N is the total number of pixels i=1,2,...,N; Y is the total number of categories (for binary classification problems, Y=2); Represents category y i This weight calculation ensures that minority class pixels have a larger proportion in the loss function.

[0069] 2. Edge Enhancement Loss: Use edge information to enhance the segmentation model's attention to details and edge areas.

[0070]

[0071] Among them, Edge Enhancement Loss(p,y) represents the edge enhancement loss function; i represents the edge weight of the i-th pixel, corresponding to the possibility that the pixel is located on the edge.

[0072] Edge weight e i for:

[0073]

[0074] Among them, K is a hyperparameter, which represents the weight multiple of edge pixels, K>1; E iis the value of the i-th pixel in the edge graph E, indicating the edge strength; E min is the minimum value of pixels in the edge graph E; E max is the maximum value of the pixels in the edge map E. Normalize the edge map E to the interval [1, K] so that the non-edge pixels (E i =E min ) has a weight of 1, and pixels with higher edge strength (E i Close to E max ) has a weight close to K.

[0075] 3. Dice Loss: Measures the degree of repetition between the predicted results and the true labels, and promotes global segmentation performance.

[0076]

[0077] Where Dice Loss(h,y) represents the Dice loss function; h i represents the predicted category of the i-th pixel.

[0078] The total loss function (adaptive edge enhancement loss function) is:

[0079]

[0080] in, Represents the weight coefficient, satisfying .

[0081] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A semantic segmentation method for aerial images of outdoor scenes in low-light environments is implemented through a segmentation model, which is characterized by: The segmentation model includes an illumination feature extraction network, an image feature extraction network, an illumination feature embedding module, a first illumination feature fusion module, and a second illumination feature fusion module; The method comprises the following steps: Step 1, decompose the original image into a V channel illumination map and an RGB image, input the V channel illumination map into the illumination feature extraction network, and output multiple illumination guide maps; In step 1, the step of inputting the V channel illumination map into the illumination feature extraction network and outputting multiple illumination guide maps includes: The illumination feature extraction network includes an illumination feature pre-extraction unit, a first illumination feature extraction unit, a second illumination feature extraction unit, and a third illumination feature extraction unit; After the V channel illumination map is input into the illumination feature pre-extraction unit, an illumination guide map Pre_LG1 is obtained; Inputting the illumination guidance map Pre_LG1 into the first illumination feature extraction unit to obtain the illumination guidance map LG1; Inputting the illumination guidance map LG1 into the second illumination feature extraction unit to obtain the illumination guidance map LG2; Inputting the illumination guidance map LG2 into the third illumination feature extraction unit to obtain the illumination guidance map LG3; Step 2, after inputting the multiple illumination guidance maps into the first illumination feature fusion module, a illumination guidance fusion map is obtained; The step 2 specifically includes the following steps: The illumination guidance map LG3 is processed by a 1×1 convolution kernel to obtain the feature map y4. At the same time, the illumination guidance map LG3 is processed by another 1×1 convolution kernel to obtain the feature map y5. The feature map y4 is upsampled and restored to the same spatial size as the illumination guidance map LG2. After the illumination guidance map LG2 is processed by a 1×1 convolution kernel and pooling, it is weightedly fused with the feature map y4 to obtain the feature map y3. The feature map y3 is upsampled and restored to the same spatial size as the illumination guidance map LG1. After the illumination guidance map LG1 is processed by a 1×1 convolution kernel and pooling, it is weightedly fused with the feature map y3 to obtain the feature map y2. The feature map y2 is upsampled and restored to the same spatial size as the illumination guidance map Pre_LG1. After the illumination guidance map Pre_LG1 is processed by a 1×1 convolution kernel and pooling, it is weightedly fused with the feature map y2 to obtain the feature map y1. After the feature maps y1, y2, y3, and y4 are concatenated, they are weighted fused with the feature map y5 to finally obtain the illumination-guided fusion map P1. Step 3: Input the RGB image into the image feature extraction network, output multiple feature maps, and input each feature map into the illumination feature embedding module for fusion with an illumination guidance map one by one, and output the illumination embedding map; The step 3 specifically comprises the following steps: The image feature extraction network includes an image feature pre-extraction unit, a first image feature extraction unit, a second image feature extraction unit, and a third image feature extraction unit; the illumination feature embedding module includes a first illumination feature embedding unit, a second illumination feature embedding unit, and a third illumination feature embedding unit; The RGB image is input into an image feature pre-extraction unit to obtain a feature map Pre_F1; the feature map Pre_F1 is input into a first image feature extraction unit to obtain a feature map F1; Input the feature map F1 and the illumination guide map LG1 together into the first illumination feature embedding unit to obtain the illumination embedding map LGF1; input the illumination embedding map LGF1 into the second image feature extraction unit to obtain the feature map F2; Input the feature map F2 and the illumination guide map LG2 together into the second illumination feature embedding unit to obtain the illumination embedding map LGF2; input the illumination embedding map LGF2 into the third image feature extraction unit to obtain the feature map F3; Input the feature map F2 and the illumination guide map LG3 together into the third illumination feature embedding unit to obtain the illumination embedding map LGF3; Step 4, inputting the illumination embedding map and the illumination guided fusion map into the second illumination feature fusion module to obtain the illumination embedding fusion map; The step 4 specifically comprises the following steps: The illumination embedding map LGF3 and the illumination guided fusion map P1 are input into the second illumination feature fusion module for final feature fusion. The illumination embedding map LG3 and the illumination guided fusion map P1 are deep convolutionally operated to generate the illumination embedding fusion map P2, and the illumination embedding fusion map P2 is upsampled through the decoder to restore the original spatial resolution of the image. Step 5: Extract the segmentation result from the illumination embedding fusion map.

2. The method for semantic segmentation of aerial images for outdoor scenes in low-light environments according to claim 1, characterized in that: The total loss function of the segmentation model is: in, Represents the weight coefficient, satisfying ; Weighted Cross Entropy Loss(p,y) represents the weighted cross entropy loss function, which is: Among them, p i represents the predicted probability of the i-th pixel; y i represents the true label of the i-th pixel; Represents the category weight, corresponding to the category y to which pixel i belongs i ; Category weight for: Where N represents the total number of pixels i=1,2,...,N; Y represents the total number of categories; Represents category y i The number of pixels; Edge Enhancement Loss(p,y) represents the edge enhancement loss function, which is: e i represents the edge weight of the i-th pixel, which is: Among them, K is a hyperparameter, K>1; E i is the value of the i-th pixel in the edge graph E; E min is the minimum value of pixels in the edge graph E; E max is the maximum value of pixels in the edge map E; Dice Loss(h,y) represents the Dice loss function, which is: Among them, h i represents the predicted category of the i-th pixel.

Images