Multi-scale edge-constrained semantic segmentation upsampling optimization method and related equipment

Abstract

This invention discloses a multi-scale edge-constrained semantic segmentation upsampling optimization method and related equipment, relating to the field of remote sensing image semantic segmentation technology. The method includes: processing the remote sensing image data to be processed to obtain multiple intermediate-scale fused feature maps and a final-scale fused feature map; performing upsampling and classification prediction based on the multiple intermediate-scale fused feature maps to obtain multiple intermediate prediction results; then training and optimizing a multi-scale edge supervision module, an upsampling optimization module, and a feature extraction network; and using the trained and optimized multi-scale edge supervision module, feature extraction network, and multiple upsampling optimization modules, generating a semantic segmentation result of the remote sensing image data to be processed based on the final-scale fused feature map. This invention improves the final generated semantic segmentation result in terms of boundary clarity, small land cover integrity, and overall segmentation accuracy.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation technology for remote sensing images, and in particular to a multi-scale edge-constrained semantic segmentation upsampling optimization method and related equipment. Background Technology

[0002] Remote sensing image classification is a key task in remote sensing image understanding. With the continuous improvement of image spatial resolution, the fine-grained structure and spatial heterogeneity of ground features are increasingly enhanced, and traditional methods based on artificial features show limitations in feature representation and generalization performance. Deep learning-based semantic segmentation methods have demonstrated significant advantages in this field through end-to-end pixel-level modeling. Fully convolutional networks (FCNs) achieved end-to-end pixel-level prediction for the first time, laying the foundation for semantic segmentation. U-Net, through its symmetric encoder-decoder structure and skip connections, effectively fuses shallow spatial features with deep semantic features, exhibiting excellent boundary preservation. Subsequent improvements to U-shaped or U-like structures have further enhanced its application performance in remote sensing image segmentation. However, high-resolution remote sensing image semantic segmentation still faces unique challenges: complex ground feature types, large scale differences, blurred boundaries, and adjacent ground features with similar spectra but different semantics place higher demands on the network's ability to extract abstract features and restore details.

[0003] Existing semantic segmentation networks face three key challenges when processing high-resolution remote sensing imagery. First, they inherently struggle with handling differences in land scale and complex scene context: on one hand, to acquire a sufficient global receptive field to understand more abstract semantic information, the network needs to perform multiple downsampling operations, inevitably leading to a gradual attenuation of spatial detail information (especially small targets and fine boundaries); on the other hand, to effectively acquire multi-scale information for accurate prediction of small land cover types, operations such as parallel structures, attention mechanisms, or spatial pyramids all increase computational overhead. Therefore, ensuring spatial detail while extracting abstract features and reducing computational overhead during multi-scale extraction are among the core challenges in improving the accuracy of detail segmentation in complex remote sensing scenes. Second, there is an imbalance between the encoder's powerful feature extraction capabilities and the decoder's limited feature representation capabilities. Current research focuses primarily on enhancing feature extraction in the encoding stage by introducing deeper backbone networks or more complex contextual modeling modules to obtain more discriminative deep semantic features. However, during the decoding stage, the process of upsampling these high-dimensional semantic features to recover pixel-level predictions is often simplified, such as by bilinear interpolation or simple convolutional layers. Their limited expressive power makes it difficult to fully mine and fuse the rich spatial and semantic information contained in deep features, causing the upsampling process to become a key bottleneck limiting further improvement in model performance. Therefore, it is urgent to design a refined upsampling mechanism with strong feature reconstruction and fusion capabilities to balance high-level semantic extraction and shallow detail recovery. Third, existing semantic segmentation networks typically lack effective supervision mechanisms for intermediate network layers during training, which limits the model's feature learning and representation optimization. Since the supervision signal is mainly concentrated in the final output layer, feature extraction and transmission in the intermediate processes of the network are difficult to adequately guide, potentially affecting the model's semantic understanding and segmentation accuracy in complex scenes.

[0004] To address the aforementioned challenges and problems, existing technologies have explored various approaches. To extract stronger abstract features, researchers have introduced more powerful backbone networks, such as ResNet and DenseNet. To balance multi-scale information, multi-scale contextual modeling methods have been proposed, such as the pyramid pooling module in PSPNet and the dilated convolution and pyramid pooling in the DeepLab series. However, these methods still have shortcomings in detail preservation and recovery. Some studies have proposed maintaining high-resolution features through parallel multi-resolution branches, which alleviates detail loss to some extent, but the computational cost is high, and subsequent research shows that it still struggles to balance the contradiction between deep semantic modeling and detail preservation. As a necessary process for recovering details, the decoder's upsampling process is crucial for balancing semantic modeling and detail preservation. However, many current studies tend to greatly enhance the encoder to indirectly simplify the decoder, often reducing the decoder's upsampling process to interpolation or simple convolution, making it difficult to fully utilize deep feature information. Furthermore, studies on upsampling typically employ dense feature connections and attention enhancement, resulting in low refinement and shallow coupling with the encoder.

[0005] Therefore, while existing technologies have made progress in encoder feature extraction, the decoder upsampling process still suffers from low feature utilization efficiency, insufficient detail recovery, and insufficient coupling with the encoder. Furthermore, the training process lacks an effective intermediate supervision mechanism. There is an urgent need for a semantic segmentation method that can combine the advantages of feature extraction, achieve refined upsampling, and introduce effective process supervision to systematically solve these problems. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies. Specifically, it provides a multi-scale edge-constrained semantic segmentation upsampling optimization method and related equipment, as detailed below: 1) In a first aspect, the present invention provides a semantic segmentation upsampling optimization method with multi-scale edge constraints, the specific technical solution of which is as follows: The process involves acquiring remote sensing image data to be processed; performing shallow feature extraction on the remote sensing image data to obtain shallow feature maps; inputting the shallow feature maps into a multi-scale edge supervision module for edge prediction and multi-scale downsampling processing to obtain edge prediction results and multiple downsampled feature maps at different scales; inputting the shallow feature maps into an optimized feature extraction network for deep semantic feature extraction to obtain deep semantic feature maps; inputting the deep semantic feature maps and multiple downsampled feature maps at different scales into multiple cascaded upsampling optimization modules for feature fusion and resolution restoration to obtain multiple intermediate-scale fused feature maps and a final-scale fused feature map; based on multiple The fused feature maps at intermediate scales are upsampled and classified for prediction to obtain multiple intermediate prediction results. The edge prediction results are then compared with preset edge labels to calculate the edge supervision loss, and the multi-scale edge supervision module is trained and optimized. The multiple intermediate prediction results are then compared with semantic segmentation labels to calculate the classification loss, and the classification loss is weighted and summed according to the training stage to obtain the deep supervision loss. The upsampling optimization module and the feature extraction network are trained and optimized based on the deep supervision loss. Using the trained and optimized multi-scale edge supervision module, feature extraction network, and multiple upsampling optimization modules, the semantic segmentation result of the remote sensing image data to be processed is generated based on the fused feature map at the final scale.

[0007] The beneficial effects of the multi-scale edge-constrained semantic segmentation upsampling optimization method provided by this invention are as follows: A multi-scale edge supervision module extracts downsampled feature maps at multiple scales, which are then progressively fused and their resolution restored using a cascaded upsampling optimization module. This process introduces multi-scale information while avoiding complex parallel computation structures, effectively mitigating the spatial detail attenuation problem caused by multiple downsampling while reducing computational overhead, thus improving the recovery capability for small targets and fine boundaries. The designed deep and shallow feature fusion unit and feature diversification unit constitute a refined upsampling optimization module. This module can fully mine and fuse the spatial and semantic information of deep semantic feature maps and shallow downsampled feature maps, enhancing the feature reconstruction and representation capabilities of the upsampling process and compensating for the capability imbalance between the encoder and decoder. The introduced edge supervision mechanism and segmented weighted depth supervision mechanism supervise the output of the multi-scale edge supervision module and the multiple intermediate prediction results generated by the upsampling optimization module, respectively, providing direct and phased guidance signals for feature learning and optimization in the intermediate layers of the network, improving network training efficiency and the accuracy of process features. In summary, these improvements work together to improve the final semantic segmentation results in terms of boundary clarity, small land cover integrity, and overall segmentation accuracy.

[0008] Based on the above scheme, the semantic segmentation upsampling optimization method with multi-scale edge constraints of the present invention can be further improved as follows.

[0009] Furthermore, the process of obtaining edge prediction results and multiple downsampled feature maps of different scales through the multi-scale edge supervision module includes: performing convolution operations on the shallow feature map to obtain a first feature; inputting the first feature into the first branch to perform convolution and classification operations to obtain the edge prediction result; and inputting the first feature into the second branch to perform at least two different downsampling operations to obtain multiple downsampled feature maps of different scales.

[0010] The beneficial effects of adopting the above-mentioned further scheme are: using a multi-scale edge supervision module to simultaneously generate edge prediction results and downsampled feature maps of multiple different scales, the network can explicitly model and optimize the perception of ground object boundaries in the early stages of training with low computational overhead, while directly providing structured, multi-granular spatial context information for the subsequent upsampling optimization process, thereby enhancing the network's ability to extract features of boundaries and multi-scale structures in complex scenes.

[0011] Furthermore, the optimized feature extraction network is a network obtained by modifying the DenseNet121 network structure; the modifications include: reducing the number of convolutional blocks in at least one dense block in the DenseNet121 network structure, replacing the average pooling operation in at least one transition layer with the max pooling operation, and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the shallow feature map in the initial stage.

[0012] The beneficial effects of adopting the above-mentioned further solutions are: reducing the number of convolutional blocks in dense blocks directly reduces feature redundancy and the number of model parameters, achieving a lightweight feature extraction network. Replacing average pooling with max pooling enhances the specificity and discriminative power of the feature response. Adjusting the initial convolutional layer parameters to reduce the initial downsampling factor effectively alleviates the attenuation of spatial detail information in the network input stage, preserving a richer foundation of underlying details for subsequent refined upsampling.

[0013] Furthermore, the upsampling optimization module includes a deep-shallow feature fusion unit and a feature diversification unit connected in sequence. In the multiple cascaded upsampling optimization modules, the deep-shallow feature fusion unit of the first upsampling optimization module is used to: normalize the deep semantic feature map and the downsampled feature map of the first scale from multiple downsampled feature maps of different scales, and then concatenate and fuse them with the learned parameters after scaling. The feature diversification unit of the first upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep-shallow feature fusion unit, and output the fused feature map of the first intermediate scale. The deep-shallow feature fusion units of each upsampling optimization module from the second to the last upsampling optimization module... The function is used to: receive the fusion feature map output by the previous upsampling optimization module, and then normalize and scale it with the corresponding downsampling feature maps of multiple different scales before concatenating and fusing them; the feature diversification unit of each upsampling optimization module from the second to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the current deep and shallow feature fusion unit, and output the corresponding intermediate scale fusion feature map; wherein, the fusion feature map output by the last upsampling optimization module is determined as the final scale fusion feature map, and the fusion feature maps output by the other upsampling optimization modules are determined as multiple intermediate scale fusion feature maps.

[0014] The beneficial effects of adopting the above-mentioned further scheme are as follows: the deep and shallow feature fusion unit achieves effective calibration and fusion of deep and shallow features under a unified dimension through normalization and learnable parameter scaling. The feature diversification unit captures diverse contextual features through multi-branch parallel processing. The cascaded multi-modules gradually fuse deep semantic information with shallow details at different scales and restore resolution, thereby systematically constructing a multi-scale and detailed fusion feature map sequence, realizing efficient utilization and refined reconstruction of deep features.

[0015] Furthermore, the weighted summation of classification loss based on training stage includes: dividing the training process into multiple training stages; assigning different weight coefficients to different types of classification loss functions in each training stage; and weighting and summing the calculation results of different types of loss functions corresponding to each intermediate prediction result according to the weight coefficients of the training stage in which they are located, thus constituting the contribution of the intermediate prediction result to the deep supervision loss.

[0016] The beneficial effects of adopting the above-mentioned further approach are: dividing the training process into multiple training stages and dynamically assigning weights to different types of classification loss functions makes the network training process more adaptive. Different optimization objectives are emphasized at different training stages; for example, classification accuracy may be more focused in the early stages, while boundary fit may be more important in the later stages. This supervision method provides differentiated and dynamic gradient guidance for multiple intermediate prediction results, optimizes the feature learning path of the network's intermediate layers, and improves overall training efficiency and the final performance of the model.

[0017] 2) In a second aspect, the present invention also provides a semantic segmentation upsampling optimization system with multi-scale edge constraints, the specific technical solution of which is as follows: The system includes a data acquisition module, a first extraction module, a downsampling processing module, a second extraction module, a fusion and restoration module, a classification prediction module, a first training optimization module, a second training optimization module, and a semantic segmentation module. The data acquisition module acquires the remote sensing image data to be processed. The first extraction module performs shallow feature extraction on the remote sensing image data to obtain a shallow feature map. The downsampling processing module inputs the shallow feature map into a multi-scale edge supervision module for edge prediction and multi-scale downsampling, obtaining edge prediction results and multiple downsampling feature maps at different scales. The second extraction module inputs the shallow feature map into an optimized feature extraction network for deep semantic feature extraction, obtaining a deep semantic feature map. The fusion and restoration module sequentially inputs the deep semantic feature map and multiple downsampling feature maps at different scales into multiple cascaded upsampling optimization modules for feature fusion and restoration. Resolution restoration yields fused feature maps at multiple intermediate scales and a final scale fused feature map. The classification prediction module performs upsampling and classification prediction based on the fused feature maps at multiple intermediate scales, generating multiple intermediate prediction results. The first training optimization module calculates the edge supervision loss using the edge prediction results and preset edge labels, and trains and optimizes the multi-scale edge supervision module. The second training optimization module calculates the classification loss using the multiple intermediate prediction results and semantic segmentation labels, and performs a weighted summation of the classification losses according to the training stage to obtain the deep supervision loss. Based on this deep supervision loss, it trains and optimizes the upsampling optimization module and the feature extraction network. The semantic segmentation module uses the trained and optimized multi-scale edge supervision module, feature extraction network, and multiple upsampling optimization modules to generate semantic segmentation results for the remote sensing image data to be processed based on the final scale fused feature map.

[0018] Based on the above scheme, the semantic segmentation upsampling optimization system with multi-scale edge constraints of the present invention can be further improved as follows.

[0019] Furthermore, the downsampling processing module is specifically used to: perform convolution operations on the shallow feature map to obtain the first feature; input the first feature into the first branch to perform convolution and classification operations to obtain the edge prediction result; input the first feature into the second branch to perform downsampling operations at least two different magnifications to obtain multiple downsampling feature maps of different scales.

[0020] Furthermore, the optimized feature extraction network is a network obtained by modifying the DenseNet121 network structure; the modifications include: reducing the number of convolutional blocks in at least one dense block in the DenseNet121 network structure, replacing the average pooling operation in at least one transition layer with the max pooling operation, and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the shallow feature map in the initial stage.

[0021] Furthermore, the upsampling optimization module includes a deep-shallow feature fusion unit and a feature diversification unit connected in sequence. In the multiple cascaded upsampling optimization modules, the deep-shallow feature fusion unit of the first upsampling optimization module is used to: normalize the deep semantic feature map and the downsampled feature map of the first scale from multiple downsampled feature maps of different scales, and then concatenate and fuse them with the learned parameters after scaling. The feature diversification unit of the first upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep-shallow feature fusion unit, and output the fused feature map of the first intermediate scale. The deep-shallow feature fusion units of each upsampling optimization module from the second to the last upsampling optimization module... The function is used to: receive the fusion feature map output by the previous upsampling optimization module, and then normalize and scale it with the corresponding downsampling feature maps of multiple different scales before concatenating and fusing them; the feature diversification unit of each upsampling optimization module from the second to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the current deep and shallow feature fusion unit, and output the corresponding intermediate scale fusion feature map; wherein, the fusion feature map output by the last upsampling optimization module is determined as the final scale fusion feature map, and the fusion feature maps output by the other upsampling optimization modules are determined as multiple intermediate scale fusion feature maps.

[0022] Furthermore, the second training optimization module is specifically used to: divide the training process into multiple training stages; assign different weight coefficients to different types of classification loss functions in each training stage; and sum the calculation results of different types of loss functions corresponding to each intermediate prediction result according to the weight coefficients of the training stage in which they are located, to constitute the contribution of the intermediate prediction result to the deep supervision loss.

[0023] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so as to enable the electronic device to implement any of the above-mentioned semantic segmentation upsampling optimization methods for multi-scale edge constraints.

[0024] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any of the above-mentioned semantic segmentation upsampling optimization methods for multi-scale edge constraints.

[0025] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is a flowchart illustrating a semantic segmentation upsampling optimization method with multi-scale edge constraints according to an embodiment of the present invention. Figure 2 A schematic diagram of the framework for a multi-scale edge-constrained semantic segmentation upsampling optimization method provided by the present invention; Figure 3 This is a schematic diagram of the upsampling optimization module in the framework of this invention; Figure 4 This is the semantic segmentation result of some test images on the Potsdam test set according to the present invention; Figure 5 This is the semantic segmentation result of some test images on the Vaihingen test set according to the present invention; Figure 6 This is a schematic diagram of the structure of a semantic segmentation upsampling optimization system with multi-scale edge constraints according to an embodiment of the present invention. Detailed Implementation

[0027] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0028] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0029] like Figure 1 As shown in the figure, a semantic segmentation upsampling optimization method with multi-scale edge constraints according to an embodiment of the present invention includes the following steps: S1. Obtain the remote sensing image data to be processed. The specific implementation process is as follows: S10. Download the required image and label files from the specified, publicly available, or authorized remote sensing datasets. For example, data can be obtained from the official release channels of the Potsdam and Vaihingen datasets provided by the International Society for Photogrammetry and Remote Sensing (ISPRS). For the Potsdam dataset, high-resolution orthophoto maps (TOP) containing four bands (red, green, blue, and near-infrared) and corresponding ground truth labels are required. For the Vaihingen dataset, high-resolution images containing three bands (near-infrared, red, and green) and their corresponding ground truth labels are required. These data files are usually stored in standard image formats such as GeoTIFF or PNG and must be downloaded completely from the data hosting platform to a specified directory on your local machine or server, either programmatically (e.g., using Python's requests library or the wget command) or manually.

[0030] S11. After the data files are downloaded, an integrity verification and data loading process needs to be performed. Specifically, image processing libraries (such as OpenCV, PIL, or specialized geospatial data processing libraries such as GDAL) are used to read the downloaded image and label files. This process needs to check the readability of each file, confirm that the number of image bands, spatial size (e.g., the original Potsdam image patch may be 6000×6000 pixels), and bit depth (e.g., 8 bits) meet expectations, and verify that the image and label are strictly aligned spatially. Simultaneously, the image data needs to be loaded from the file format and converted into a multidimensional array form in computer memory that can be directly manipulated by subsequent processing frameworks (such as PyTorch or TensorFlow), typically represented as a tensor with shape (height, width, number of channels).

[0031] S12. Since the original datasets may have different storage specifications and numerical ranges, this step aims to convert all the remote sensing image data to be processed into a unified internal representation format to eliminate the impact of source data differences on subsequent algorithms. This process mainly includes two key operations: numerical range normalization and channel order adjustment. First, the integer values ​​of image pixels (e.g., the range of 0-255) are converted into 32-bit floating-point numbers and linearly normalized to a fixed numerical range, such as [0,1] or [-1,1]. Assuming the original pixel values ​​are... Normalized pixel values It can be calculated using the following formula: ,in, Represents the intensity value of a pixel in the original image. This represents the predefined mean. This represents a predefined standard deviation. Alternatively, a simpler linear scaling can be used: Secondly, the channel order of image data is changed from the common (height, width, number of channels) to the more efficient (number of channels, height, width) format of deep learning frameworks, that is, the channel dimension is transposed.

[0032] S13. The format-standardized remote sensing image data needs to be effectively organized for use in subsequent training and testing processes. This step typically involves creating a structured data index in memory or high-speed storage. For example, lists or dictionaries can be created for the Potsdam and Vaihingen datasets, where each record is associated with the following information: the standardized image tensor, the corresponding ground truth label tensor, and possible image identifiers. All processed remote sensing image data is systematically organized into a logical data pool, thus completing the entire "acquisition" process and providing direct and standardized input for the shallow feature extraction in the subsequent step S2.

[0033] The remote sensing image data to be processed refers to the original high-resolution aerial or satellite imagery that needs to be pixel-level classified into land cover categories and is input into the semantic segmentation method of this invention, along with the accurate registered real segmentation label map used for supervised training or evaluation. This data typically originates from publicly available remote sensing semantic segmentation benchmark datasets, such as the Potsdam and Vaihingen datasets. The Potsdam dataset provides high-resolution orthophotos of a region, containing labels for various typical urban land cover categories, including impermeable surfaces, buildings, low vegetation, trees, and vehicles. The Vaihingen dataset provides images of a region containing similar semantic categories. Each piece of remote sensing image data to be processed contains a multi-band (e.g., RGB, RGB-IR) image file and a single-channel label file, where each pixel's integer value corresponds to a specific semantic category. This data is the source of model learning and inference; its quality, resolution, and scene complexity directly affect the application effect and performance evaluation of the method of this invention.

[0034] S2. Perform shallow feature extraction on the remote sensing image data to be processed to obtain a shallow feature map. The specific implementation process is as follows: S20. The standardized remote sensing image data to be processed obtained in step S1 is used as the input for this step. This input data is a data set with a shape of... The four-dimensional tensor, in which This indicates the number of samples in a batch. This represents the number of channels in the input image. and These represent the height and width of the input image, respectively. For an input of size 224×224 pixels, and The value of each tensor is 224. This tensor is loaded into the memory of the computing device, awaiting forward propagation computation.

[0035] S21. Construct a two-dimensional convolutional layer to perform shallow feature extraction. The key parameters of this convolutional layer are configured as follows: the kernel size is 3×3, the stride of the convolution operation is set to 1, and pixel padding is applied around the input feature map, with one padding layer to ensure that the spatial dimensions of the output feature map do not change when the stride is 1. The number of input channels of this convolutional layer is equal to the number of channels of the remote sensing image data to be processed. The number of output channels is set to 64. This convolutional layer contains a set of learnable weight parameters. and a learnable bias parameter .

[0036] S22, shape is The input tensor and the weight parameters of the convolutional layer Perform a two-dimensional convolution operation. The core computational idea of ​​this operation can be represented by the following formula: ,in, This represents the input four-dimensional image tensor. The symbol represents a two-dimensional convolution operation. This represents the bias vector, which is broadcast and added to each channel of the output feature map. This represents the output tensor after the convolution operation. Through this step, the input data is transformed into an intermediate feature representation with 64 channels.

[0037] S23. In order for the network to learn and represent complex nonlinear relationships, the output of the convolution operation... An element-wise modified linear unit activation function (ReLU) is applied. The ReLU function sets all values ​​less than zero in the input tensor to zero, while leaving values ​​greater than or equal to zero unchanged. Its mathematical expression is: This operation introduces nonlinearity, resulting in the activated feature tensor. .

[0038] S24. The feature tensor obtained after the above convolution and activation operations. This is formally defined as a shallow feature map. The shape of this shallow feature map is... Among them, space size and It remains consistent with the input image. This shallow feature map contains primary features with local spatial context information extracted from the original image, and it will be simultaneously fed into two subsequent key components: a multi-scale edge supervision module and an optimized feature extraction network, for further processing.

[0039] Shallow feature maps refer to the feature representations generated after processing the original input image through a few convolutional layers in the initial stage of the network. They are located at the front end of the entire neural network architecture, close to the input layer. A key characteristic of shallow feature maps is that they preserve high spatial resolution; their size is usually the same as or similar to the input image, thus containing rich detail information, such as low-level visual cues like edges, textures, and corners of objects. In this invention, shallow feature maps specifically refer to the feature tensors extracted by a single 3×3 convolutional layer (64 output channels). These features have not undergone depthwise nonlinear transformations and downsampling, resulting in lower semantic abstraction, but they are crucial for recovering accurate object boundaries. As the common starting point for subsequent multi-scale edge supervision modules and deep semantic feature extraction networks, shallow feature maps act as a bridge connecting raw pixel information with high-level semantic understanding.

[0040] S3. Input the shallow feature map into the multi-scale edge supervision module to perform edge prediction and multi-scale downsampling processing respectively, and obtain the edge prediction result and multiple downsampled feature maps at different scales.

[0041] The multi-scale edge supervision module is a neural network substructure with two independent branches. During training, it receives supervision from edge labels and its function is to simultaneously learn the edge features of the image and generate multi-scale spatial context information. The entire module performs feature transformation during forward propagation and optimizes its parameters through a loss function during backpropagation.

[0042] The specific implementation process of S3 includes: S30. Perform a convolution operation on the shallow feature map to obtain the first feature; this step is the entry operation of the multi-scale edge supervision module. The shallow feature map input to this step is a shape... The tensor of, where, It refers to the batch size. and These are the height and width of the feature map. The module uses a 2D convolutional layer with a kernel size of 3, a stride of 1, and padding of 1 to process the input. This convolutional layer has 64 input channels and 64 output channels. This convolution operation can be represented as: ,in, This represents the shallow feature map of the input. This represents a two-dimensional convolution operation. and These are the weights and bias parameters of this 3×3 convolutional layer. It is an activation function applied element-wise. The tensor obtained after this operation This is referred to as the first characteristic. Its spatial dimensions... Consistent with the shallow feature map, the number of channels remains 64, but it incorporates a wider range of local contextual information, providing a unified input basis for the processing of the subsequent two branches.

[0043] S31. Input the first feature into the first branch for convolution and classification operations to obtain the edge prediction result. The specific implementation process is as follows: The first branch is an edge prediction branch, whose purpose is to extract the boundary information of ground features in the image from the first feature. The layer structure of this branch consists of a 1×1 convolutional layer and a classification output layer. First feature First, a 1×1 convolutional layer is used. This layer integrates and compresses information between channels, mapping the features of 64 channels to fewer channels, such as two channels (corresponding to the "edge" and "non-edge" categories, respectively). Then, a pixel-wise softmax function is applied to this two-channel feature map, transforming the values ​​of the two channels at each spatial location into a probability distribution. The formula for calculating the softmax function is: ,in, This indicates that at a certain pixel position, the first... One channel ( The original output value of the image is used. After softmax, the probability map of the channel representing the "edge" category is taken as the probability prediction that each pixel is an edge. This probability map is the edge prediction result, and its shape is... Each pixel has a value between 0 and 1, with higher values ​​indicating a higher confidence that the location is a boundary. During the training phase, this edge prediction result is calculated using the binary edge label converted from the true segmentation label, employing the binary cross-entropy loss function shown in formula (1). The training is supervised, and the parameters of the 1×1 convolutional layer in the first branch and the 3×3 convolutional layer that produces the first feature are optimized using the backpropagation algorithm. Formula (1) is: , is the cross-entropy loss for binary classification, where N is the number of samples in a batch. It is the actual label of the nth sample, with a value of 0 or 1. It is the probability that the prediction of the nth sample is correct, and its value is between (0, 1).

[0044] S32. Input the first feature into the second branch and perform downsampling operations at at least two different magnifications to obtain multiple downsampled feature maps at different scales. The specific implementation process is as follows: The second branch is a multi-scale feature generation branch that does not involve trainable parameters and mainly performs deterministic spatial downsampling operations. This branch receives the first feature. As input, three downsampling ratios are applied in the specific implementation: 2x downsampling, 4x downsampling, and 8x downsampling. This is typically achieved through pooling operations. For example, a 2x downsampling is achieved using a max pooling layer with a kernel size of 2×2 and a stride of 2. Concatenating a pooling layer twice achieves 4x downsampling. Concatenating a pooling layer three times achieves 8x downsampling. Mathematically, for a 2×2 max pooling operation, the output feature map at a certain location... value The corresponding 2×2 local region in the input feature map The maximum value determines: Through such operations, from the same first feature Three sets of feature maps are derived: one set is a feature map that has undergone a 2x downsampling, and its spatial size becomes One set consists of feature maps that have undergone two 2x downsampling operations (i.e., 4x downsampling), resulting in a spatial size of [missing information]. One set consists of feature maps that have undergone three 2x downsampling operations (i.e., 8x downsampling), resulting in a spatial size of [missing information]. These three sets of feature maps, together with the unsampling original first feature (which can be considered as one scale), constitute a set of downsampled feature maps at multiple different scales. These feature maps retain the number of channels of the first feature (64 channels) but have different spatial receptive fields. They will be input into the subsequent upsampling optimization process to provide multi-scale edge detail information for different stages.

[0045] The first feature refers to the intermediate feature representation obtained after performing an initial 3×3 convolution and ReLU activation transformation on the input shallow feature map within the multi-scale edge supervision module. It serves as the common input source for the two parallel branches within the module, acting as a bridge between them. The spatial resolution of the first feature is identical to that of the input shallow feature map, and its channel count is typically designed to be 64. It contains local visual patterns that have undergone preliminary nonlinear enhancement. These patterns include both gradient information that can be used to determine object boundaries and fundamental features that can be used for multi-scale context modeling.

[0046] The edge prediction result is a probability map generated from the output of the first branch in the multi-scale edge supervision module. For each pixel location in the input image, it predicts the probability that the pixel belongs to an "object boundary." This map is a single-channel probability map obtained by performing a 1×1 convolution on the first feature to compress the channels, and then applying it to a softmax classification function. During model training, the edge prediction result needs to be compared with the binary edge map extracted from the true semantic segmentation labels, and a binary cross-entropy loss is calculated to drive the network to learn how to accurately identify boundaries between various types of ground features.

[0047] The multiple downsampled feature maps at different scales are a set of feature maps with the same number of channels but different spatial dimensions, generated by the second branch of the multi-scale edge supervision module. These feature maps are obtained by performing spatial downsampling operations on the first feature at different magnifications (e.g., 2x, 4x, 8x). They capture feature information of the original image at different spatial granularities: higher-resolution feature maps are rich in detail, while lower-resolution feature maps provide broader context. This set of feature maps, as a multi-scale information carrier, is directly fed into different stages of the subsequent upsampling optimization module to enhance the network's ability to perceive boundaries and structures at different scales when recovering details.

[0048] S4. Input the shallow feature map into the optimized feature extraction network to extract deep semantic features and obtain the deep semantic feature map; The optimized feature extraction network is a modified DenseNet121 network structure. The modifications include: reducing the number of convolutional blocks in at least one dense block of the DenseNet121 network structure; replacing average pooling operations with max pooling operations in at least one transition layer; and adjusting the kernel size and stride of the initial convolutional layers to reduce the downsampling factor of shallow feature maps in the initial stage. Specifically: 1) The first modification is reducing the number of convolutional blocks in the dense blocks. In the DenseNet121 network structure, the original sequence of the number of convolutional blocks in the four dense blocks is [6, 12, 24, 16]. According to Table 1, the optimized network reduces the number of convolutional blocks in the second and third dense blocks. Specifically, the number of convolutional blocks in the second dense block is reduced from 12 to 6, and the number of convolutional blocks in the third dense block is reduced from 24 to 12. The number of convolutional blocks in the first and fourth dense blocks remains unchanged, at 6 and 16 respectively. This modification directly reduces the total number of network parameters and computational burden, achieving a lightweight network.

[0049] 2) The second modification is replacing the average pooling operation in at least one transition layer with a max pooling operation. In DenseNet121, each transition layer downsamples using 2×2 average pooling after a 1×1 convolution. For example, as shown in Table 1, in the optimized network, the pooling operations in all three transition layers are replaced from average pooling to max pooling. For instance, the downsampling operation in transition layer 1 is changed from "2×2 Averagepool, Stride2" to "2×2 Maxpool, Stride2"; the same replacement is made for transition layers 2 and 3. The max pooling operation selects the maximum value within a local region as the output. Formulaically, for a 2×2 region of the input feature map... Output value for: This operation enhances the specificity of features and helps to retain more significant feature responses during feature transfer.

[0050] 3) The third modification is adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the input image in the initial stage. The initial part of the standard DenseNet121 consists of a 7×7 convolutional layer (stride 2) and a 3×3 max pooling layer (stride 2), which causes the input image to undergo two 2x downsampling steps in the initial stage, resulting in a rapid decrease in resolution. For example, as shown in Table 1, the optimized network changes the initial convolutional layer to a 3×3 convolutional layer (stride 1) and keeps the subsequent pooling layer as a 2×2 max pooling layer (stride 2). This means that for a 224×224 input, after the initial 3×3 convolution (stride 1, padding 1), the feature map size remains 224×224; after another 2×2 max pooling (stride 2), the feature map size becomes 112×112. Compared to the original network which directly downsamples to 56×56, the optimized network only performs a 2x downsampling once in the initial stage, preserving more spatial detail information of the input. It is important to clarify here that the "input image" refers to the "shallow feature map" passed from step S2, which has a size of 224×224, rather than the original remote sensing image data.

[0051] The specific implementation process of S4 is as follows: The input to S4 is the shallow feature map from step S2, and its shape is... ,in, The batch size is specified. This shallow feature map first enters the optimized initial convolutional and pooling layers. It passes through a 3×3 convolutional layer (64 input channels, 64 output channels, stride 1, padding 1) and a ReLU activation function, maintaining the same spatial size. Then, it passes through a 2×2 max-pooling layer (stride 2), increasing the output feature map size to 112×112 while retaining 64 channels. This output serves as the input to the first dense block. Subsequently, the feature map passes through four modified dense blocks and transition layers in sequence. Each dense block consists of multiple convolutional blocks connected sequentially, each containing two convolutional layers: a 1×1 bottleneck convolutional layer for dimensionality reduction and a 3×3 main convolutional layer. Each convolutional layer within a dense block receives the concatenation of feature maps from all previous layers along the channel dimension as input. After the first dense block (containing 6 convolutional blocks) and transition layer 1 (1×1 convolution + 2×2 max-pooling), the feature map size becomes 56×56, and the number of channels increases. After passing through the second dense block (containing 6 convolutional blocks) and transition layer 2, the size becomes 28×28. After passing through the third dense block (containing 12 convolutional blocks) and transition layer 3, the size becomes 14×14. Finally, the feature map passes through the fourth dense block (containing 16 convolutional blocks). There is no transition layer after this block, so the spatial size of the output feature map remains 14×14, but the number of channels increases significantly. The feature map output from the fourth dense block is the deep semantic feature map obtained in this step. This feature map has a small spatial size (14×14) but a very high channel dimension and rich semantic information. It encodes the high-level, abstract visual concepts and global context of the input image and is the core representation for the network to understand image content. This deep semantic feature map will be fed into the upsampling optimization process in step S5 for fusion and resolution restoration with the multi-scale detail features provided by the multi-scale edge supervision module.

[0052] In this context, the deep semantic feature map refers to the high-level feature representation generated after deep nonlinear transformation of multiple dense blocks and transition layers in the optimized feature extraction network. It is located at the back end of the network, far from the input layer. The key characteristics of the deep semantic feature map are its powerful semantic abstraction ability and wide receptive field, enabling it to understand high-level concepts in images such as "building clusters," "vegetation areas," and "road networks." However, its spatial resolution is relatively low, and detailed information is lost during multiple downsampling processes. In this invention, the deep semantic feature map specifically refers to the feature tensor output from the last dense block of the optimized DenseNet network, with a spatial size of 14×14. It serves as the cornerstone for semantic guidance in subsequent upsampling processes.

[0053] Dense blocks are a core component of the DenseNet network structure. A dense block is composed of multiple convolutional blocks tightly connected sequentially. Its design feature is dense connectivity; that is, within a dense block, the input to each convolutional block is a concatenation of channels from the output feature maps of all previous convolutional blocks within that block. This connectivity promotes feature reuse, alleviates the vanishing gradient problem, and achieves powerful representation capabilities with fewer parameters. A standard dense block contains multiple convolutional blocks with the same structure.

[0054] In this architecture, the convolutional block is the basic unit that makes up the dense block. In the DenseNet121 architecture, each convolutional block typically contains two consecutive convolutional layers. The first layer is a 1×1 convolution, known as the "bottleneck layer," whose main function is to reduce the number of channels in the input feature map to improve computational efficiency. The second layer is a 3×3 convolution, which is the main layer for learning spatial features. Each convolutional layer is usually followed by a batch normalization layer and a ReLU activation function. The output of the convolutional block is concatenated with its input along the channel dimension and used as one of the inputs to subsequent convolutional blocks.

[0055] S5. Input the deep semantic feature map and multiple downsampled feature maps at different scales into multiple cascaded upsampling optimization modules for feature fusion and resolution restoration to obtain multiple intermediate scale fused feature maps and a final scale fused feature map. The upsampling optimization module includes a deep-shallow feature fusion unit and a feature diversification unit connected in sequence. In the multiple cascaded upsampling optimization modules, the deep-shallow feature fusion unit of the first upsampling optimization module is used to: normalize the deep semantic feature map and the downsampled feature map of the first scale from multiple downsampled feature maps of different scales, and then concatenate and fuse them with the learned parameters after scaling. The feature diversification unit of the first upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep-shallow feature fusion unit, and output the fused feature map of the first intermediate scale. The deep-shallow feature fusion units of each upsampling optimization module from the second to the last upsampling optimization module... The function is used to: receive the fusion feature map output by the previous upsampling optimization module, and then normalize and scale it with the corresponding downsampling feature maps of multiple different scales before concatenating and fusing them; the feature diversification unit of each upsampling optimization module from the second to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the current deep and shallow feature fusion unit, and output the corresponding intermediate scale fusion feature map; wherein, the fusion feature map output by the last upsampling optimization module is determined as the final scale fusion feature map, and the fusion feature maps output by the other upsampling optimization modules are determined as multiple intermediate scale fusion feature maps.

[0056] The specific implementation process of S5 is as follows: The S50 upsampling optimization module consists of two sequentially connected sub-units: a deep and shallow feature fusion unit responsible for receiving two types of input and performing standardization and fusion. This unit does not contain traditional convolutional or fully connected layers; its core operations are feature normalization and learnable parameter scaling, followed by feature concatenation. Internally, the unit maintains a set of learnable one-dimensional parameter vectors. Its length is related to the number of channels of the input features. Equal, that is This unit functions algorithmically, rather than being a stack of multiple network layers. The Feature Diversification Unit is a convolutional module with a complex branching structure. Its design includes four parallel processing branches, each consisting of a series of convolutional layers with specific kernel sizes, designed to capture diverse contextual information through receptive fields of varying sizes. The final outputs of all branches are concatenated along the channel dimension and then fused and compressed through a final 1×1 convolutional layer. This unit contains a large number of trainable parameters and is the core of the feature transformation performed by its upsampling optimization module.

[0057] S51, The deep and shallow feature fusion unit of the first upsampling optimization module receives two inputs: deep semantic feature map. (shape assumed to be) (and multiple downsampled feature maps at different scales, the first scale downsampled feature map) (Based on the context, this is an 8x downsampled feature, with the shape being...) Since the two have different spatial dimensions, it is first necessary to analyze the deep semantic feature maps. An upsampling operation is performed, using bilinear interpolation to enlarge the spatial size from 14×14 to 28×28, resulting in the upsampled deep features. Next, the units respectively... and The same processing flow applies. For any input feature... (Its shape is) , here For deep features For shallow features The calculation process follows formula (2): ,in, This is a shallow feature map or a deep semantic feature map, with a size of [size missing]. , For the number of channels, This refers to the feature size. These are the normalized and scaled features. For a defined set of channels The corresponding learnable parameters, i.e. , In practice, this formula should be understood as being applied per channel. For feature maps... The Feature map of each channel Its processed output for: ,in, It is the first The spatial location of each channel The value, Indicates the first The entire two-dimensional feature matrix of each channel. It corresponds to the first The learnable scaling parameters for each channel belong to the learnable parameter vector. Part of it. and It refers to the spatial height and width of the feature map. It is a very small constant used to prevent the denominator from being zero. In the original context of formula (2), it represents the total number of eigenvector elements. In this specific application, This is equivalent to summing the squares of all spatial positions in the current channel, i.e. . It is the normalized and scaled first... Each channel feature map.

[0058] To each and After performing the above calculations on each channel, the processed deep features are obtained. and shallow features The final operation of the deep and shallow feature fusion unit is to... and The data is stitched together along the channel dimension to obtain the fusion result. ,Right now This fusion result It will be used as input to the feature diversification unit in the first upsampling optimization module.

[0059] S52. The feature diversification unit receives the fusion result from the deep and shallow feature fusion unit. As input. Assume The shape is ,in This unit processes the input features in parallel through four branches, and then fuses the results. Specifically, 1) the first branch: processes the input... A 2×2 average pooling operation is performed, with padding used to maintain the output space size at 28×28. A 1×1 convolutional layer is then used to reduce the number of channels. 2) Second branch: directly address the input Use a 1×1 convolutional layer to compress the number of channels to 3) The third branch: First, use a 1×1 convolution to reduce the number of channels to... Then, it passes through two convolutional layers with kernel sizes of 1×3 and 3×1, respectively. Next, it passes through a convolutional layer with a kernel size of 1×3 to halve the number of channels. Finally, it passes through a convolutional layer with a kernel size of 3×1. 4) Fourth branch: First, a 1×1 convolution is used to reduce the number of channels to... Then, it passes through two convolutional layers with kernel sizes of 1×7 and 7×1, respectively. Next, it passes through a convolutional layer with a kernel size of 1×7 to halve the number of channels. Finally, it passes through a convolutional layer with a kernel size of 7×1.

[0060] Each convolutional layer is typically followed by a batch normalization layer and a ReLU activation function. The output feature maps of the four branches are concatenated along the channel dimension to obtain a channel-wise concatenation layer. The feature maps are then processed. Finally, a 1×1 convolutional layer is used to fuse these concatenated features and compress the number of channels. The output of this 1×1 convolutional layer is the first intermediate-scale fused feature map output by the first upsampling optimization module. Its spatial dimensions are 28×28, and the number of channels is... .

[0061] S53. The deep and shallow feature fusion units of the second and subsequent upsampling optimization modules have a similar workflow to the first module, but the input sources are different. Taking the second upsampling optimization module as an example, its deep and shallow feature fusion unit receives two inputs: the fused feature map at the first intermediate scale output from the previous module (i.e., the first module). (shape is) This includes downsampled feature maps at multiple different scales, and corresponding downsampled feature maps at different scales. According to the cascade order, the second module should correspond to a 4x downsampled feature map. (shape is) ).because The space dimensions (28×28) are less than (56×56), it is necessary to first... Upsample to 56×56 (e.g., using bilinear interpolation) to obtain Then, and Each feature map is fed into the same processing flow as in the second step: that is, for each channel of each feature map, the normalization and learnable parameter scaling operations defined by formula (2) are applied. The aisle: ,for The aisle: ,in, , and These are elements in the independent learnable parameter vectors of the deep and shallow feature fusion units in the second module. The processed features are obtained. and Then, the two are concatenated along the channel dimension to obtain the fusion result of the second module. The third module follows the same processing method, with its shallow input corresponding to a 2x downsampled feature map. (shape is) The feature map output by the second module is upsampled and then fused with it.

[0062] S54. For the second and subsequent upsampling optimization modules, the operation of their feature diversification units is completely consistent with that of the feature diversification unit in the first module, having the same four-branch structure. This unit receives the fusion result output by the deep / shallow feature fusion unit of its respective module (e.g., the second module's...). The input is taken as input. After parallel processing through four branches and final 1×1 convolutional fusion compression, the output is a fused feature map corresponding to the intermediate scale. For example, the second upsampling optimization module outputs a fused feature map of the second intermediate scale. Its spatial dimensions are 56×56; the third module outputs the fused feature map at the third intermediate scale. Its spatial dimensions are 112×112. After the last cascaded upsampling optimization module (the third module in this example) completes processing, its output fused feature map... This refers to the fused feature map that is determined to be the final scale. Prior to this, the module output... and These are then identified as fused feature maps at multiple intermediate scales. These fused feature maps at different scales will be used for subsequent deep supervision and final prediction.

[0063] The final-scale fused feature map refers to the fused feature map output by the last upsampling optimization module after processing by multiple cascaded upsampling optimization modules. It represents the highest resolution and richest feature representation generated by the network after completing the fusion of all shallow and deep features and the integration of multi-scale information. Its spatial size is consistent with the largest-scale shallow feature input to the upsampling process (usually a 2x downsampling feature), for example, 112×112. This feature map will be directly used to generate the network's final semantic segmentation prediction map.

[0064] Among the multiple downsampled feature maps at different scales, the first-scale downsampled feature map refers to the first feature map in the sequence of downsampled feature maps generated by the multi-scale edge supervision module that is sent to the upsampling optimization process for processing. This "first scale" corresponds to the feature with the largest downsampling factor, i.e., the 8x downsampled feature map, which has the smallest spatial size (e.g., 28×28). It is the shallow detail feature that is first fused with the deep semantic feature map.

[0065] Learnable parameters refer to variables whose values ​​are automatically adjusted and optimized during model training through backpropagation and optimizers. In the deep and shallow feature fusion unit, this refers to the vector in formula (2). Each It is a scalar specifically used to scale the corresponding channel. The parameters are normalized feature values. These parameters do not have a fixed initial value range. The model learns to assign an appropriate scaling factor to each channel through training data to optimize the representation of features.

[0066] The fusion result refers to the new feature tensor generated by concatenating the two processed feature tensors along the channel dimension after normalization and learnable parameter scaling operations on the two types of input features (such as the output features of the previous module and the shallow features at the current scale) in the deep and shallow feature fusion unit of the upsampling optimization module. It is the output of the deep and shallow feature fusion unit and serves as the input to the feature diversification unit. For example, the fusion result of the first module... .

[0067] In this context, the fused feature map refers to the feature map output after complete processing by the upsampling optimization module (i.e., sequentially passing through the deep and shallow feature fusion unit and the feature diversification unit). It includes both intermediate-scale fused feature maps and final-scale fused feature maps. The fused feature map is the product of carefully designed module fusion and transformation of deep semantic information and shallow detail information, and it serves as the direct basis for subsequent prediction steps.

[0068] S6. Based on the fused feature maps at multiple intermediate scales, upsampling and classification predictions are performed respectively to obtain multiple intermediate prediction results. The specific implementation process is as follows: S60. Based on the description and cascade design in step S5, assume that there exists A fusion feature map at an intermediate scale. For example, in an instance with three upsampling optimization modules, These fused feature maps are denoted as follows: Each fused feature map has a different spatial size. and number of channels , where index .For example, The size can be , The size can be , The size can be . This indicates the batch size. These feature maps are direct inputs to the processing in step S6.

[0069] S61, For each fused feature map A separate 1×1 convolutional layer is used as the classifier. The function of this 1×1 convolutional layer is to increase the number of channels in the feature map. Total number of maps to target semantic categories Assume the number of semantic categories in the dataset is... (For example, in the Potsdam dataset) Then the weight shape of this 1×1 convolutional layer is: For the first The operation of applying a fused feature map to its classification prediction layer can be represented as: ,in, This indicates a convolution operation (here, a 1×1 convolution). and It is the first The learnable weights and bias parameters of each classification prediction layer. It is the output after the convolution operation, and its shape is This tensor Includes for each spatial location The raw scores (logits) of each category.

[0070] S62, due to Space dimensions If the output size is smaller than the network's desired final output size (i.e., the original input image size, e.g., 224×224), each image needs to be... Upsampling is performed to a uniform full resolution. The upsampling factor is determined by the original scale of the feature map. For example, for a 28×28 feature map... It needs to be upsampled 8 times to 224×224; for a size of 56×56 It requires upsampling by 4 times; for a size of 112×112 This requires upsampling by a factor of 2. Upsampling typically employs bilinear interpolation, a deterministic, differentiable spatial transformation method. After upsampling, a set of dimensions is obtained. The tensor, denoted as .

[0071] S63. To transform the original scores into probabilistic form and calculate the loss for each intermediate prediction, it is necessary to perform a process on each upsampled score tensor. Apply the Softmax function. The Softmax function operates along the channel dimension (i.e., the category dimension). Perform the operation. For the first item in the batch... A sample, in spatial location At this location, the prediction is the first. The probability of each category The calculation formula is: In the formula, It is a tensor The Middle The sample, the first Each channel, in spatial location The value of . It is the corresponding predicted probability, satisfying and After calculating for each position, we obtain the first position. Probability plot of intermediate predictions Its shape is .

[0072] S64. After the above four steps, for The fusion feature maps at intermediate scales generated The corresponding semantic segmentation probability map .this A set of probability graphs is defined as "multiple intermediate predictions". Each intermediate prediction... These all represent the network's complete pixel-level semantic understanding of the input image at a specific feature scale (resolution). These intermediate predictions will be fed into step S8, where they will be compared with the true semantic segmentation labels to calculate the various classification losses required to constitute the deep supervised loss.

[0073] The multiple intermediate prediction results refer to the set of semantic segmentation prediction probability maps generated from fused feature map branches at different intermediate scales during network training, all with the same size as the final output image. Each intermediate prediction result originates from feature representations at different depths or resolutions of the network, providing a multi-level preview of semantic segmentation. These results typically correspond to predictions output in the early or mid-stages of the upsampling optimization process. They are not directly used as the final inference result, but rather to construct a segmented weighted deep supervision mechanism. That is, during training, direct gradient signals are provided to these intermediate layers of the network to alleviate the gradient vanishing problem, accelerate convergence, and improve the representation quality of intermediate features, thereby indirectly optimizing the final segmentation performance.

[0074] S7. Calculate the edge supervision loss by combining the edge prediction results with the preset edge labels, and train and optimize the multi-scale edge supervision module. The specific implementation process is as follows: S70. During the forward propagation of the model, after the input data passes through steps S2 and S3, the edge prediction result is obtained from the output of the first branch of the multi-scale edge supervision module. This edge prediction result is a shape... The probability tensor is denoted as . ,in It refers to the batch size. and This refers to the height and width of the feature map (e.g., 224). Each element in the tensor... Indicates the first in the batch The spatial location of each sample The probability of being predicted as a boundary point. Simultaneously, the preset edge label corresponding to the current batch is retrieved from the data loader. This preset edge label is a shape that is also... The binary tensor, denoted as Each element in the tensor The value can be 0 or 1, where 1 indicates that the location is a real feature boundary and 0 indicates that it is not a boundary. These two sets of data are the basis for calculating the loss.

[0075] S71. The edge supervision loss is calculated using the binary cross-entropy loss function defined in formula (1). This function measures the difference in probability distribution between the edge prediction result and the preset edge label pixel by pixel. Formula (1) is as follows: , is the cross-entropy loss for binary classification, where N is the number of samples in a batch. It is the actual label of the nth sample, with a value of 0 or 1. It is the probability that the prediction of the nth sample is correct, and its value is between (0, 1).

[0076] Optionally, in actual calculations, batch and spatial dimensions are typically combined. Assume the total number of pixels is... Then the loss function can be operated as follows: ,in, The calculated edge supervision loss value is a scalar. It is the total number of pixels of all samples in a batch. It is the first The actual edge label value corresponding to each pixel is either 0 or 1. The model is for the first The probability of a pixel being predicted as an edge is between 0 and 1. The summation operation iterates through all samples and all spatial locations in the batch. The smaller the value of this loss function, the closer the edge prediction result is to the preset edge label.

[0077] S72. Calculate the scalar loss value. Then, the backpropagation algorithm is initiated. Specifically, ① using an automatic differentiation framework, it calculates the loss... To begin, the gradient of this loss is computed in reverse with respect to all learnable parameters within the multi-scale edge supervision module. These parameters primarily include the weights of the 3×3 convolutional layer used to obtain the first feature, as described in step S3. and bias And the weights and biases of the 1×1 classification convolutional layer in the first branch. ② Using a pre-configured optimizer (e.g., stochastic gradient descent SGD or Adam), the above parameters of the multi-scale edge supervision module are iteratively updated based on the calculated gradient and the preset learning rate. In the formula, for a parameter... Its update rule can be expressed as ,in It's the learning rate. This is the gradient of the loss with respect to that parameter. By repeatedly performing this process of forward calculation of loss, backpropagation of gradient, and parameter update, the multi-scale edge supervision module is trained and optimized, making the edge prediction results output by its first branch increasingly match the preset edge labels, thereby forcing the module to extract and transmit more discriminative edge feature information.

[0078] The pre-defined edge labels are auxiliary supervision signals generated by preprocessing the real semantic segmentation label images in the training dataset using a specific algorithm before model training begins. They are binary images where white pixels (value 1) mark the boundaries between regions of different semantic categories, i.e., feature boundaries; black pixels (value 0) represent the interior of a region or the background. The generation method involves converting polygon segmentation labels into vector surfaces, extracting the common boundary lines between surfaces, and finally rasterizing the vector boundary lines back into a binary image with the same dimensions as the original image. These pre-defined edge labels provide a clear learning objective for the multi-scale edge supervision module, guiding the network to focus on and strengthen the modeling of object contours.

[0079] S8. Calculate the classification loss for multiple intermediate prediction results and semantic segmentation labels respectively, and sum the classification losses according to the training stage to obtain the deep supervision loss. Train and optimize the upsampling optimization module and feature extraction network based on the deep supervision loss. The weighted summation of the classification loss based on the training phase includes: S80. Divide the training process into multiple training phases. Specifically, before training begins, determine the total number of training rounds based on a preset total number of training rounds. The entire training process is divided into three consecutive training phases: early, middle, and late. This division can be based on a fixed proportion of training epochs. For example, a division proportion vector can be defined. ,satisfy So, the early stage includes rounds 1 to 2. Round; the intermediate stage includes the subsequent rounds. It's the turn of the first Round; the later stages include the remaining rounds up to the [number missing]. Round. Within each training phase, the weight coefficients in step S8... and Keep it fixed.

[0080] Multiple training phases refer to the artificial division of the entire model training process into several consecutive periods with different training strategies, based on training epochs. These are typically divided into three phases: early, middle, and late. Each training phase can be configured with different hyperparameters, such as the learning rate and the weight coefficients of the loss function. The purpose of dividing the training into phases is to simulate a coarse-to-fine learning process: the early phase may focus more on fitting basic features and model stability; the middle phase aims to balance different optimization objectives; and the late phase focuses on optimizing details and refining boundaries. This phased training strategy helps improve the final performance of the model and accelerates convergence.

[0081] S81. In each training phase, different weight coefficients are assigned to different types of classification loss functions. Specifically, this method uses two classification loss functions: multi-class cross-entropy loss. and Dice loss For each training phase, a pair of weight coefficients needs to be predefined. ,in It is Dice's loss The weight, It is a multi-class cross-entropy loss The weights. For example, in the early stages, the weights might be set. To emphasize cross-entropy loss; in the mid-stage, it may be set to To balance the two types of losses; in later stages, it may be set to We focus more on Dice loss to optimize the segmentation boundary. These weight coefficients are hyperparameters that are set before training and remain unchanged during the corresponding phases.

[0082] S82. The calculation results of different types of loss functions corresponding to each intermediate prediction result are weighted and summed according to the weight coefficients of their respective training stages. This sum constitutes the contribution of the intermediate prediction result to the deep supervision loss. Specifically, assume that there are... Intermediate prediction results and the corresponding semantic segmentation tags For the first Intermediate prediction results First, calculate the two types of losses separately: multi-class cross-entropy loss. and Dice loss The calculation process uses formulas (4) and (5). Then, based on the training stage to which the current training round belongs, the weight coefficients corresponding to that stage are obtained. and Next, calculate the first... Weighted loss contribution of intermediate prediction results : ,this This refers to the contribution of the intermediate prediction result to the deep supervision loss. For all... Repeat this process for each intermediate prediction result to obtain... Each contribution section Formula (4) is as follows: ,in, For multi-class cross-entropy loss, For the number of categories, For the first in a batch In the nth sample The actual label of the class is one-hot encoded. The model predicts that it is the first The probability of a class. Formula (5) is: ,in, The total number of pixels in the image. The total number of categories, In the first Predicted as category on each pixel The probability, In the first The pixel at the pixel The actual label of the class (still one-hot encoded). To smooth out terms and prevent the denominator from being 0.

[0083] The upsampling optimization module and the feature extraction network are trained and optimized based on deep supervised loss. The specific implementation process is as follows: S83, Calculate the total depth supervision loss The total depth-supervised loss is the sum of the contributions from all intermediate predictions in step S6. This is applied to the four outputs of the upsampling optimization process (assuming...). The losses are added together. The specific formula is: ,in, This is the deep supervision loss scalar that is ultimately used for backpropagation.

[0084] S84. Using an automatic differentiation framework, supervise the total depth loss. The process begins by back-calculating the gradient of the loss with respect to all learnable parameters in the network. The key point is that this loss calculation path simultaneously flows through the upsampling optimization module and the feature extraction network. Specifically, the gradient is traced back through the classification prediction layer and upsampling operation in step S6 to each learnable parameter of the upsampling optimization module in step S5 (including the feature diversification unit and the deep / shallow feature fusion unit within it). This process then traces back to the optimized feature extraction network in step S4 (i.e., all convolutional layers, batch normalization layers, and other parameters of the modified DenseNet121). The parameters of the multi-scale edge supervision module do not directly receive gradients from this loss; they are optimized separately by the edge supervision loss in step S7.

[0085] S85. Using a preset optimizer (such as stochastic gradient descent SGD or Adam), perform an iterative update on all parameters in the upsampling optimization module and the feature extraction network based on the calculated gradient. For a single parameter... Its update rule can be expressed as ,in It is the learner rate of the optimizer. This is the gradient of the deep supervised loss with respect to this parameter. By repeatedly performing forward propagation, loss calculation, backpropagation, and parameter update cycles on the entire training set, the parameters of the upsampling optimization module and the feature extraction network are continuously optimized, thereby making the multiple intermediate predictions generated by the network and the final semantic segmentation result increasingly accurate.

[0086] The contribution of intermediate predictions to the deep supervision loss refers to the loss component generated by a specific intermediate prediction when calculating the total deep supervision loss. It is a scalar value obtained by weighting and summing the intermediate prediction with the different types of classification losses (such as multi-class cross-entropy loss and Dice loss) calculated from the ground semantic segmentation labels, according to the weight coefficients preset for the current training stage. Each intermediate prediction produces such a contribution. The sum of the contributions from all intermediate predictions constitutes the total deep supervision loss used for backpropagation. This concept emphasizes that in the deep supervision mechanism, each intermediate output layer is directly responsible for the final training objective, and its prediction error independently contributes to the total loss, thus providing more direct and richer supervision signals for feature learning at the network layers.

[0087] S9. Using the trained and optimized multi-scale edge supervision module, feature extraction network, and multiple upsampling optimization modules, the semantic segmentation result of the remote sensing image data to be processed is generated based on the fused feature map of the final scale. The specific implementation process is as follows: S90. Before processing new remote sensing image data, the network parameters obtained from training and optimization in steps S7 and S8 need to be fully loaded. This includes the weights and biases of the 3×3 convolutional layers and 1×1 classification convolutional layers in the multi-scale edge supervision module; the parameters of all convolutional layers and batch normalization layers in the optimized feature extraction network (i.e., the modified DenseNet121); and the learnable parameter vector γ of the deep and shallow feature fusion units and the weights and biases of all convolutional layers in the feature diversification units in all upsampling optimization modules. Load all these parameters into the instances of the corresponding network layers and set the entire model to evaluation mode. In evaluation mode, the batch normalization layers in the network use running statistics instead of batch statistics, and the Dropout layer (if present) is also disabled to ensure determinism and consistency in the inference process.

[0088] S91. This step is basically the same as the data preprocessing in step S1 of the training phase, but only includes necessary standardization and resizing operations, without data augmentation. Specifically, the following steps are performed: First, read the input remote sensing image data and convert it from the original image format (such as GeoTIFF, PNG) to a floating-point tensor. Next, normalize the image pixel values ​​according to the mean and standard deviation of the training time sample. Then, crop or scale the image to the input size specified by the network, for example, 224×224 pixels. If the input image size is larger than 224×224, it can be divided into multiple 224×224 tiles for separate processing, or scaled down to 224×224 as a whole. The processed data forms a tensor of shape (1, C, 224, 224), where C is the number of image channels, and the first dimension 1 indicates a batch size of 1.

[0089] S92. Input the preprocessed image tensor into the shallow feature extraction convolutional layer defined in step S2. This layer performs 3×3 convolution and ReLU activation operations, and outputs a shallow feature map with shape (1, 64, 224, 224).

[0090] S93. Input the shallow feature map into the trained multi-scale edge supervision module. This module works according to the process in step S3: First, it obtains the first feature through a 3×3 convolutional layer; then, the first feature enters two branches. The first branch (edge ​​prediction branch) outputs the edge prediction result through a 1×1 convolutional layer and the Softmax function, but this result is usually not used or saved during the inference stage unless additional edge analysis is required. The second branch (multi-scale downsampling branch) downsamples the first feature by 2x, 4x, and 8x, generating multiple downsampled feature maps at different scales. These feature maps are key information required for subsequent steps.

[0091] S94. Input the same shallow feature map into the trained and optimized feature extraction network. The network performs forward propagation according to the DenseNet structure modified in step S4, and after initial convolution, pooling, four dense blocks and transition layers, it finally outputs a deep semantic feature map with shape (1, C_d, 14, 14), where C_d is the number of channels of the deep feature.

[0092] S95. The deep semantic feature map and multiple downsampled feature maps of different scales generated by the multi-scale edge supervision module are sequentially input into the cascaded, trained upsampling optimization module. As described in step S5, each module sequentially performs deep and shallow feature fusion and feature diversification operations. This process gradually restores the resolution of the feature map and fuses multi-scale information. Finally, the last upsampling optimization module outputs the final scale fused feature map, whose shape is assumed to be (1, C_f, 112, 112), where C_f is the number of channels in the feature map.

[0093] S95. This is the crucial step in obtaining the final output. The first sub-step is upsampling: the fused feature map at the final scale is upsampled by a factor of 2 using bilinear interpolation, changing its spatial size from 112×112 to 224×224, consistent with the network input size. The second sub-step is classification prediction: a 1×1 convolutional layer (which is the classifier provided for the final scale branch in step S6 during training) is used to process the upsampled feature map. This 1×1 convolutional layer maps the number of channels in the feature map from C_f to the total number of semantic categories C. Its operation can be represented as: ,in, It is the final scale fusion feature map after upsampling, with a shape of (1,C_f,224,224). This represents a 1×1 convolution operation. and These are the trained weights and bias parameters of the classification prediction layer, with shapes (C,C_f,1,1) and (C,), respectively. This is the output score map, with a shape of (1, C, 224, 224). The third sub-step is to apply the Softmax function: scalarize the score map along the channel dimension (i.e., the category dimension C). The Softmax function is applied to convert the raw scores into class probabilities. For a pixel at spatial location (i,j), the probability that it belongs to class c is... for: ,in, It is a score chart The value at position (i,j) in channel c of category. The fourth sub-step is to perform the Argmax operation: for each pixel position (i,j), select the category index with the highest probability value as the predicted category label for that pixel. Mathematically, the predicted category label map... Each element is calculated as follows: This is how it was obtained It is a two-dimensional integer matrix of shape (224, 224), which is the semantic segmentation result, where each pixel value represents the predicted category ID.

[0094] S97. Since the input image may be cropped or scaled during preprocessing, the generated 224×224 semantic segmentation result needs to be mapped back to the geometric space of the original input image. If scaling occurred during input, the prediction map needs to be scaled to its original size; if block processing was performed during input, the prediction results of each block need to be stitched together. Finally, the processed semantic segmentation result is converted into a standard image format (such as PNG) or geospatial data format (such as GeoTIFF) for storage or visualization, thereby completing the semantic segmentation task of the entire remote sensing image data to be processed.

[0095] The semantic segmentation result is the final output of the method in this invention. It is a labeled map that corresponds perfectly to the spatial size of the input image, with each pixel assigned a discrete category label. It visually displays the semantic category of each land feature in the input remote sensing image, such as buildings, roads, vegetation, and water bodies. This result exists in the form of two-dimensional raster data, typically saved as a single-channel image file, where different integer values ​​or colors represent different semantic categories. The semantic segmentation result is fundamental data for applications such as quantitative analysis of remote sensing images, geographic information updates, resource surveys, and environmental monitoring. In this invention, the semantic segmentation result is obtained from the final-scale fused feature map after upsampling, classification prediction, and Argmax operations, reflecting the network's comprehensive and refined semantic understanding of the input image.

[0096] In this invention, ① an improved DenseNet feature extractor (i.e., an optimized feature extraction network) is proposed. This module reduces feature size decay during the feature extraction process while maintaining the network's feature extraction capabilities, thus preserving more detailed feature information; at the same time, the number of network layers in DenseNet121 is modified to make the network more lightweight. Specifically, the optimized feature extraction network is implemented by modifying the standard DenseNet121 structure. The modifications include: reducing the number of convolutional blocks contained in at least one dense block in the network to reduce parameter redundancy and computational burden; replacing the average pooling operation used for downsampling in at least one transition layer with a max pooling operation to enhance the specificity of the feature response; and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the input image in the initial stage, thereby preserving higher resolution spatial details at earlier layers. These modifications work together to enable the network to better balance the abstract semantic expression capability and spatial detail preservation capability of features when extracting deep semantic feature maps. ② An upsampling optimization module and a multi-scale edge supervision module are constructed. The upsampling optimization module is mainly used to fuse deep and shallow features and restore the resolution of deep features layer by layer, achieving efficient utilization and modeling of the extracted abstract features. This module internally includes a deep / shallow feature fusion unit and a feature diversification unit. The deep / shallow feature fusion unit receives input from deep semantic feature maps and shallow downsampled feature maps, and normalizes them based on the channel L2 norm and learnable parameter vectors. The scaling operation calibrates the features, and then the processed features are concatenated along the channel dimension to form a fusion result. The feature diversification unit extracts and compresses diverse features from the fusion result through multiple parallel branches with different convolutional kernel sizes, outputting a higher-resolution fusion feature map. Multiple such modules are cascaded to gradually restore the spatial size of the feature map. The multi-scale edge supervision module implements two functions through a simple architecture: extracting shallow multi-scale features and edge features, and coupling this information to the upsampling optimization module. This module first convolves the input shallow feature map to obtain the first feature; then, the first feature is fed into two branches. The first branch generates edge prediction results through convolution and classification operations, which are supervised by preset edge labels during training. The second branch performs downsampling operations on the first feature at least two different magnifications (e.g., 2x, 4x, 8x) to generate multiple downsampled feature maps of different scales. These downsampled feature maps are directly input into the upsampling optimization modules at different stages, achieving the provision of clear multi-scale spatial context information and edge prior information for the subsequent feature fusion and restoration process without significantly increasing computational overhead. ③ Construct a segmented weighted depth supervision mechanism for the upsampling process and an edge supervision mechanism for the multi-scale edge supervision module. The segmented weighted depth supervision mechanism mainly supervises the multiple intermediate prediction results corresponding to the fused feature maps of multiple intermediate scales output by the upsampling optimization module to achieve better training results. This mechanism uses two loss functions: multi-class cross-entropy loss. With Dice loss The training process is divided into several phases (e.g., early, middle, and late stages), with each phase consisting of a weighted sum of the two losses. and Assign different weight coefficients and For each intermediate prediction result, calculate its... and Then, the values ​​are weighted and summed according to the weights of the current stage to obtain the contribution of the prediction result to the total depth supervision loss. The sum of all contributions constitutes the total depth supervision loss. This is used to simultaneously optimize the parameters of the upsampling optimization module and the feature extraction network. The edge supervision mechanism mainly supervises the edge prediction results output by the multi-scale edge supervision module to obtain better edge detail features, and adopts the binary classification cross-entropy loss function defined by formula (1). The loss is calculated and specifically used to optimize the parameters of the multi-scale edge supervision module itself. The specific implementation steps are as follows: S101. Obtain the Potsdam and Vaihingen high-resolution remote sensing datasets and preprocess the data, including cropping, data augmentation, and data partitioning.

[0097] S102. Construct a multi-scale edge supervision module to enable the network to extract and optimize multi-scale and shallow edge features of images. S103. Optimize the DenseNet121 feature extraction network by reducing the resolution decay of each layer, optimizing the number of convolutional layers, and modifying the pooling type, while ensuring that the extraction capability of the optimized network is not reduced.

[0098] S104. Construct an upsampling optimization process and module to achieve efficient utilization of deep semantic features and a refined feature resolution recovery process.

[0099] S105. Design a segmented weighted depth supervision mechanism to supervise the upsampling process and optimize the learning process and training objectives. Simultaneously, design an edge supervision mechanism to ensure the accuracy of extracted shallow feature edge information.

[0100] In step S101, to reduce the computational burden during training, the dataset is cropped to 256×256 pixels, with the training set cropping overlap set to 0.15 and the test set cropping overlap set to 0. Simultaneously, the segmented training set labels are converted from raster to vector plane and then from vector plane to line to obtain the label boundary information, providing preset edge labels for the subsequent multi-scale edge supervision module. To increase the number of samples and enhance the robustness of network training, data augmentation is performed on the segmented training data (including preset edge labels), including horizontal flipping, vertical flipping, 90-degree rotation, and 270-degree rotation. Furthermore, before each training round, the 256×256 training data needs to be randomly cropped to 224×224 pixels to improve the diversity of the training data.

[0101] In step S102, a multi-scale edge supervision module is constructed to enhance the network's attention to edge information during the upsampling process. The specific construction steps are as follows: A 3×3 convolution is performed on the input image to obtain 64×64 channel features, and these features are then input into two branches. Specifically: 1) In the first branch, the features are convolved with 1×1 and the binary classification result is calculated. This result is then used for edge supervision along with the preset edge labels. Specifically, the binary cross-entropy loss is calculated and the features are backpropagated to optimize the image edges. The formula for calculating the binary cross-entropy loss is shown in equation (1): ,in, For binary classification, cross-entropy loss, This represents the number of samples in a batch. It is the first The actual label of each sample is either 0 or 1. It is the first The probability of a sample being correctly predicted takes a value between (0, 1).

[0102] 2) In the second branch, the features are downsampled by 2, 4 and 8 times and input into the upsampling optimization process for subsequent calculations. At the same time, the unsampled features are input into the optimized feature extraction network for abstract deep semantic feature map extraction.

[0103] In step S3, the DenseNet121 network was optimized to ensure the lightweight and efficient feature extraction network. The specific optimization steps are as follows: the number of convolutional blocks in each dense block of the original DenseNet121 was changed from 6, 12, 24, 16 to 6, 6, 12, 16, reducing feature redundancy and better coupling subsequent modules; feature size attenuation during feature extraction was reduced to preserve feature detail information and upsampling accuracy as much as possible; max pooling was used instead of average pooling to ensure feature specificity. Specific optimization data are shown in Table 1.

[0104] Table 1: In step S4, an upsampling optimization process and an upsampling optimization module were designed to efficiently utilize deep semantic feature maps to achieve refined feature resolution recovery. The specific construction steps are as follows: (1) The features output by the feature extraction network are input into two branches. One branch upsamples the features by 16 times and performs multi-classification, and inputs them into the segmented weighted deep supervision mechanism to calculate the loss. In the other branch, the 1414-size features and multiple downsampled feature maps of different scales (28×28, 56×56, and 112×112) output by the multi-scale edge supervision module are input into different stages of the upsampling optimization module. After multiple upsampling optimization modules, fused feature maps of 28×28, 56×56, and 112×112 are output three times, upsampled by 8 times, 4 times, and 2 times respectively, and then multi-classified. All of these are input into the segmented weighted deep supervision mechanism to calculate the loss. At the same time, the result obtained by upsampling the 112×112-size features by 2 times and classifying them is used as the final semantic segmentation result. (2) The input features are first fed into the deep and shallow feature fusion unit, upsampled to the same size, and the L2 norm values ​​of the two are calculated respectively. Then, the feature values ​​of each channel after normalization are multiplied by a learnable parameter, which allows the network to automatically adjust the feature values ​​to a level conducive to model optimization. The calculation process is shown in formula (2): ,in, This is a shallow feature map or a deep semantic feature map, with a size of [size missing]. , For the number of channels, This refers to the feature size. These are the normalized and scaled features. For a defined set of channels The corresponding learnable parameters, i.e. Each parameter is specifically used to scale the corresponding channel. eigenvalues. This is a very small constant to prevent the feature value from being zero. Finally, the calculated light and dark features are concatenated, and a 1×1 convolution is used to fuse the concatenated results before inputting them into the feature diversification unit.

[0105] The feature diversification unit is divided into four branches. The first branch first performs a 2×2 average pooling operation on the input features, maintaining the feature size by padding, and then uses a 1×1 convolution to reduce the number of channels to 1 / 4 of the original number of input channels. The second branch directly integrates the information of the input features through a 1×1 convolution and compresses the number of channels to 1 / 4 of the original number of channels. The third branch first reduces the number of channels to 1 / 2 of the original number of channels through a 1×1 convolution, then applies convolution operations with kernel sizes of 1×3 and 3×1 in sequence, and then performs a convolution operation with a kernel size of 1×3 to further halve the number of channels to 1 / 4 of the initial number of channels, and finally performs a convolution operation with a kernel size of 3×1. The fourth branch first uses a 1×1 convolution to reduce the number of channels to 1 / 2 of the original number of channels, then performs convolution operations with kernel sizes of 1×7 and 7×1 in sequence, then performs a convolution operation with a kernel size of 1×7 to halve the number of channels to 1 / 4 of the original number of channels, and finally applies a convolution operation with a kernel size of 7×1. Finally, the features output from the four branches are concatenated and multi-path information is fused through 1×1 convolution, while the number of channels is compressed to half of the number of input feature channels. This result will be used as the input for the next stage upsampling optimization module.

[0106] In step S5, a segmented weighted deep supervision mechanism was designed to supervise the upsampling process and optimize the learning process and training objectives. The specific steps are as follows: input the fused feature maps of multiple intermediate scales output in the upsampling optimization process into the weighted deep supervision module, calculate the Dice loss and multi-class cross-entropy loss with the semantic segmentation labels respectively, and add them in weight, as shown in formula (3): ,in, To deeply monitor losses, and These are the weights for Dice loss and multi-class cross-entropy loss, respectively. The specific weight settings are as follows: the network training is divided into three training stages: early, middle, and late. The weights for each training stage can be set as needed. Dice loss for output features of different layers, The multi-class cross-entropy loss is calculated for the output features of different layers. The formula for calculating the multi-class cross-entropy loss is shown in equation (4): ,in, For multi-class cross-entropy loss, For the number of categories, For the first in a batch In the nth sample The actual label of the class is one-hot encoded. The model predicts that it is the first The probability of a class.

[0107] The formula for calculating Dice loss is shown in formula (5): ,in, The total number of pixels in the image. The total number of categories, In the first Predicted as category on each pixel The probability, In the first The pixel at the pixel The actual label of the class (still one-hot encoded). To smooth out terms and prevent the denominator from being 0.

[0108] Figure 2 This paper demonstrates the overall framework and data flow process of the high-resolution remote sensing image semantic segmentation upsampling optimization method with multi-scale edge constraints of the present invention. The method begins with inputting a remote sensing image data to be processed, which first undergoes shallow feature extraction through a Conv3×3 convolutional layer to obtain a shallow feature map with a spatial size of 224×224. This shallow feature map is then input in parallel to two core modules: a multi-scale edge supervision module and an optimized feature extraction network (i.e.,...). Figure 2 (A simplified version of DenseNet). In the multi-scale edge supervision module, the shallow feature maps are further processed to generate multi-scale and edge information. Internally, this module first transforms the features and then inputs them into two branches: the first branch outputs a boundary prediction map through convolution and classification operations. This prediction map, along with the boundary information independently computed by this module (derived from the processing of training labels), is used for edge supervision during training by calculating the binary cross-entropy loss. The first branch optimizes the module parameters; the second branch performs parallel downsampling operations of 2x, 4x, and 8x on the features, generating three sets of downsampled feature maps at different scales: Features112×112, Features56×56, and Features28×28. Simultaneously, the shallow feature map is input into the optimized feature extraction network, which performs deep semantic feature extraction through a series of modified dense blocks and transition layers, ultimately outputting a deep semantic feature map with a spatial size of 14×14. Figure 2 The deep semantic feature map (14×14) is labeled as Features14×14. Subsequently, the network process splits into two paths: the first path directly upsamples the deep semantic feature map by a factor of 16 to a size of 224×224 and performs classification prediction; the result will participate in the final loss calculation. The second path enters the upsampling optimization process, which is the core of this method. In the upsampling optimization process, the deep semantic feature map (14×14) and multiple downsampled feature maps of different scales (Features28×28, Features56×56, Features112×112) from the multi-scale edge supervision module are sequentially fed into three cascaded upsampling optimization modules (i.e.,...). Figure 2 The processing is carried out in the upsampling optimization modules 1, 2, and 3. Each upsampling optimization module contains a deep-shallow feature fusion unit and a feature diversification unit, responsible for fusing deep semantic features with shallow detail features of the corresponding scale and improving resolution. Upsampling optimization module 1 outputs a feature map of 28×28 features, upsampling optimization module 2 outputs a feature map of 56×56 features, and upsampling optimization module 3 outputs a feature map of 112×112 features. These output feature maps of different scales are upsampled by 8 times, 4 times, and 2 times respectively to restore the size to 224×224. Then, each is passed through a classifier (such as a Conv1×1 layer) to generate corresponding prediction results. All these prediction results, along with the prediction results of the first path, are fed into the segmented weighted deep supervision module, compared with the real labels (i.e., semantic segmentation labels), and the multi-class cross-entropy loss is calculated through weighted calculation. With Dice loss This is used to construct the deep supervised loss, optimizing the main body of the network. Finally, the Feature112×112 feature map output by the upsampling optimization module 3 undergoes a final upsampling operation of 2x, followed by a Conv1×1 convolutional layer and an Argmax classification operation to generate the network's final semantic segmentation result (i.e., Figure 2 The prediction map in the image is used to complete pixel-level semantic segmentation of the entire input image.

[0109] Figure 3The detailed internal structure and data flow process of the upsampling optimization module, a key component in the framework of this invention, are described. The input to this module consists of two parts: a deep semantic feature map (i.e.,...) Figure 3 Deep features) and shallow feature maps (i.e. Figure 3 Shallow features in the model typically have different spatial dimensions and number of channels. The module's processing flow begins with the shallow feature fusion unit (i.e., deep and shallow feature fusion unit). Figure 3 (Deep and shallow feature fusion module in the middle). First, the deep semantic feature map with a smaller spatial size is aligned with the shallow feature map size through an Upsample by 2 operation. Then, the two feature maps are independently normalized to L2 norm, which is achieved by formula (2), which is: ,in, Indicates input features, For the processed features, It is a learnable parameter vector. This is a minimal constant. This process calculates the sum of squares and takes the square root of all elements in each channel of the feature to obtain the L2 norm, and then uses learnable parameters... The normalized feature values ​​are scaled. The two feature streams, after normalization and scaling, are fed into a concatenation + Conv1×1 stage, where they are concatenated along the channel dimension, and then a 1×1 convolutional layer is used to integrate the concatenated fusion result and adjust the number of channels. The fusion result is then fed into a feature diversification unit (i.e.,...). Figure 3The feature diversification module performs multi-branch parallel processing to extract diverse contextual information. This unit contains four branches: the first branch first performs an Average function with a kernel size of 2×2 on the input. The first branch uses average pooling to maintain the size, then passes through a Conv1×1 layer to compress the number of channels to Channels / / 4 (one-quarter) of the input channels. The second branch directly passes through a Conv1×1 layer to compress the number of channels to Channels / / 4. The third branch first uses Conv1×1 to halve the number of channels to Channels / / 2, then passes through Conv1×3 (1×3 convolution) and Conv3×1 (3×1 convolution) layers, then passes through a Conv1×3 layer to halve the number of channels to Channels / / 4, and finally passes through a Conv3×1 layer. The fourth branch first uses Conv1×1 to halve the number of channels to Channels / / 2, then passes through Conv1×7 (1×7 convolution) and Conv7×1 (7×1 convolution) layers, then passes through a Conv1×7 layer to halve the number of channels to Channels / / 4, and finally passes through a Conv7×1 layer. After the four branches are processed, their outputs are concatenated along the channel dimension and finally fused and compressed by another Conv1×1 convolutional layer to output the processed feature map, thus completing the forward propagation process of the entire upsampling optimization module.

[0110] It should be noted that, in one embodiment, the overall accuracy of semantic segmentation results is verified using the Intersection over Union (IoU) metric. Experiments were conducted using two datasets for training and testing. The Potsdam dataset used 54,675 augmented images for training and 4,374 unaugmented images for testing; the Vaihingen dataset used 12,250 augmented images for training and 627 unaugmented images for testing. Furthermore, the network presented in this paper is primarily compared in accuracy with three other high-performance classic semantic segmentation networks: DeepLabV3+, FLANet, and CaraNet.

[0111] As shown in Table 2, in the Potsdam dataset, our proposed method achieves the highest IoU across most categories, and leads in overall mIoU. Compared to the other three comparison networks, the IoU for each category and the overall metric show a gradual upward trend. Our proposed method improves mIoU performance by 2.4% compared to DeepLabV3+, 1.4% compared to FLANet, and 1.0% compared to CaraNet.

[0112] As shown in Table 3, our method achieved the highest mIoU value of 73.9% on the Vaihingen dataset, which is significantly better than the benchmark DeepLabV3+ model, with an absolute improvement of 5.0%, and an improvement of 2.3% and 4.6% compared to FLANet and CaraNet, respectively.

[0113] Table 2: Table 3: Figure 4 The performance differences of the proposed method and multiple benchmark models in semantic segmentation results on two typical regions of the Vaihingen test set are visually demonstrated through a side-by-side comparison. The upper sequence is labeled 'a', and the lower sequence is labeled 'b', representing two different test samples. For each sample, the sequence from left to right clearly shows the progressive relationship between data processing and results: the starting column is the original remote sensing image, i.e., the input remote sensing image data to be processed; followed by the semantic segmentation label (i.e.,...). Figure 4 The labels in the first column represent manually annotated ground truth values, serving as the gold standard for evaluating the performance of all methods. Subsequent columns show the processing results of different semantic segmentation network models on the same input image. Specific comparison models include DeepLabV3+, FLANet, and CaraNet, while the last column represents the method of this invention (i.e.,...). Figure 4 The output results of the method described in this paper are shown. In all the result images, different colors represent different semantic categories, such as impermeable surfaces, buildings, low vegetation, trees, cars, and background. Through visual comparison, it can be observed that compared with the segmentation results of methods such as DeepLabV3+, FLANet, and CaraNet, the semantic segmentation results generated by the method of this invention are clearer and sharper at the boundaries of ground objects, the outlines of small ground objects (such as cars) are more complete, and the ability to distinguish ground objects with similar spectral features but different semantics (such as low vegetation and trees) is stronger. The overall segmentation map has less noise and smoother homogeneous regions. This visual improvement directly confirms the quantitative analysis data in Table 2, that is, the method of this invention achieves the highest mean intersection-union ratio (mIoU) on the Potsdam dataset. For example, the IoU improvement is significant in categories such as buildings and cars. This is due to the multi-scale edge supervision module introduced in this invention to strengthen boundary information, the upsampling optimization module to refine the restoration of details, and the segmented weighted depth supervision mechanism to optimize the overall training process.

[0114] Figure 5The performance differences between the proposed method and multiple benchmark models in semantic segmentation of two typical regions on the Vaihingen test set are visually demonstrated through side-by-side comparison. The upper sequence is labeled 'a', and the lower sequence is labeled 'b', representing two different test samples. For each sample, the visual information flow from left to right clearly illustrates the complete process from the original input to the final prediction: the leftmost side represents the remote sensing image data to be processed (i.e., the input data). Figure 5 The remote sensing imagery in the imagery; followed by semantic segmentation labels (i.e., the evaluation benchmark). Figure 5 The following three columns show the processing results of three comparison models: DeepLab V3+, FLANet, and CaraNet on the same image; the rightmost column shows the method of this invention (i.e., Figure 5 The semantic segmentation results generated by the method described in this paper are shown. All result images use color coding to distinguish different semantic categories, including impermeable surfaces, buildings, low vegetation, trees, and cars. Visual analysis reveals that in complex scenes, such as when buildings and trees intersect in region a, or when the boundaries between small roads and vegetation are blurred in region b, the segmentation results of DeepLab V3+, FLANet, and CaraNet may exhibit coarse boundaries, loss of small objects, or category confusion. In contrast, the semantic segmentation results output by the method of this invention demonstrate more accurate boundary localization, more complete outlines of small features (such as cars), and less internal fragmentation noise; for example, it more accurately restores the rectangular outlines of buildings and the clustered morphology of trees. This improvement in visual effect corroborates the quantitative indicators in Table 3, demonstrating the superiority of the method of this invention in achieving the highest mIoU (73.9%) on the Vaihingen dataset. The key lies in the multi-scale edge supervision module effectively extracting and utilizing edge prior information, and the upsampling optimization module realizing the fine fusion of shallow and deep features and resolution restoration, thereby significantly improving the geometric accuracy and semantic consistency of the final semantic segmentation result.

[0115] The beneficial effects of this invention are as follows: Firstly, addressing the weakness of existing semantic segmentation networks in their upsampling modules, this invention introduces an upsampling optimization module coupled with a multi-scale edge supervision module. This improves the efficiency of the upsampling optimization module in utilizing deep semantic feature maps and enhances the network's ability to recover land class boundaries. Secondly, the optimized feature extraction network reduces feature redundancy and increases the network's feature specificity and information capacity. Finally, by employing a segmented weighted deep supervision mechanism, the network's upsampling and the entire training process are optimized.

[0116] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation. The scheme after adjusting the order is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0117] like Figure 6 As shown, a multi-scale edge-constrained semantic segmentation upsampling optimization system 200 according to an embodiment of the present invention includes a data acquisition module 201, a first extraction module 202, a downsampling processing module 203, a second extraction module 204, a fusion recovery module 205, a classification prediction module 206, a first training optimization module 207, a second training optimization module 208, and a semantic segmentation module 209. The data acquisition module 201 is used to: acquire remote sensing image data to be processed; The first extraction module 202 is used to: extract shallow features from the remote sensing image data to be processed to obtain a shallow feature map; The downsampling processing module 203 is used to: input the shallow feature map into the multi-scale edge supervision module, perform edge prediction and multi-scale downsampling processing respectively, and obtain the edge prediction result and multiple downsampling feature maps of different scales; The second extraction module 204 is used to: input the shallow feature map into the optimized feature extraction network to extract deep semantic features, and obtain a deep semantic feature map; The fusion and restoration module 205 is used to: input the deep semantic feature map and multiple downsampled feature maps of different scales into multiple cascaded upsampling optimization modules for feature fusion and resolution restoration, so as to obtain multiple intermediate scale fused feature maps and a final scale fused feature map; The classification prediction module 206 is used to: perform upsampling and classification prediction based on the fused feature maps at multiple intermediate scales to obtain multiple intermediate prediction results; The first training optimization module 207 is used to: calculate the edge supervision loss by combining the edge prediction results with the preset edge labels, and to train and optimize the multi-scale edge supervision module. The second training optimization module 208 is used to: calculate the classification loss for multiple intermediate prediction results and semantic segmentation labels respectively, and to perform weighted summation of the classification loss according to the training stage to obtain the deep supervision loss, and to train and optimize the upsampling optimization module and the feature extraction network based on the deep supervision loss; The semantic segmentation module 209 is used to generate semantic segmentation results of the remote sensing image data to be processed based on the fused feature map of the final scale, by utilizing the trained and optimized multi-scale edge supervision module, feature extraction network and multiple upsampling optimization modules.

[0118] Optionally, in the above technical solution, the downsampling processing module 203 is specifically used to: perform convolution operation on the shallow feature map to obtain a first feature; input the first feature into the first branch to perform convolution and classification operations to obtain an edge prediction result; input the first feature into the second branch to perform at least two different downsampling operations to obtain multiple downsampling feature maps of different scales.

[0119] Optionally, in the above technical solution, the optimized feature extraction network is a network obtained by modifying the DenseNet121 network structure; the modification includes: reducing the number of convolutional blocks in at least one dense block in the DenseNet121 network structure, replacing the average pooling operation in at least one transition layer with the max pooling operation, and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the shallow feature map in the initial stage.

[0120] Optionally, in the above technical solution, the upsampling optimization module includes a deep and shallow feature fusion unit and a feature diversification unit connected in sequence; in the multiple cascaded upsampling optimization modules, the deep and shallow feature fusion unit of the first upsampling optimization module is used to: normalize the deep semantic feature map and the downsampling feature map of the first scale from multiple downsampling feature maps of different scales, and then concatenate and fuse them with the learned parameters after scaling; the feature diversification unit of the first upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep and shallow feature fusion unit, and output the fusion feature map of the first intermediate scale; the deep and shallow feature fusion unit of each upsampling optimization module from the second upsampling optimization module to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep and shallow feature fusion unit, and output the fusion feature map of the first intermediate scale; The feature fusion unit is used to: receive the fused feature map output by the previous upsampling optimization module, and then merge it with the corresponding downsampling feature maps of multiple different scales after normalization and scaling by learnable parameters; the feature diversification unit of each upsampling optimization module from the second to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the current deep and shallow feature fusion unit, and output the corresponding intermediate scale fused feature map; wherein, the fused feature map output by the last upsampling optimization module is determined as the final scale fused feature map, and the fused feature maps output by the other upsampling optimization modules are determined as multiple intermediate scale fused feature maps.

[0121] Optionally, in the above technical solution, the second training optimization module 208 is further specifically used to: divide the training process into multiple training stages; assign different weight coefficients to different types of classification loss functions in each training stage; and sum the calculation results of different types of loss functions corresponding to each intermediate prediction result according to the weight coefficients of the training stage in which they are located, to constitute the contribution of the intermediate prediction result to the deep supervision loss.

[0122] It should be noted that the beneficial effects of the multi-scale edge-constrained semantic segmentation upsampling optimization system 200 provided in the above embodiments are the same as those of the multi-scale edge-constrained semantic segmentation upsampling optimization method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0123] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned semantic segmentation upsampling optimization methods with multi-scale edge constraints.

[0124] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned semantic segmentation upsampling optimization methods for multi-scale edge constraints.

[0125] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0126] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A semantic segmentation upsampling optimization method with multi-scale edge constraints, characterized in that, include: Acquire remote sensing image data to be processed; Shallow feature extraction is performed on the remote sensing image data to be processed to obtain a shallow feature map; The shallow feature map is input into the multi-scale edge supervision module, where edge prediction and multi-scale downsampling are performed respectively to obtain the edge prediction result and multiple downsampled feature maps at different scales. The shallow feature map is input into an optimized feature extraction network to extract deep semantic features, thereby obtaining a deep semantic feature map; The deep semantic feature map and the multiple downsampled feature maps at different scales are sequentially input into multiple cascaded upsampling optimization modules for feature fusion and resolution restoration, resulting in multiple intermediate-scale fused feature maps and a final-scale fused feature map. Based on the fused feature maps at multiple intermediate scales, upsampling and classification predictions are performed respectively to obtain multiple intermediate prediction results; The edge prediction results are compared with the preset edge labels to calculate the edge supervision loss, and the multi-scale edge supervision module is trained and optimized. The classification loss is calculated for each of the multiple intermediate prediction results and the semantic segmentation label. The classification loss is then weighted and summed according to the training stage to obtain the deep supervision loss. The upsampling optimization module and the feature extraction network are trained and optimized based on the deep supervision loss. Using the trained and optimized multi-scale edge supervision module, the feature extraction network, and the multiple upsampling optimization modules, the semantic segmentation result of the remote sensing image data to be processed is generated based on the fused feature map at the final scale.

2. The semantic segmentation upsampling optimization method with multi-scale edge constraints according to claim 1, characterized in that, The process of obtaining edge prediction results and downsampled feature maps of multiple different scales through a multi-scale edge supervision module includes: The shallow feature map is convolved to obtain the first feature; The first feature is input into the first branch for convolution and classification operations to obtain the edge prediction result; The first feature is input into the second branch and downsampled at least two different magnifications to obtain the multiple downsampled feature maps of different scales.

3. The semantic segmentation upsampling optimization method with multi-scale edge constraints according to claim 1, characterized in that, The optimized feature extraction network is a network obtained by modifying the DenseNet121 network structure; the modifications include: reducing the number of convolutional blocks in at least one dense block in the DenseNet121 network structure, replacing the average pooling operation in at least one transition layer with the max pooling operation, and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the shallow feature map in the initial stage.

4. A semantic segmentation upsampling optimization method with multi-scale edge constraints according to any one of claims 1 to 4, characterized in that, The upsampling optimization module includes a deep and shallow feature fusion unit and a feature diversification unit connected in sequence; In the multiple cascaded upsampling optimization modules, the deep and shallow feature fusion unit of the first upsampling optimization module is used to: normalize the deep semantic feature map and the first scale downsampling feature map of the multiple downsampling feature maps of different scales, and then concatenate and fuse them with the learnable parameters after scaling. The feature diversification unit of the first upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the deep and shallow feature fusion unit, and output a fusion feature map at the first intermediate scale; The deep and shallow feature fusion unit of each upsampling optimization module from the second upsampling optimization module to the last upsampling optimization module is used to: receive the fused feature map output by the previous upsampling optimization module, and then normalize and scale the downsampling feature map of the corresponding scale in the multiple downsampling feature maps of different scales before splicing and fusing it. The feature diversification unit of each upsampling optimization module from the second upsampling optimization module to the last upsampling optimization module is used to: perform multi-branch parallel processing and feature compression on the fusion result output by the current deep and shallow feature fusion unit, and output the fusion feature map corresponding to the intermediate scale; Among them, the fusion feature map output by the last upsampling optimization module is determined as the fusion feature map of the final scale, and the fusion feature maps output by the remaining upsampling optimization modules are determined as the fusion feature maps of the multiple intermediate scales.

5. The semantic segmentation upsampling optimization method with multi-scale edge constraints according to claim 1, characterized in that, The weighted summation of the classification loss based on the training phase includes: The training process is divided into multiple training stages; in each training stage, different weight coefficients are assigned to different types of classification loss functions; the calculation results of different types of loss functions corresponding to each intermediate prediction result are weighted and summed according to the weight coefficients of the training stage in which they are located, which constitutes the contribution of the intermediate prediction result to the deep supervision loss.

6. A semantic segmentation upsampling optimization system with multi-scale edge constraints, characterized in that, It includes a data acquisition module, a first extraction module, a downsampling processing module, a second extraction module, a fusion recovery module, a classification prediction module, a first training optimization module, a second training optimization module, and a semantic segmentation module; The data acquisition module is used to: acquire remote sensing image data to be processed; The first extraction module is used to: perform shallow feature extraction on the remote sensing image data to be processed to obtain a shallow feature map; The downsampling processing module is used to: input the shallow feature map into the multi-scale edge supervision module, perform edge prediction and multi-scale downsampling processing respectively, and obtain the edge prediction result and multiple downsampling feature maps of different scales; The second extraction module is used to: input the shallow feature map into an optimized feature extraction network to extract deep semantic features, thereby obtaining a deep semantic feature map; The fusion and restoration module is used to: sequentially input the deep semantic feature map and the multiple downsampled feature maps of different scales into multiple cascaded upsampling optimization modules for feature fusion and resolution restoration, to obtain multiple intermediate-scale fusion feature maps and a final-scale fusion feature map; The classification prediction module is used to: perform upsampling and classification prediction based on the fused feature maps of the multiple intermediate scales respectively, and obtain multiple intermediate prediction results; The first training optimization module is used to: calculate the edge supervision loss by combining the edge prediction result with the preset edge label, and to train and optimize the multi-scale edge supervision module; The second training optimization module is used to: calculate the classification loss for the multiple intermediate prediction results and semantic segmentation labels respectively, and perform a weighted summation of the classification loss according to the training stage to obtain the deep supervision loss, and perform training optimization on the upsampling optimization module and the feature extraction network based on the deep supervision loss; The semantic segmentation module is used to generate the semantic segmentation result of the remote sensing image data to be processed based on the fused feature map of the final scale, using the trained and optimized multi-scale edge supervision module, the feature extraction network, and the multiple upsampling optimization modules.

7. The semantic segmentation upsampling optimization system with multi-scale edge constraints according to claim 6, characterized in that, The downsampling processing module is specifically used for: The shallow feature map is convolved to obtain the first feature; The first feature is input into the first branch for convolution and classification operations to obtain the edge prediction result; The first feature is input into the second branch and downsampled at least two different magnifications to obtain the multiple downsampled feature maps of different scales.

8. The semantic segmentation upsampling optimization system with multi-scale edge constraints according to claim 6, characterized in that, The optimized feature extraction network is a network obtained by modifying the DenseNet121 network structure; the modifications include: reducing the number of convolutional blocks in at least one dense block in the DenseNet121 network structure, replacing the average pooling operation in at least one transition layer with the max pooling operation, and adjusting the kernel size and stride of the initial convolutional layer to reduce the downsampling factor of the shallow feature map in the initial stage.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the semantic segmentation upsampling optimization method with multi-scale edge constraints as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the semantic segmentation upsampling optimization method with multi-scale edge constraints as claimed in the claim.

Images