Semantic feature enhancement based super high resolution image semantic segmentation algorithm

Abstract

This is a semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement, belonging to the field of computer vision. Its key features include the following steps: Step 1, input the original image. I 0; Step 2, process the original image I 0 is preprocessed to obtain the image I 1 and image P 1; Step 3, process the image I 1 and image P Step 1: Perform bi-branch feature extraction; Step 4: Perform feature fusion on the feature map obtained from bi-branch feature extraction in Step 3 to obtain the final feature map; Step 5: Generate a composite image using the final feature map obtained in Step 4. I The final semantic segmentation mask with the same resolution is 0. In the semantic feature enhancement-based ultra-high resolution image semantic segmentation algorithm of this application, the advantages of wavelet transform in time-frequency analysis and multi-resolution representation of signals are utilized to achieve lossless compression and information enhancement of the original image. Furthermore, a dual-branch feature extraction method is adopted to balance the contradiction between segmentation accuracy, memory consumption, and inference speed.

Description

Technical Field

[0001] The semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement belongs to the field of computer vision. Background Technology

[0002] Semantic segmentation is a key task in computer vision, aiming to classify images at the pixel level, dividing the input image into semantically consistent regions and assigning a semantic label to each pixel. Compared with traditional image classification, semantic segmentation can achieve precise target localization and boundary delineation, thus having wide applications in many high-resolution imaging fields. However, semantic segmentation of ultra-high-resolution images faces significant challenges, mainly in terms of high computational complexity, large memory consumption, and slow processing speed. Ultra-high-resolution images are large in size and have a large number of pixels, requiring substantial computational resources, resulting in long training and inference times for traditional models and deep learning-based models.

[0003] To address the aforementioned challenges, the introduction of fully convolutional networks (WCNNs) established an end-to-end learning framework for dense prediction tasks, marking a milestone in semantic segmentation. Based on this architecture, subsequent models such as Deeplabv3+ and Feature Pyramid Networks (PPNs) have demonstrated significant performance. Deeplabv3+ introduces the Xception model and combines dilated spatial pyramid pooling with depthwise separable convolutions to capture multi-scale contextual information; PPNs effectively fuses multi-scale features by constructing feature pyramids, enhancing the representation of objects of different sizes. However, these models are structurally complex, and the need to process ultra-high-resolution images often results in a large number of parameters, posing a significant challenge to computational efficiency.

[0004] To overcome the challenges of ultra-high resolution image segmentation, various solutions have emerged in existing technologies: Scheme 1: Utilizing both global context and local detail information. For example, the scheme described in the paper: Chen, W., Jiang, Z., Wang, Z., Cui, K., & Qian, X. (2019). Collaborative global-local networks for memory-efficient segmentation of ultra-high resolution images. In Proceedings of the IEEE / CVFconference on computer vision and pattern recognition (pp. 8924-8933). This scheme processes the downsampled overall image and the cropped high-resolution image patches through global and local branches respectively, and uses a deep feature fusion mechanism to perform hierarchical fusion of local details and global context information to alleviate the problem of detail loss when only processing scaled images.

[0005] Option 2: The paper Cheng, HK, Chung, J., Tai, YW, & Tang, CK (2020). CascadePSP: Toward class-agnostic and very high-resolution segmentation via global and local refinement. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition (pp. 8890-8899), the CascadePSP cascade refinement architecture was proposed. This structure, as a general optimization module, can perform cascade refinement on any existing model output to improve model performance without fine-tuning its basic parameters, thereby preserving structural details and object boundaries in subsequent processing stages.

[0006] Subsequent research has advanced the field through various structural innovations, including: Scheme 3, as described in the paper Huynh, C., Tran, AT, Luu, K., & Hoai, M. (2021). Progressivesemantic segmentation. In Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition (pp.16755-16764), employs a multi-stage processing architecture from coarse to fine, progressively increasing image resolution to recursively reconstruct and refine details lost during downsampling, thereby achieving continuous optimization of boundary segmentation.

[0007] Solution 4: The paper Li, Q., Yang, W., Liu, W., Yu, Y., & He, S. (2021). From contexts to locality: Ultra-high resolution image segmentation via locality-aware contextual correlation. In Proceedings of the IEEE / CVF International Conference on Computer Vision (pp. 7252-7261), a dedicated framework for ultra-high resolution image segmentation is proposed. It strategically decouples the problem from the novel local segmentation model and the contextual semantic refinement network, taking into account both local accuracy and global consistency.

[0008] Option 5: The ISDNet framework was proposed in the paper Guo, S., Liu, L., Gan, Z., Wang, Y., Zhang, W., Wang, C., ... & Xu, K. (2022). ISDNet: Integrating shallow and deep networks for efficient ultra-high resolution segmentation. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 4361-4370), it focuses on utilizing the feature collaboration of different network depths and enhances the model's discriminative power and scale robustness through a relation-aware feature fusion module.

[0009] Scheme 6: The paper Ji, D., Zhao, F., & Lu, H. (2023). Guided patch-grouping wavelet transformer with spatial congruence for ultra-high resolution segmentation. arXiv preprint arXiv:2307.00711 proposes a guided patch-grouping wavelet transform model. It first divides the ultra-high resolution image into image patches and adopts a dynamic grouping strategy to manage complexity. Its core is to use a lightweight multi-head wavelet transform network to efficiently handle the computational challenges brought by ultra-high resolution data while extracting low-frequency local features.

[0010] However, in practice, the aforementioned existing technologies typically require segmenting the input image into small blocks and processing them separately, leading to computational redundancy and reduced inference speed. Furthermore, these methods usually downsample the ultra-high-resolution image before segmentation, an operation that inevitably results in blurred edges, loss of semantic information, and reduced segmentation accuracy. Summary of the Invention

[0011] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art and provide an ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement. It utilizes the advantages of wavelet transform in time-frequency analysis and multi-resolution representation of signals, and achieves lossless compression and information enhancement of the original image through preprocessing based on multiple two-dimensional discrete Haar wavelet transforms. Furthermore, it adopts a bi-branch feature extraction method to balance the contradiction between segmentation accuracy, memory consumption and inference speed.

[0012] The technical solution adopted by this invention to solve its technical problem is: a semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement, characterized by the following steps: Step 1: Input the ultra-high resolution original image to be segmented. I 0; Step 2, process the original image I 0. Preprocessing is performed on the original image. I Multiple two-dimensional discrete Haar wavelet transforms of 0 yield an information-enhanced full-resolution image: Image I 1. Information-preserving downsampled image: Image P 1; Step 3, the image obtained in Step 2 I 1 and image P 1. Perform bi-branch feature extraction and obtain the feature maps after feature extraction for each bi-branch; Step 4: Perform feature fusion on the feature map obtained from the bi-branch feature extraction in Step 3 to obtain the final feature map; Step 5: Generate a composite image using the final feature map obtained in Step 4. I 0. The final semantic segmentation mask with the same resolution.

[0013] Preferably, step 2 includes the following steps: Step 2-1: Apply two-dimensional discrete Haar wavelet transform to the original image. I 0 is decomposed to obtain four subbands: low-frequency approximation component: subband LL, and three high-frequency detail components: subband LH, subband HL, and subband HH; Step 2-2: Perform a second two-dimensional discrete Haar wavelet transform on the subband LL to obtain the image. P 1; Steps 2-3 involve upsampling subbands LH, HL, and HH to the original image. I 0 resolution; Steps 2-4: Extract the original image I The enhanced image is obtained by adding 0 element-wise to the three high-frequency subbands after upsampling in steps 2-3. I 1.

[0014] Preferably, the image I The formula for calculating 1 is: I 1= I 0+Upsample(LH, HL, HH), where Upsample(·) represents the upsampling operation.

[0015] Preferably, in step 3, the image I Taking 1 as input, after feature extraction via the global branch, the output consists of two feature maps at different scales: Feature Map F 1 and feature map F 2; with images P Taking 1 as input, after feature extraction through local branches, the output is a deep feature map: Feature Map F 3.

[0016] Preferably, step 4 includes the following steps: Step 4-1, process the feature map output in step 3. F 2. Upsampling is performed using a method based on two-dimensional discrete Haar wavelet inverse transform; Step 4-2, process the feature map output in step 3. F 3. Similarly, perform two-dimensional discrete Haar wavelet inverse transform upsampling to align its resolution with the feature map. F The scale of 1; Step 4-3: Align the resolution of the feature maps. F 1. Feature map after upsampling F2 and the upsampled feature map F 3. Aggregate the data to obtain the final feature map that combines global context and local details.

[0017] Preferably, in step 5, the final feature map fused in step 4 is input into a lightweight prediction head, which outputs a class probability map for each pixel. Then, it is upsampled or directly output to obtain a map similar to the original image. I 0. The final semantic segmentation mask with the same resolution.

[0018] Compared with the prior art, the beneficial effects of this invention are: In the semantic feature enhancement-based ultra-high resolution image semantic segmentation algorithm of this application, the advantages of wavelet transform in time-frequency analysis and multi-resolution representation of signals are utilized to achieve lossless compression and information enhancement of the original image. Furthermore, a dual-branch feature extraction method is adopted to balance the contradiction between segmentation accuracy, memory consumption, and inference speed.

[0019] Compared to traditional downsampling methods that suffer from edge blurring and loss of high-frequency details due to local smoothing, the technical solution in this application uses two-dimensional discrete wavelet transform to generate downsampled images. P 1. Wavelet transform, based on its multi-resolution analysis framework, can jointly represent images in the frequency and spatial domains, thereby achieving image size reduction while more completely preserving the structural information and semantic content of the image. Therefore, the resulting image... P Not only does it meet the requirements in terms of scale, but its semantic integrity and information fidelity are also significantly better than traditional downsampling results. This provides crucial high-quality input for subsequent refined and highly reliable feature extraction in local branches, laying the foundation for performance improvement.

[0020] This application generates a detail-enhanced image by adding the original image element-wise to its high-frequency detail components obtained through wavelet decomposition. I 1. Essentially, this approach involves targeted enhancement of edge information representing object contours and texture components depicting surface characteristics in images, significantly improving the response intensity of key local features. Thanks to this, even with a lightweight global branch design (relatively shallow in depth and limited in parameters), the enhanced network can extract semantic features containing richer spatial details and stronger discriminative power from the enhanced input. These enhanced features can more accurately encode the relative positions, morphological structures, and contextual relationships of elements in a scene, thereby significantly improving the model's overall accuracy in understanding complex global scene layouts and semantic relationships.

[0021] In the dual-branch feature extraction scheme, a shallow network is used to process full-resolution images, while a deep network processes small-sized images. The global branch has a low computational burden, ensuring processing speed; although the local branch has a deep network, the overall computational and memory overhead is controllable due to the small input size. The two work together to achieve a balance between accuracy and efficiency.

[0022] In this application, a two-dimensional discrete Haar wavelet inverse transform is used to upsample the multi-scale feature maps generated by the "global branch" and "local branch." This operation has several advantages. First, it offers high-quality detail recovery. As an accurate signal reconstruction method, it effectively utilizes high-frequency detail subbands (horizontal, vertical, and diagonal directions) to recover edges and textures, significantly outperforming traditional interpolation methods and avoiding blurred results. Second, it boasts excellent computational efficiency. This transform uses fixed, simple filter kernels, requires no learnable parameters, is computationally fast with low resource consumption, and avoids introducing checkerboard artifacts that might occur with transposed convolution. Finally, it is particularly suitable for segmentation tasks. The aforementioned characteristics make it highly valuable in tasks such as semantic segmentation that require high edge accuracy, improving the clarity of segmentation boundaries and overall accuracy at a relatively low computational cost. Attached Figure Description

[0023] Figure 1 This is a flowchart of a semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement.

[0024] Figure 2 This is a flowchart of the original image preprocessing process for a semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement.

[0025] Figure 3 This is a schematic diagram of the two-dimensional discrete Haar wavelet transform process.

[0026] Figure 4 This is an example of the effect of two-dimensional discrete Haar wavelet transform.

[0027] Figure 5 This is an example image from Example 1 of an ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement.

[0028] Figure 6 This is a real-world annotation diagram of Example 1 of the ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement.

[0029] Figure 7 This is an example image from Example 2 of an ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement.

[0030] Figure 8 This is a real-world annotation diagram of Example 2 of the ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement. Detailed Implementation

[0031] Figures 1-8 This is the preferred embodiment of the present invention, which is described below in conjunction with the accompanying drawings. Figures 1-8 The present invention will be further described below.

[0032] like Figure 1 As shown, the semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement includes the following steps: Step 1: Input the ultra-high resolution original image to be segmented; Input the ultra-high resolution raw image to be segmented I 0.

[0033] Step 2, Preprocessing of the original image; Combination Figure 2 The preprocessing of the original image includes the following steps: Step 2-1: Decompose the original image using two-dimensional discrete Haar wavelet transform; For the original image I 0 performs the first two-dimensional discrete Haar wavelet transform, and further combines Figure 3 First, they are passed through low-pass filters. F L and high-pass filter F H The images are processed separately to obtain two image data sets based on the original image. Then, the two image data sets are passed through a low-pass filter again. F L and high-pass filter F H The original image is processed and decomposed into four sub-bands: a low-frequency approximation component (LL) and three high-frequency detail components: sub-band LH (horizontal), sub-band HL (vertical), and sub-band HH (diagonal). Sub-band LL contains the main contour and overall information of the image, while sub-bands LH, HL, and HH contain detailed information such as edges and textures. Figure 4 As shown.

[0034] Step 2-2: Perform a second-order two-dimensional discrete Haar wavelet transform decomposition on the low-frequency sub-band; Performing LL on subbands as follows Figure 3 The second two-dimensional discrete Haar wavelet transform shown yields an information-preserving downsampled image. P 1.

[0035] Furthermore, because each wavelet transform performs pixel-by-pixel downsampling in the row and column directions, the image after two transformations... P The spatial resolution of 1 is converted into the original image. I0 / 4. Because the two-dimensional discrete Haar wavelet transform itself possesses orthogonal and complete mathematical properties, this means that the basis function space it constructs can completely represent the original signal without redundancy or loss. Specifically, the sub-band LL obtained from downsampling can, theoretically, be accurately and losslessly reconstructed from the original image using the corresponding inverse transform, thus ensuring the image... P 1. After undergoing downsampling, the low-frequency components are not blurred or discarded, but are compressed and preserved in the most efficient way, thereby maintaining the low-frequency information of the original image that carries the main structure and macroscopic semantics to the maximum extent, providing a stable and reliable foundation for subsequent semantic understanding.

[0036] Steps 2-3: Upsample the high-frequency subband to the original image resolution; Upsampling is performed on the sub-bands LH, HL, and HH obtained in step 2-1 to upsample them to the original image. I 0 resolution.

[0037] Steps 2-4: Element-wise addition of the high-frequency subband to the original image; Original image I The image is obtained by element-wise addition of 0 to the three high-frequency subbands after upsampling in steps 2-3, resulting in the enhanced full-resolution image. I 1. Image I The formula for calculating 1 is: I 1= I 0+Upsample(LH, HL, HH), where Upsample(·) represents the upsampling operation.

[0038] After element-wise addition of the high-frequency subbands to the original image, targeted enhancement of high-frequency detail components in the image, especially the edge contours that characterize object shapes and the texture structures that define surface properties, significantly strengthens these local features crucial for semantic understanding in the feature space. This enhancement does not uniformly boost all information but selectively highlights those components that contribute most to object category discrimination and boundary delineation, thereby significantly optimizing the high-level semantic representation capability of the feature map.

[0039] Thanks to this, neural networks are able to more clearly distinguish the complex boundaries between adjacent objects and more keenly capture subtle internal texture changes in subsequent semantic segmentation tasks, ultimately achieving a systematic improvement in pixel-level classification accuracy, boundary localization precision, and robustness to morphological changes.

[0040] Through the above steps, after performing step 2, we obtain the result based on the original image. I Two outputs of 0: Information-enhanced full-resolution image: Image I1. Downsampled image with information preservation: Image P 1.

[0041] Step 3: Extraction of bi-branch features from the preprocessed original image; In step 3, a two-branch feature extraction scheme using global and local branches is adopted. Specifically, the image obtained in step 2 is... I 1 and image P 1. Feed the data into the global branch and local branch respectively for feature extraction: When performing feature extraction in the global branch: using the image I 1 is used as input. This branch employs a lightweight, shallow network with few parameters and a simple structure (e.g., the first four stages of an STDC network). After feature extraction from the global branch, two feature maps of different scales are output: Feature Map F 1 and feature map F 2.

[0042] The advantages of global branch feature processing lie in its efficient computational performance and powerful feature extraction capabilities: on the one hand, its inference process is extremely fast, enabling efficient processing of enhanced full-resolution images while maintaining low computational complexity; on the other hand, this design can systematically extract global contextual features covering the entire scene from the image, thereby accurately understanding the overall layout, spatial structure, and semantic associations and relative relationships between different objects, providing important high-level semantic guidance for subsequent refined segmentation.

[0043] When performing feature extraction of local branches: using images P 1 is used as input. This local branch employs a network with strong representational capabilities and a deep structure (e.g., DeepLabv3+ with ResNet-101 as its backbone). After feature extraction by the local branch, a deep feature map is output: Feature Map F 3.

[0044] The advantage of local branch feature extraction is that it can extract features from images. P After downsampling, the size of the input is reduced to only 1 / 4 of the original. This design significantly reduces the consumption of GPU memory resources and greatly reduces the forward computation required by deep networks. Thanks to this, this branch can efficiently and focusedly learn and extract fine local structures and highly discriminative semantic features that are crucial for semantic discrimination from this reduced-size but fully preserved representation. This allows it to effectively complement other branches in the network and jointly improve the overall performance of the model.

[0045] Step 4: Feature fusion of dual-branch feature extraction; Step 4 includes the following steps: Step 4-1: Extract the feature map from the global branch feature extraction in Step 3. F 2. Upsampling is performed using a method based on two-dimensional discrete Haar wavelet inverse transform. After this operation, the feature map... F The resolution of 2 was improved to match the feature map. F 1. Completely identical (e.g., 1 / 8 of the original input image size), thus laying an aligned scale foundation for subsequent cross-scale feature fusion.

[0046] Compared to conventional bilinear interpolation or transposed convolution operations, this upsampling mechanism can more effectively recover and enhance the spatial details and structural information in the feature map that may have been weakened during the earlier downsampling process through its inherent high-frequency reconstruction characteristics.

[0047] Step 4-2: Extract the feature map from the local branch features in Step 3. F 3. Similarly, perform two-dimensional discrete Haar wavelet inverse transform upsampling to align its resolution with the feature map. F A scale of 1 (e.g., 1 / 8 of the original input image size).

[0048] Step 4-3: Align the resolution of the feature maps. F 1. Feature map after upsampling F 2 and the upsampled feature map F 3. Aggregate the data to obtain the final feature map that combines global context and local details.

[0049] Step 5: Obtain the final semantic segmentation mask with the same resolution as the original image; The feature map fused in step 4 is input into a lightweight prediction head, which outputs a class probability map for each pixel. This map is then upsampled or directly output to obtain a map similar to the original image. I 0. The final semantic segmentation mask with the same resolution.

[0050] The semantic feature enhancement-based ultra-high resolution image semantic segmentation algorithm described in this application is further validated through two examples: Example 1: In this example, taking the DeepGlobe high-resolution remote sensing image land cover classification task as an example, the implementation process of the above-mentioned semantic feature enhancement-based ultra-high resolution image semantic segmentation algorithm is explained in detail.

[0051] Step 1, Data Preparation.

[0052] The publicly available DeepGlobe dataset was used. This dataset contains 803 satellite images, each 2448×2448 pixels, covering various landforms including urban areas, farmland, forests, and water bodies, and labeled with seven land cover categories. The images were randomly divided into training, validation, and test sets in a 60%:20%:20% ratio. The final result was... Figure 5 The image shown is the original image. I 0.

[0053] Step 2, Preprocessing of the original image. According to... Figure 2 As shown, using the PyTorch framework, the two-dimensional discrete Haar wavelet transform and its inverse transform can be implemented with the help of torch.nn.functional or a custom layer to obtain the image. P 1 and image I 1.

[0054] Step 3, in the feature extraction of the global branch: the first four stages (Stage 1-Stage 4) of the STDC network are used, and its subsequent classification layers are removed. The input is an image. I 1 (2448×2448). Output feature map F 1 and feature map F The dimensions of 2 are respectively the input I 1 / 8 (306×306) and 1 / 16 (153×153).

[0055] In feature extraction of local branches: a DeepLabv3+ architecture is adopted, with the backbone network being a ResNet-101 pre-trained on ImageNet. The input is an image. P 1 (612×612). Output feature map F The image size is 3. P 1 / 32 of 1 (approximately 19×19) corresponds to the original image. I 1 / 128 of 0 (approximately 19 × 19).

[0056] Step 4, Feature Fusion: The feature maps are then fused together. F 2. Upsample to 306×306 using two-dimensional discrete Haar wavelet inverse transform. The feature map... F 3. Upsample to 306×306 using inverse wavelet transform. Then, the feature map... F 1. Feature map after upsampling F 2. Feature map after upsampling F 3. Concatenate along the channel dimension.

[0057] Step 5, Prediction Head: A 1×1 convolutional layer is used to map the number of channels in the fused feature map to the number of classes (7 classes), and then bilinear interpolation is used to upsample back to the original resolution of 2448×2448 to obtain the final segmentation prediction map, as shown below. Figure 6 As shown.

[0058] Step 6, Model Training Steps: Hardware environment: Ubuntu 20.04 operating system, single NVIDIA GeForce GTX 1070 GPU (8GB video memory).

[0059] Software environment: Python 3.8, PyTorch 1.9.0.

[0060] Training parameters: Stochastic gradient descent (SGD) optimizer with momentum of 0.9. Initial learning rate set to 0.001, using a multinomial learning rate decay strategy. Batch size set to 8. Training for a total of 8000 iterations.

[0061] Training strategy: First, train the model on the training set, evaluate its performance on the validation set at regular intervals, and save the optimal model.

[0062] Step 7, Evaluation and Results.

[0063] The model was evaluated on a test set. The primary evaluation metric was the mean Intersection over Union (mIoU). The inference speed (Frames Per Second, FPS) and GPU memory usage on a single image were also recorded. Experiments demonstrate that the proposed semantic feature-enhanced ultra-high resolution image semantic segmentation algorithm achieves both high accuracy and speed while maintaining low memory usage.

[0064] Example 2: In this example, using the Cityscapes dataset, the application of the present invention in urban scene understanding is demonstrated, and the implementation process of the above-mentioned semantic feature-enhanced ultra-high resolution image semantic segmentation algorithm is explained in detail.

[0065] Step 1, Data Preparation.

[0066] The Cityscapes finely annotated dataset was used. Image size was 1024×2048 (partially 1024×2048, uniformly adjusted or padded). It contained 30 classes, with 19 classes typically evaluated. The officially defined training set (2975 images), validation set (500 images), and test set (1525 images) were used. Similar normalization and data augmentation (such as random horizontal flipping and random scaling) were performed, ultimately resulting in... Figure 7 The image shown is the original image. I 0.

[0067] Steps 2-5. The network backbone structure is the same as in Example 1. Because the input image resolution (1024×2048) differs from DeepGlobe's, the image... I 1. Image remains unchanged in size. P The dimensions are changed to 256×512. Correspondingly, the feature map... F 1. Feature Map F 2. Feature Map F The scale of 3 also changes proportionally, ultimately resulting in the following: Figure 8 The final segmentation prediction map is shown.

[0068] Step 6, Model Training: Train on the Cityscapes training set. Initial learning rate is 0.001, using "poly" learning rate decay. Batch size is set to 8. Train for 160,000 iterations. Due to the complexity of the Cityscapes scene, the number of training iterations can be increased appropriately.

[0069] Pre-trained weights: can be initialized using backbone network weights pre-trained on ImageNet or other large datasets to accelerate convergence.

[0070] Step 7, Evaluation and Results.

[0071] The algorithm was evaluated on the Cityscapes test set. Experimental results show that the semantic feature-enhanced ultra-high resolution image semantic segmentation algorithm proposed in this application maintains a more competitive inference speed while improving accuracy.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement, characterized in that: Includes the following steps: Step 1: Input the ultra-high resolution original image to be segmented. I 0; Step 2, process the original image I 0. Preprocessing is performed on the original image. I Multiple two-dimensional discrete Haar wavelet transforms of 0 yield an information-enhanced full-resolution image: Image I 1. Downsampled image with information preservation: Image P 1; Step 3, the image obtained in Step 2 I 1 and image P 1. Perform bi-branch feature extraction and obtain the feature maps after feature extraction for each bi-branch; Step 4: Perform feature fusion on the feature map obtained from the bi-branch feature extraction in Step 3 to obtain the final feature map; Step 5: Generate a composite image using the final feature map obtained in Step 4. I 0. The final semantic segmentation mask with the same resolution; Step 2 includes the following steps: Step 2-1: Apply two-dimensional discrete Haar wavelet transform to the original image. I 0 is decomposed to obtain four subbands: low-frequency approximation component: subband LL, and three high-frequency detail components: subband LH, subband HL, and subband HH; Step 2-2: Perform a second two-dimensional discrete Haar wavelet transform on the subband LL to obtain the image. P 1; Steps 2-3 involve upsampling subbands LH, HL, and HH to the original image. I 0 resolution; Steps 2-4: Extract the original image I The enhanced image is obtained by adding 0 element-wise to the three high-frequency subbands after upsampling in steps 2-3. I 1.

2. The ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement according to claim 1, characterized in that: image I The formula for calculating 1 is: I 1= I 0+Upsample(LH, HL, HH), where Upsample(·) represents the upsampling operation.

3. The ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement according to claim 1, characterized in that: In step 3, using images I Taking 1 as input, after feature extraction via the global branch, the output consists of two feature maps at different scales: Feature Map F 1 and feature map F 2; with images P Taking 1 as input, after feature extraction through local branches, the output is a deep feature map: Feature Map F 3.

4. The semantic segmentation algorithm for ultra-high resolution images based on semantic feature enhancement according to claim 3, characterized in that: Step 4 includes the following steps: Step 4-1, process the feature map output in step 3. F 2. Upsampling is performed using a method based on two-dimensional discrete Haar wavelet inverse transform; Step 4-2, process the feature map output in step 3. F 3. Similarly, perform two-dimensional discrete Haar wavelet inverse transform upsampling to align its resolution with the feature map. F The scale of 1; Step 4-3: Align the resolution of the feature maps. F 1. Feature map after upsampling F 2 and the upsampled feature map F 3. Aggregate the data to obtain the final feature map that combines global context and local details.

5. The ultra-high resolution image semantic segmentation algorithm based on semantic feature enhancement according to claim 1, characterized in that: In step 5, the final feature map fused in step 4 is input into a lightweight prediction head, which outputs a class probability map for each pixel. This map is then upsampled or directly output to obtain a map similar to the original image. I 0. The final semantic segmentation mask with the same resolution.

Images