Remote sensing image building segmentation method and device, and remote sensing image processing equipment

Abstract

This invention provides a method, apparatus, and remote sensing image processing device for building segmentation in remote sensing images. The method includes: extracting features from the original remote sensing image based on a visual backbone encoder to obtain an image feature map; determining whether the image feature map is single-scale or multi-scale; when the image feature map is single-scale, performing downsampling and upsampling operations on the image feature map to generate multiple extended feature maps, fusing the image feature map and the multiple extended feature maps to obtain a hierarchical enhanced feature map set; when the image feature map is multi-scale, performing channel scale alignment on the image feature map to obtain multiple scale-aligned feature maps, fusing the multiple scale-aligned feature maps step-by-step based on a top-down path to obtain a multi-scale fused feature set; and inputting the hierarchical enhanced feature map set or the multi-scale fused feature map set into a building decoder to obtain the building segmentation result. This invention improves the representation capability of multi-scale building targets in remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, specifically to a method, apparatus, and remote sensing image processing equipment for building segmentation in remote sensing images. Background Technology

[0002] With the rapid development of high-resolution remote sensing satellites and UAV technologies, remote sensing imagery is increasingly widely used in urban construction, land use monitoring, and environmental assessment. Buildings, as one of the most important ground features in remote sensing images, are crucial for accurate extraction in urban planning, population estimation, and disaster prevention and mitigation. However, building targets in remote sensing images exhibit significant scale variations: the same scene may contain large factories covering thousands of square meters, or it may contain detached houses with only a few pixels; the resolution of images acquired by different sensors also varies greatly, ranging from sub-meter high-resolution satellite data to medium- and low-resolution images of tens of meters, resulting in a vast scale range for building targets. This scale diversity poses a significant challenge to the accurate identification and segmentation of building targets.

[0003] Existing deep learning-based remote sensing image building segmentation methods typically employ an encoder-decoder architecture. However, the scale types of image feature maps output by different visual backbone encoders (such as ViT, Swing Transformer, etc.) differ fundamentally: some encoders output single-scale feature maps, lacking multi-scale information, making it difficult for the model to simultaneously capture large-scale context and small-scale details; while hierarchical encoders can output multi-scale feature maps, their features at different levels differ significantly in channel dimension and semantic level. High-level features are semantically rich but have low resolution, while low-level features have high resolution but insufficient semantic abstraction. When directly used for segmentation tasks, there is a significant semantic gap between high-level semantics and low-level details, which can easily lead to semantic conflicts during fusion.

[0004] Existing segmentation methods typically employ feature enhancement schemes tailored to specific encoder architectures, lacking a unified adaptive processing mechanism. This makes it difficult to simultaneously adapt to single-scale and multi-scale encoder outputs, resulting in limited model generalization ability and decreased segmentation accuracy. Therefore, there is an urgent need for a remote sensing image building segmentation method, device, and remote sensing image processing equipment capable of adaptively processing encoder outputs of different scale types, effectively enhancing the feature representation capability of multi-scale buildings, and thereby improving the accuracy and precision of building segmentation in remote sensing images. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus, and remote sensing image processing device for building segmentation in remote sensing images, in order to solve the technical problem that the lack of a unified adaptive processing mechanism in the existing technology makes it difficult to adapt to the output of single-scale and multi-scale encoders at the same time, resulting in low accuracy and precision of building segmentation in remote sensing images.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a method for building segmentation in remote sensing images, comprising: The original remote sensing image is used as a visual backbone encoder to extract features and obtain an image feature map. Determine the scale type of the image feature map, wherein the scale type is single-scale or multi-scale; When the image feature map is single-scale, downsampling and upsampling operations are performed on the image feature map to generate multiple extended feature maps with different resolutions. The image feature map and the multiple extended feature maps are then fused to obtain a set of hierarchical enhanced feature maps. When the image feature map is multi-scale, channel scale alignment is performed on the multi-scale image feature map to obtain multiple scale-aligned feature maps, and the multiple scale-aligned feature maps are fused step by step based on a top-down path to obtain a multi-scale fused feature set. The hierarchical enhanced feature map set or the multi-scale fused feature map set is input into the building decoder to obtain the building segmentation result.

[0007] In one possible implementation, determining the scale type of the image feature map includes: Detect the output data structure of the visual backbone encoder; If the output data structure is a single feature map tensor, then the scale type is determined to be single scale; If the output data structure is an ordered set containing multiple feature map tensors, and at least two of the multiple feature map tensors have different spatial resolutions, then the scale type is determined to be multi-scale.

[0008] In one possible implementation, performing downsampling and upsampling operations on the image feature maps to generate multiple extended feature maps with different resolutions includes: The image feature map is downsampled multiple times using max pooling to obtain multiple downsampled feature maps; The image feature map is subjected to multiple transposed convolutional upsampling to obtain multiple upsampled feature maps; The plurality of upsampled feature maps and the plurality of downsampled feature maps constitute the plurality of extended feature maps.

[0009] In one possible implementation, the max pooling downsampling and the transposed convolution upsampling use a 2×2 convolution kernel with a stride of 2.

[0010] In one possible implementation, fusing the image feature map and the plurality of extended feature maps to obtain a hierarchical enhanced feature map set includes: The image feature map and each of the extended feature maps are channel-adjusted to obtain multiple channel-adjusted feature maps with the same number of channels. The multiple channel-adjusted feature maps are then concatenated along the channel dimension to obtain the set of hierarchical enhanced feature maps.

[0011] In one possible implementation, after obtaining the set of hierarchical enhanced feature maps, the following is also included: Cross-scale interactive enhancement is achieved by adjusting feature maps of different channels in the hierarchical enhanced feature map set using a cross-scale attention mechanism.

[0012] In one possible implementation, the plurality of scale-aligned feature maps include a first scale-aligned feature map, a second scale-aligned feature map, and a third scale-aligned feature map with resolutions from high to low. Then, the plurality of scale-aligned feature maps are fused step-by-step based on a top-down path to obtain a multi-scale fused feature set, including: The first scale-aligned feature map is upsampled to obtain the first upsampled aligned feature map; The first upsampled aligned feature map and the second scale aligned feature map are fused to obtain the first fused feature map; The first fused feature map is upsampled to obtain the second upsampled aligned feature map; The second upsampled aligned feature map and the third scale aligned feature map are fused to obtain a second fused feature map; The first fused feature map and the second fused feature map constitute the multi-scale fused feature set.

[0013] In one possible implementation, fusing the first upsampled aligned feature map and the second scale aligned feature map to obtain a first fused feature map includes: Determine the fusion weights of the first upsampled aligned feature map and the second scale aligned feature map; The first upsampled aligned feature map and the second scale aligned feature map are fused based on the fusion weights to obtain a first fused feature map.

[0014] Secondly, the present invention also provides a remote sensing image building segmentation device, comprising: The feature extraction unit is used to extract features from the input raw remote sensing image based on the visual backbone encoder to obtain an image feature map; A scale type determination unit is used to determine the scale type of the image feature map, wherein the scale type is single-scale or multi-scale; A single-scale processing branch unit is used to perform downsampling and upsampling operations on the image feature map when the image feature map is single-scale, generate multiple extended feature maps with different resolutions, and fuse the image feature map and the multiple extended feature maps to obtain a set of hierarchical enhanced feature maps. The multi-scale processing branch unit is used to perform channel scale alignment on the multi-scale image feature map when the image feature map is multi-scale, to obtain multiple scale-aligned feature maps, and to perform step-by-step fusion of the multiple scale-aligned feature maps based on a top-down path to obtain a multi-scale fused feature set. The building segmentation unit is used to input the hierarchical enhanced feature map set or the multi-scale fused feature map set into the building decoder to obtain the building segmentation result.

[0015] Thirdly, the present invention also provides a remote sensing image processing device, comprising an image receiver, a processor, and a memory, wherein, The image receiver is used to receive raw remote sensing images acquired by the remote sensing image acquisition device; The memory is used to store programs; The processor, coupled to the memory and the image receiver, is used to execute the program stored in the memory to implement the steps in the remote sensing image building segmentation method described in any of the above possible implementations.

[0016] The beneficial effects of this invention are as follows: The remote sensing image building segmentation method provided by this invention determines the scale type of the image feature map and then adopts a targeted feature enhancement strategy based on the scale type. This eliminates the need for manual adaptation to specific encoder architectures, achieving compatibility and adaptation with different visual backbone encoders. Furthermore, when the visual backbone encoder outputs a single-scale feature map, multiple extended feature maps of different resolutions are generated through downsampling and upsampling operations, and then fused with the original feature map to construct a hierarchical enhanced feature map set with rich scale information. This compensates for the lack of multi-scale information in single-scale encoders, enabling subsequent segmentation to simultaneously perceive both large-scale and small-scale buildings. When the visual backbone encoder outputs a multi-scale feature map, dimensional differences between different levels of features are eliminated through channel scale alignment, and then a top-down path is used for step-by-step fusion. This allows high-level semantic information to effectively guide low-level detailed features, and low-level detailed information to supplement the spatial accuracy of high-level semantics, thereby bridging the semantic gap between multi-scale features and generating a semantically consistent and detail-rich multi-scale fused feature set.

[0017] In other words, by enhancing the feature maps of images of different scales, the building decoder can obtain high-quality feature inputs that contain both global semantics and preserve local details, significantly improving the segmentation ability of multi-scale building targets, especially achieving excellent results in the extraction of complete regions of large-scale buildings and the accurate localization of small-scale independent dwellings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic flowchart of an embodiment of the remote sensing image building segmentation method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 3 This is a schematic flowchart of an embodiment of feature map fusion provided by the present invention; Figure 4 A schematic diagram of an embodiment of the remote sensing image building segmentation device provided by the present invention; Figure 5 This is a schematic diagram of an embodiment of the remote sensing image processing device provided by the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] It should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this invention illustrate operations implemented according to some embodiments of the invention. It should be understood that the operations in the flowcharts may be implemented out of order, and steps without logical contextual relationships may be reversed or performed simultaneously. Furthermore, those skilled in the art, guided by the content of this invention, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This invention provides a method, apparatus, and remote sensing image processing device for building segmentation in remote sensing images, which will be described below.

[0024] Figure 1 This is a schematic flowchart of an embodiment of the remote sensing image building segmentation method provided by the present invention, as shown below. Figure 1 As shown, remote sensing image building segmentation methods include: S101. Based on the visual backbone encoder, feature extraction is performed on the input raw remote sensing image to obtain the image feature map.

[0025] Specifically, the visual backbone encoder can be various variants based on the Transformer architecture, including but not limited to Visual Transformer (ViT), Swing Transformer, DeiT, PVT, CSWin Transformer, etc.

[0026] S102. Determine the scale type of the image feature map, which is either single-scale or multi-scale.

[0027] In this context, "single-scale" refers to an image feature map having a single spatial resolution, while "multi-scale" refers to an image feature map comprising multiple feature maps with different spatial resolutions.

[0028] Specifically, the image feature map output by ViT is of single scale, while the image feature map output by Swin Transformer is of multi-scale.

[0029] S103. When the image feature map is single-scale, downsampling and upsampling operations are performed on the image feature map to generate multiple extended feature maps with different resolutions. The image feature map and the multiple extended feature maps are then fused to obtain a set of hierarchical enhanced feature maps.

[0030] Downsampling reduces the spatial resolution of image feature maps by decreasing the number of pixels, thereby expanding the receptive field and extracting higher-level semantic features; upsampling increases the number of pixels and improves resolution by interpolation or deconvolution to restore spatial detail information; the combination of the two can generate a series of multi-scale feature maps with varying resolutions from low to high and complementary semantic and detail information.

[0031] S104. When the image feature map is multi-scale, perform channel scale alignment on the multi-scale image feature map to obtain multiple scale-aligned feature maps, and perform step-by-step fusion of the multiple scale-aligned feature maps based on the top-down path to obtain a multi-scale fused feature set.

[0032] The specific implementation method for channel scale alignment of multi-scale image feature maps is as follows: perform a 1×1 convolution operation on each image feature map to unify the number of channels of the image feature maps.

[0033] Meanwhile, 1×1 convolution not only achieves alignment of channel dimensions, but also performs preliminary semantic adjustments on features at different levels through learnable parameters, partially bridging the semantic gap between high-level and low-level features.

[0034] Because high-level features are semantically rich but have low resolution, while low-level features are detailed but semantically weak, there is a significant semantic gap between the two. By fusing from top to bottom, high-level semantic information can be gradually injected into low-level features, while low-level details are used to gradually correct the spatial precision of high-level features. This process can smoothly bridge the semantic differences and generate a unified feature representation that is rich in global semantics while retaining local details.

[0035] S105. Input the hierarchical enhanced feature map set or the multi-scale fused feature map set into the building decoder to obtain the building segmentation result.

[0036] It should be understood that the remote sensing image building segmentation method in this embodiment of the invention can be implemented in any device based on the remote sensing image building segmentation method, such as a remote sensing satellite ground data processing station, a UAV remote sensing image real-time processing system, or an urban geographic information service platform. Specifically, the remote sensing image building segmentation method is stored in the aforementioned device as a pre-programmed program. When the device starts up or receives a remote sensing image, the program is invoked, the remote sensing image building segmentation method is implemented, and multi-scale feature enhancement and building segmentation are performed on the input image.

[0037] Compared with existing technologies, the remote sensing image building segmentation method provided in this invention determines the scale type of the image feature map and then adopts a targeted feature enhancement strategy based on the scale type. This eliminates the need for manual adaptation to specific encoder architectures, achieving compatibility and adaptation with different visual backbone encoders. Furthermore, when the visual backbone encoder outputs a single-scale feature map, it generates multiple extended feature maps of different resolutions through downsampling and upsampling operations, which are then fused with the original feature map to construct a hierarchical enhanced feature map set rich in scale information. This compensates for the lack of multi-scale information in single-scale encoders, enabling subsequent segmentation to simultaneously perceive both large-scale and small-scale buildings. When the visual backbone encoder outputs a multi-scale feature map, it eliminates dimensional differences between different levels of features through channel scale alignment, and then performs step-by-step fusion using a top-down path. This allows high-level semantic information to effectively guide low-level detailed features, while low-level detailed information supplements the spatial accuracy of high-level semantics, thereby bridging the semantic gap between multi-scale features and generating a semantically consistent and detail-rich multi-scale fused feature set.

[0038] In other words, by enhancing the feature maps of images of different scales, the building decoder can obtain high-quality feature inputs that contain both global semantics and preserve local details, significantly improving the segmentation ability of multi-scale building targets, especially achieving excellent results in the extraction of complete regions of large-scale buildings and the accurate localization of small-scale independent dwellings.

[0039] Specifically, the results of building segmentation from remote sensing images have the following practical applications: First, in the fields of urban planning and land management, low accuracy in building segmentation can lead to false positives or false negatives, resulting in the failure to identify illegal buildings or the misidentification of compliant buildings as illegal. This makes it difficult for law enforcement officers to accurately locate targets, increases on-site verification costs, and may even lead to legal disputes. In urban renewal projects, it is necessary to accurately calculate the base area and total building area of ​​existing buildings. Low segmentation accuracy can lead to overestimation or underestimation of the area, directly affecting the economic calculation of the development plan and the developer's expected return on investment.

[0040] Secondly, in the field of disaster emergency response, such as after an earthquake, it is necessary to quickly identify collapsed and damaged buildings. Low segmentation accuracy can lead to missed detections (collapsed buildings are not identified, resulting in insufficient allocation of rescue resources) or false detections (non-collapsed buildings are marked as collapsed, wasting rescue efforts), directly threatening the golden rescue time for trapped personnel.

[0041] Third, in the field of infrastructure monitoring, such as in railway / power corridor inspection scenarios, there is a risk that buildings along the line may threaten the safety of the line (e.g., illegal buildings near high-voltage lines). Low segmentation accuracy can lead to missed detections, causing line tripping or geological disaster risks; false detections will increase unnecessary on-site line inspection work, consuming manpower and resources.

[0042] In some embodiments of the present invention, such as Figure 2 As shown, step S102 includes: S201, Detect the output data structure of the visual backbone encoder.

[0043] S202. If the output data structure is a single feature map tensor, then the scale type is determined to be single scale. S203. If the output data structure is an ordered set containing multiple feature map tensors, and at least two of the multiple feature map tensors have different spatial resolutions, then the scale type is determined to be multi-scale.

[0044] Specifically, the image feature map is a four-dimensional tensor, with dimensions represented as B×C×H×W, where B is the batch size, C is the number of feature channels, H is the feature map height, and W is the feature map width.

[0045] In practical implementation, the above recognition process can be accomplished by determining whether the output data structure is a container type (such as a list or tuple) and obtaining its container length (i.e., the number of feature maps). For example, if the length is greater than 1 and the spatial dimensions of each element are different, it is multi-scale; otherwise, it is single-scale.

[0046] The size type adaptive recognition mechanism proposed in this invention does not require manual specification of the encoder type and can dynamically accommodate the output structures of different vision backbone encoders.

[0047] In some embodiments of the present invention, step S103, which involves performing downsampling and upsampling operations on the image feature maps to generate multiple extended feature maps with different resolutions, includes: Multiple max pooling downsampling operations are performed on the image feature map to obtain multiple downsampled feature maps.

[0048] Multiple transposed convolutional upsampling operations are performed on the image feature maps to obtain multiple upsampled feature maps; Multiple upsampled feature maps and multiple downsampled feature maps together form multiple extended feature maps.

[0049] Specifically, max pooling effectively preserves the salient response of features by retaining the maximum activation values ​​in local regions, while expanding the receptive field and extracting higher-level semantic information. Transposed convolution, compared to traditional bilinear interpolation, has learnable parameters and can adaptively learn upsampling methods based on data, generating more expressive, high-resolution features.

[0050] In a specific embodiment of the present invention, the convolution kernel used for max pooling downsampling and transposed convolution upsampling is 2×2, and the stride is 2.

[0051] The size of the downsampled feature map obtained after one max pooling downsampling is B×C×H / 2×W / 2, and the size of the downsampled feature map obtained after two max pooling downsampling is B×C×H / 4×W / 4.

[0052] The size of the upsampled feature map obtained after one transposed convolution upsampling is B×C×2H×2W, and the size of the upsampled feature map obtained after two transposed convolution upsampling is B×C×4H×4W.

[0053] In a specific embodiment of the present invention, the image feature map F_single is subjected to two max pooling downsampling operations and two transposed convolution upsampling operations to obtain F_down1, F_down2, F_up1 and F_up2, respectively.

[0054] To ensure the uniformity of feature dimensions, in some embodiments of the present invention, step S103 involves fusing the image feature map and multiple extended feature maps to obtain a hierarchical enhanced feature map set, including: Channel adjustment is performed on the image feature map and each extended feature map to obtain multiple channel-adjusted feature maps with the same number of channels. These multiple channel-adjusted feature maps are then concatenated along the channel dimension to obtain a set of hierarchical enhanced feature maps.

[0055] Specifically, a 1×1 convolution is performed on the image feature map and multiple extended feature maps to unify the number of channels in both. In other words, the number of channels in the multiple channel-adjusted feature maps is the same, and the resolution is a multiple of 2 in each direction.

[0056] To further improve the quality of feature representation, in some embodiments of the present invention, after obtaining the hierarchical enhanced feature map set, the method further includes: Cross-scale interactive enhancement is achieved by using a cross-scale attention mechanism to adjust feature maps of different resolutions in multiple channels of a hierarchical enhanced feature map set.

[0057] When the hierarchical enhancement feature map set contains channel-adjusted feature maps at the original resolution, 2x upsampling, 4x upsampling, 1 / 2 downsampling, and 1 / 4 downsampling resolutions... The specific implementation of the cross-scale attention mechanism is as follows: The feature map with the highest resolution (e.g., a large-scale, fine-grained feature map upsampled by 4 times) is selected as the query feature, and its spatial dimensions are flattened into a sequence, denoted as Q. The feature map with the lowest resolution (e.g., a small-scale semantic feature map downsampled by 4 times) is selected as the key-value feature, and similarly flattened into a sequence, denoted as K. Next, the cross-scale attention weight matrix is ​​calculated:

[0058] In the formula, A represents the correlation between each pixel in the high-resolution feature map and all pixels in the low-resolution feature map. This attention weight is used as the weight to perform weighted aggregation on the value feature V, resulting in the enhanced high-resolution feature Q=AV, which is then restored to its original spatial shape.

[0059] Similarly, to achieve bidirectional information flow, low-resolution features can be used as queries and high-resolution features as keys for symmetrical cross-scale interaction. Finally, the enhanced feature maps at each scale are residually joined with the original non-interacting feature maps (element-by-element addition) to obtain the final set of cross-scale enhanced feature maps.

[0060] This invention introduces a cross-scale attention mechanism. Through this mechanism, high-level features can guide low-level features to selectively enhance building areas, while low-level features can supplement edge detail information for high-level features. This achieves cross-scale interaction and enhancement of features, thereby improving the overall representation quality of the hierarchical enhanced feature map set.

[0061] In some embodiments of the present invention, the plurality of scale-aligned feature maps include first scale-aligned feature maps with resolutions ranging from low to high. Second-scale aligned feature map Align feature maps with the third scale In step S104, multiple scale-aligned feature maps are fused step-by-step based on a top-down path to obtain a multi-scale fused feature set, including: Align the feature map at the first scale. Perform an upsampling operation to obtain the first upsampled aligned feature map. ; Align the first upsampled feature map Second-scale aligned feature map Perform fusion to obtain the first fused feature map. P1 ; The first fused feature map P1 Perform an upsampling operation to obtain a second upsampled aligned feature map. P1_up ; Align the second upsampled feature map P1_upAlign feature maps with the third scale Perform fusion to obtain a second fused feature map. P2 ; First fusion feature map P1 Second fusion feature map P2 This constitutes a multi-scale fusion feature set.

[0062] The feature maps can be fused by element-wise addition or by concatenating channels and then integrating them through a convolutional layer. This embodiment preferably uses element-wise addition, which is computationally simple and can effectively preserve the original distribution of features.

[0063] The embodiments of the present invention construct a multi-scale fusion feature set that is rich in semantic information and preserves spatial details through a top-down, step-by-step fusion approach.

[0064] To further optimize the fusion effect, in some embodiments of the present invention, such as Figure 3 As shown, the first upsampled aligned feature map and the second scale aligned feature map are fused to obtain the first fused feature map, including: S301. Determine the fusion weights of the first upsampled aligned feature map and the second scale aligned feature map.

[0065] The fusion weights can be determined by using a pre-built lightweight weight determination network model.

[0066] S302. Based on the fusion weight, the first upsampled aligned feature map and the second scale aligned feature map are fused to obtain the first fused feature map.

[0067] The embodiments of the present invention achieve more refined feature fusion by setting a dynamic determination mechanism for fusion weights, which can adaptively select which regions rely more on high-level semantics and which regions rely more on low-level details based on the input content.

[0068] Similarly, the process of fusing the second upsampled aligned feature map and the third scale aligned feature map to obtain the second fused feature map can also refer to steps S301-S302, and will not be elaborated here.

[0069] In summary, the remote sensing image building segmentation method proposed in this invention has the following technical effects: It intelligently identifies the output feature structure of the visual backbone encoder through adaptive routing, constructs a feature pyramid by pooling downsampling and transposed convolution upsampling of single-scale features, and fuses multi-scale features from top to bottom after 1×1 convolution alignment, achieving semantically aligned and scale-normalized feature pyramid construction. Experimental results show that this method significantly improves the building extraction accuracy on the WHU and INRIA datasets, demonstrating good cross-scene generalization ability and practical value.

[0070] On the other hand, embodiments of the present invention also provide a remote sensing image building segmentation device, such as... Figure 4 As shown, the remote sensing image building segmentation device 400 includes: The feature extraction unit 401 is used to extract features from the input raw remote sensing image based on the visual backbone encoder to obtain an image feature map; The scale type determination unit 402 is used to determine the scale type of the image feature map, which is either single-scale or multi-scale. The single-scale processing branch unit 403 is used to perform downsampling and upsampling operations on the image feature map when the image feature map is single-scale, generate multiple extended feature maps with different resolutions, and fuse the image feature map and multiple extended feature maps to obtain a set of hierarchical enhanced feature maps. The multi-scale processing branch unit 404 is used to perform channel scale alignment on the multi-scale image feature map when the image feature map is multi-scale, to obtain multiple scale-aligned feature maps, and to perform step-by-step fusion of the multiple scale-aligned feature maps based on a top-down path to obtain a multi-scale fused feature set. The building segmentation unit 405 is used to input the hierarchical enhanced feature map set or the multi-scale fused feature map set into the building decoder to obtain the building segmentation result.

[0071] The remote sensing image building segmentation device 400 provided in the above embodiments can realize the technical solutions described in the above remote sensing image building segmentation method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above remote sensing image building segmentation method embodiments, and will not be repeated here.

[0072] like Figure 5 As shown, the present invention also provides a remote sensing image processing device 500. The remote sensing image processing device 500 includes an image receiver 501, a processor 502, a memory 503, and a display 504. Figure 5 Only some components of the remote sensing image processing device 500 are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0073] Image receiver 501 is used to receive raw remote sensing images acquired by the remote sensing image acquisition device.

[0074] Remote sensing image acquisition devices can be various types of equipment capable of acquiring ground-based remote sensing images, such as high-resolution remote sensing satellites, multispectral / hyperspectral cameras mounted on UAVs or manned aircraft, airborne digital scanners, and ground-based remote sensing test platforms. These acquisition devices acquire raw remote sensing image data through optical or radar methods and transmit it to image receivers for subsequent processing.

[0075] In some embodiments, processor 502 may be a microcontroller in a remote sensing image processing device, used to run program code stored in memory 503 or process data, such as the remote sensing image building segmentation method of the present invention.

[0076] In some embodiments, memory 503 may be an internal storage unit of the remote sensing image processing device 500, such as a hard disk or memory of the remote sensing image processing device 500. In other embodiments, memory 503 may also be an external storage device of the remote sensing image processing device 500, such as a pluggable hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the remote sensing image processing device 500.

[0077] Furthermore, the memory 503 may include both internal storage units and external storage devices of the remote sensing image processing device 500. The memory 503 is used to store application software and various types of data installed on the remote sensing image processing device 500.

[0078] In some embodiments, display 504 may be an LED display, a liquid crystal display, a touch-screen liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 504 is used to display information from the remote sensing image processing device 500 and to display a visual user interface. Components 501-504 of the remote sensing image processing device 500 communicate with each other via a bus.

[0079] In some embodiments of the present invention, when the processor 502 executes the remote sensing image building segmentation program in the memory 503, the following steps can be implemented: The original remote sensing image is used as a visual backbone encoder to extract features and obtain an image feature map. Determine the scale type of the image feature map, whether it is single-scale or multi-scale; When the image feature map is single-scale, downsampling and upsampling operations are performed on the image feature map to generate multiple extended feature maps with different resolutions. The image feature map and multiple extended feature maps are then fused to obtain a set of hierarchical enhanced feature maps. When the image feature map is multi-scale, channel scale alignment is performed on the multi-scale image feature map to obtain multiple scale-aligned feature maps. Then, the multiple scale-aligned feature maps are fused step by step based on a top-down path to obtain a multi-scale fused feature set. The hierarchical enhanced feature map set or the multi-scale fused feature map set is input into the building decoder to obtain the building segmentation result.

[0080] It should be understood that when the processor 502 executes the building segmentation program of the base remote sensing image in the memory 503, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0081] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0082] The present invention provides a detailed description of a remote sensing image building segmentation method, apparatus, and remote sensing image processing device. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for building segmentation in remote sensing images, characterized in that, include: The original remote sensing image is used as a visual backbone encoder to extract features and obtain an image feature map. Determine the scale type of the image feature map, wherein the scale type is single-scale or multi-scale; When the image feature map is single-scale, downsampling and upsampling operations are performed on the image feature map to generate multiple extended feature maps with different resolutions. The image feature map and the multiple extended feature maps are then fused to obtain a set of hierarchical enhanced feature maps. When the image feature map is multi-scale, channel scale alignment is performed on the multi-scale image feature map to obtain multiple scale-aligned feature maps, and the multiple scale-aligned feature maps are fused step by step based on a top-down path to obtain a multi-scale fused feature set. The hierarchical enhanced feature map set or the multi-scale fused feature map set is input into the building decoder to obtain the building segmentation result.

2. The remote sensing image building segmentation method according to claim 1, characterized in that, The determination of the scale type of the image feature map includes: Detect the output data structure of the visual backbone encoder; If the output data structure is a single feature map tensor, then the scale type is determined to be single scale; If the output data structure is an ordered set containing multiple feature map tensors, and at least two of the multiple feature map tensors have different spatial resolutions, then the scale type is determined to be multi-scale.

3. The remote sensing image building segmentation method according to claim 1, characterized in that, The step of performing downsampling and upsampling operations on the image feature maps to generate multiple extended feature maps with different resolutions includes: The image feature map is downsampled multiple times using max pooling to obtain multiple downsampled feature maps; The image feature map is subjected to multiple transposed convolutional upsampling to obtain multiple upsampled feature maps; The plurality of upsampled feature maps and the plurality of downsampled feature maps constitute the plurality of extended feature maps.

4. The remote sensing image building segmentation method according to claim 3, characterized in that, The max pooling downsampling and the transposed convolution upsampling use a 2×2 convolution kernel with a stride of 2.

5. The remote sensing image building segmentation method according to claim 1, characterized in that, The process of fusing the image feature map and the multiple extended feature maps to obtain a hierarchical enhanced feature map set includes: The image feature map and each of the extended feature maps are channel-adjusted to obtain multiple channel-adjusted feature maps with the same number of channels. The multiple channel-adjusted feature maps are then concatenated along the channel dimension to obtain the set of hierarchical enhanced feature maps.

6. The remote sensing image building segmentation method according to claim 5, characterized in that, After obtaining the hierarchical augmentation feature map set, it also includes: Cross-scale interactive enhancement is achieved by adjusting feature maps of different channels in the hierarchical enhanced feature map set using a cross-scale attention mechanism.

7. The remote sensing image building segmentation method according to claim 1, characterized in that, The multiple scale-aligned feature maps include a first scale-aligned feature map, a second scale-aligned feature map, and a third scale-aligned feature map with resolutions from high to low. These multiple scale-aligned feature maps are then fused step-by-step based on a top-down path to obtain a multi-scale fused feature set, including: The first scale-aligned feature map is upsampled to obtain the first upsampled aligned feature map; The first upsampled aligned feature map and the second scale aligned feature map are fused to obtain the first fused feature map; The first fused feature map is upsampled to obtain the second upsampled aligned feature map; The second upsampled aligned feature map and the third scale aligned feature map are fused to obtain a second fused feature map; The first fused feature map and the second fused feature map constitute the multi-scale fused feature set.

8. The remote sensing image building segmentation method according to claim 7, characterized in that, The step of fusing the first upsampled aligned feature map and the second scale aligned feature map to obtain a first fused feature map includes: Determine the fusion weights of the first upsampled aligned feature map and the second scale aligned feature map; The first upsampled aligned feature map and the second scale aligned feature map are fused based on the fusion weights to obtain a first fused feature map.

9. A remote sensing image building segmentation device, characterized in that, include: The feature extraction unit is used to extract features from the input raw remote sensing image based on the visual backbone encoder to obtain an image feature map; A scale type determination unit is used to determine the scale type of the image feature map, wherein the scale type is single-scale or multi-scale; A single-scale processing branch unit is used to perform downsampling and upsampling operations on the image feature map when the image feature map is single-scale, generate multiple extended feature maps with different resolutions, and fuse the image feature map and the multiple extended feature maps to obtain a set of hierarchical enhanced feature maps. The multi-scale processing branch unit is used to perform channel scale alignment on the multi-scale image feature map when the image feature map is multi-scale, to obtain multiple scale-aligned feature maps, and to perform step-by-step fusion of the multiple scale-aligned feature maps based on a top-down path to obtain a multi-scale fused feature set. The building segmentation unit is used to input the hierarchical enhanced feature map set or the multi-scale fused feature map set into the building decoder to obtain the building segmentation result.

10. A remote sensing image processing device, characterized in that, Includes an image receiver, processor, and memory, among which, The image receiver is used to receive raw remote sensing images acquired by the remote sensing image acquisition device; The memory is used to store programs; The processor, coupled to the memory and the image receiver, is used to execute the program stored in the memory to implement the steps in the remote sensing image building segmentation method according to any one of claims 1 to 8.

Images