A remote sensing image target segmentation method and network based on dilation multi-scale fusion

Abstract

The present invention discloses a remote sensing image target segmentation method and network based on dilated multi-scale fusion, which belongs to the field of remote sensing image processing technology. The method includes: preprocessing and feature map extraction of the original remote sensing image; processing the extracted feature map using dilated convolution, attention mechanism and attention matrix to obtain an enhanced feature map; performing multi-scale feature extraction and fusion on the enhanced feature map, and outputting a target segmentation result with the same resolution as the original remote sensing image through a decoder. By combining dilated convolution and multi-scale feature fusion, the present invention can effectively improve the segmentation accuracy of target areas in remote sensing images, especially in complex scenes and low-contrast areas. By applying an efficient attention mechanism and feature enhancement in the feature extraction process, the performance of the segmentation model is further optimized, so that it can run efficiently under different computing resources, and has broad application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image target segmentation method and network based on expansion multi-scale fusion. Background Art

[0002] With the rapid development of remote sensing technology, the application of remote sensing images has been widely applied in many fields, including geographic information systems, environmental monitoring, agriculture, and urban planning. Remote sensing images usually have complex scenes and high-dimensional features. They not only contain rich geographic information, but also involve target objects of different scales and types. In these images, the boundaries of the targets are often blurred and affected by factors such as noise, low contrast, and complex backgrounds, making the image segmentation task more difficult. Traditional image segmentation methods, such as those based on thresholding, edge detection, and region growing, although effective in some cases, often perform poorly when dealing with complex backgrounds, low contrast, or small targets, and have limited ability to segment multi-scale targets.

[0003] In recent years, deep learning methods, especially convolutional neural networks (CNNs), have made significant progress in the field of image segmentation. By automatically learning high-level features of images, deep learning can overcome the limitations of traditional methods to a certain extent, especially in complex image scenes. However, existing deep learning methods still have some problems, especially in remote sensing images. Due to the large differences in target size, complex backgrounds, and the large number of low-contrast areas, the model has difficulty in effectively processing features of different scales. In addition, the current mainstream convolution operations mostly rely on fixed-size convolution kernels, which easily ignore long-range dependencies with important information in the image, resulting in unsatisfactory segmentation results for some small targets or low-contrast areas.

[0004] Therefore, how to improve the target segmentation accuracy of remote sensing images is a technical problem that those skilled in the art need to solve urgently. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a remote sensing image target segmentation method and network based on dilated multi-scale fusion. This method enhances the perception ability of complex terrain and diverse targets by combining dilated convolution with attention mechanism, and can effectively improve the segmentation accuracy of target areas in remote sensing images, especially in complex scenes and low-contrast areas.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] In a first aspect, an embodiment of the present invention provides a remote sensing image target segmentation method based on dilation multi-scale fusion, the method comprising the following steps:

[0008] Step 1: Preprocess the original remote sensing image and extract feature maps;

[0009] Step 2: Process the extracted feature map using dilated convolution, attention mechanism, and attention matrix to obtain an enhanced feature map;

[0010] Step 3: Perform multi-scale feature extraction and fusion on the enhanced feature map, and output the target segmentation result with the same resolution as the original remote sensing image through the decoder.

[0011] Furthermore, in step 1, the original remote sensing image is preprocessed and feature maps are extracted. The specific process includes:

[0012] Step 1.1: Preprocess the original remote sensing image, including normalization, image enhancement and denoising, contrast enhancement and detail enhancement;

[0013] Step 1.2: Combine the convolutional neural network and the transformer, and take advantage of the diffusion model and pre-trained model to extract features from the pre-processed remote sensing image.

[0014] Step 1.3: The extracted features are fused through the channel dimension, and then the spatial resolution is restored to generate a feature map F with spatial details and global context information suitable for image analysis tasks.

[0015] Furthermore, in step 2, the extracted feature map is processed using dilated convolution, attention mechanism, and attention matrix to obtain an enhanced feature map; the specific process includes:

[0016] Step 2.1: The feature map F obtained in step 1 is passed through the normalization layer and then expanded through multiple dilated convolution layers to enhance the fusion of local and global information and obtain the feature map after the dilated convolution operation.

[0017] Step 2.2: Perform multi-path feature fusion. This process includes upper and lower branches, where:

[0018] Upper branch: First, the feature map Perform global average pooling to obtain global context information; then perform nonlinear transformation through the spiking neuron layer, and enhance the expressiveness of the feature map through the activation function;

[0019] Lower branch: contains two layers of dilated convolution, after which activation function is applied to maintain the validity and nonlinear representation of the output;

[0020] Finally, the outputs of the upper and lower branches are fused through addition operations to obtain the fused feature map

[0021] Step 2.3: Fuse the feature map After processing through a normalization layer and three convolutional layers, we obtain the query, key, and value respectively;

[0022] Step 2.4: Construct a fixed attention matrix;

[0023] Step 2.5: Perform weighted calculation on the attention matrix and feature map to obtain the adjusted feature map

[0024] Step 2.6: Adjust the feature map and fusion feature map Perform residual connection to obtain the final enhanced feature map Used for subsequent remote sensing image target segmentation tasks.

[0025] Furthermore, in step 3, multi-scale feature extraction and fusion are performed on the enhanced feature map, and the decoder outputs the target segmentation result with the same resolution as the original remote sensing image. The specific process includes:

[0026] Step 3.1: Enhance the feature map obtained in step 2 Perform multi-scale convolution operations and use different convolution kernel sizes to obtain feature maps of multiple scales: and fuse these multi-scale feature maps by weighted averaging or splicing to obtain a feature map containing local features

[0027] Step 3.2: Feature Map Perform global average pooling to obtain global features Then the global features and feature maps containing local features Splice to obtain the final feature map

[0028] Step 3.3: Pass the final feature map through the decoder Converted into target segmentation results with the same resolution as the original remote sensing image.

[0029] Furthermore, in step 3, the cross entropy loss and the Dice loss are combined to construct a total loss function in a weighted manner to perform remote sensing image target segmentation training.

[0030] Furthermore, the total loss function constructed is:

[0031]

[0032] in, Indicates total loss; represents the cross entropy loss; represents Dice loss; α and β represent the weight coefficients of cross entropy loss and Dice loss, respectively.

[0033] In a second aspect, the present invention further provides a remote sensing image target segmentation network based on dilation multi-scale fusion, which applies the above-mentioned remote sensing image target segmentation method based on dilation multi-scale fusion to achieve remote sensing image target segmentation. The network includes:

[0034] Image preprocessing and feature map extraction module, used for preprocessing and feature map extraction of original remote sensing images;

[0035] The dilated multi-scale fusion module is used to process the extracted feature maps using dilated convolution, attention mechanism and attention matrix to obtain enhanced feature maps;

[0036] The target segmentation module is used to extract and fuse multi-scale features of the enhanced feature map, and output the target segmentation result with the same resolution as the original remote sensing image through the decoder.

[0037] In a third aspect, the present invention also provides an electronic device comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned remote sensing image target segmentation method based on dilation multi-scale fusion.

[0038] Compared with the prior art, the present invention has at least the following beneficial effects:

[0039] 1) Improved segmentation accuracy: By introducing dilated convolution and multi-scale feature fusion, the present invention enhances the model's ability to perceive local details and global context when processing complex remote sensing images, especially in the segmentation of low-contrast areas and small targets.

[0040] 2) Enhanced multi-scale feature extraction capability: The present invention combines dilated convolution with the attention mechanism to effectively capture the features of remote sensing images at multiple scales, improving the model’s perception of complex terrain and targets of different scales, thereby improving segmentation accuracy and robustness, especially in complex scenarios.

[0041] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0042] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0045] Figure 1 A schematic flow chart of a remote sensing image target segmentation method based on dilation multi-scale fusion provided in an embodiment of the present invention.

[0046] Figure 2 A schematic diagram of the working principle of the expansion multi-scale fusion module provided in an embodiment of the present invention.

[0047] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0049] In describing the present invention, it should be noted that some processes described in this specification and accompanying drawings include multiple operations that appear in a specific order. However, it should be understood that these operations may be performed in a different order than the order in which they appear, or may be performed in parallel. Furthermore, the use of various sequence numbers is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0051] See also Figure 1 As shown, the present invention provides a remote sensing image target segmentation method based on dilation multi-scale fusion, which aims to improve the accuracy of remote sensing image target segmentation in complex scenes. The specific implementation and working principle of the method of the present invention are as follows:

[0052] Step 1: Preprocess the original remote sensing image and extract the feature map; the specific process is as follows:

[0053] Step 1.1: Preprocessing and enhancing the input image quality: Remote sensing images are usually multi-channel images (such as RGB, depth, or multispectral) with dimensions of H×W×C, where H is the height of the image, W is the width of the image, and C is the number of channels. To ensure efficient processing and improve the performance of subsequent networks, the original remote sensing image is first preprocessed, including:

[0054] ① Normalization: Normalize the pixel values ​​and map them to a fixed range (such as [0, 1] or [-1, 1]) to ensure the numerical stability of the data and make it suitable for subsequent models.

[0055] ② Image Enhancement and Denoising: The Mamba model (adaptive feature enhancement module) removes noise and enhances the input image. Mamba enhances local image features and optimizes the visibility of distant and small objects, thereby improving the model's ability to discern complex patterns and details in remote sensing imagery. This module utilizes deep learning-based image enhancement strategies to suppress noise in areas with low signal-to-noise ratios while preserving valid features.

[0056] ③ Contrast enhancement and detail enhancement: Combining CLAHE (adaptive histogram equalization) and the enhancement method based on the Diffusion model, the local contrast of the image is enhanced, especially in low-contrast areas, thereby improving the ability to express details and providing richer input data for subsequent feature extraction.

[0057] Step 1.2: Feature Extraction (Combining Convolution and Transformer Architectures). Feature extraction combines the convolutional neural network (CNN) and transformer architectures, while incorporating the advantages of the diffusion model and pre-trained models to make feature extraction more efficient and accurate. Specifically, it includes:

[0058] ① Preliminary convolutional feature extraction: The input image passes through a preliminary convolutional layer (using a smaller convolution kernel, such as a 3×3 convolution kernel) to extract low-level features, capturing local information such as edges and textures. These convolutional layers gradually reduce the spatial size and increase the number of channels through multiple downsampling (such as convolution with a stride of 2), providing effective low-level features for subsequent modules.

[0059] ② Diffusion-enhanced feature extraction: Based on the traditional convolutional layer, a diffusion model is added. The diffusion model enhances the local and global information of the feature map by simulating the diffusion process of the image. Specifically, diffusion strengthens feature representation by gradually transferring and updating pixel information. Especially when processing complex images (such as the variable terrain and object structures in remote sensing images), it can effectively capture details and improve the recognition of textures and contours.

[0060] ④ Transformer: The features extracted by convolution are further processed using the Visual Transformer (ViT) architecture. First, the output feature map of the convolutional layer is divided into image blocks of fixed size, and each image block is flattened into a one-dimensional vector. Then, the features of each image block are mapped to a fixed dimension through linear transformation, and position encoding is added to preserve the spatial position relationship of the blocks. These features are then input into the Transformer encoder for deep learning, capturing the global and local feature dependencies of the image, and using the self-attention mechanism to efficiently model long-range dependencies.

[0061] ⑤ Pre-training and fine-tuning: To improve the model's learning efficiency and robustness, we introduced a pre-trained Transformer model trained on a large-scale remote sensing dataset. By fine-tuning the pre-trained model, the model can quickly adapt to specific task requirements and effectively improve the quality and speed of feature extraction. The pre-trained model enables the network to learn rich visual representations from a large amount of image data, reducing training time and avoiding overfitting.

[0062] Step 1.3: Feature fusion and high-resolution restoration. To maximize the advantages of convolution and transformer, the extracted features are fused along the channel dimension and then the spatial resolution is restored for further processing. Specifically,

[0063] ① Convolution and Transformer Feature Fusion: The output feature maps of the convolutional layer and the Transformer encoder are concatenated in the channel dimension (Concat operation) to obtain a fused feature map. This preserves both the local details captured by the convolutional layer and the global context learned by the transformer, making the feature map richer and more comprehensive.

[0064] ② Super-resolution restoration: Perform high-resolution restoration on the fused feature maps. Using a convolutional super-resolution network (such as ESRGAN or deconvolution layers), the feature maps are upsampled to restore their spatial resolution, ensuring that finer local information and details are captured when restoring the image. This step not only improves the visual quality of the image but also strengthens the reliance of subsequent tasks (such as object detection and classification) on high-resolution images.

[0065] ③ Multi-scale feature fusion: After the final upsampling operation, the image's expressiveness is further enhanced through cross-scale feature fusion, combining feature information at multiple scales. This process introduces multiple layers of feature maps and performs weighted fusion of features at different scales, enabling the model to more comprehensively process image regions of varying resolutions and enhance its ability to capture image details.

[0066] Step 1.4: Output the final feature map. Through the above steps, the feature map F is finally generated. The feature map F has strong spatial details and global context information, which is suitable for subsequent image analysis tasks.

[0067] Step 2: Construct an expansion multi-scale fusion module and use expansion convolution, attention mechanism and attention matrix to process the extracted feature map F to obtain the enhanced feature map like Figure 2 As shown, specifically including:

[0068] Step 2.1: Dilated convolution operation. The goal of this step is to expand the input feature map through dilated convolution, increase the receptive field, and enable the model to capture a wider range of context information. Especially for complex scenes in remote sensing images, dilated convolution can effectively enhance the fusion of local and global information. The feature map F obtained in step 1 is passed through the normalization layer and then through three dilated convolution layers to obtain

[0069] Then multiply Q and K to get Where g represents the grouping factor of the number of channels.

[0070] then Multiplying by V, we get the feature map of the application of dilated self-attention, which is then fused with F by skip link to get

[0071]

[0072] in, represents matrix multiplication, Represents matrix addition.

[0073] Step 2.2: Multi-path fusion of feature maps. After obtaining the feature map after the dilated convolution operation After that, multi-path feature fusion is performed. This process includes upper and lower branches, which aims to extract information at different levels and fuse them to improve the final feature representation capability. Among them:

[0074] Upper branch: First A global average pooling (GAP) operation is performed to obtain global context information. Then, a nonlinear transformation is performed through a spiking neuron layer (SNN), and the activation function (ReLU) is used to further enhance the expressiveness of the feature map.

[0075] Lower branch: The lower branch contains two layers of dilated convolutions to further extract detailed information from the image. A sigmoid activation function is applied after each dilated convolution layer to maintain the validity and nonlinear representation of the output.

[0076] Fusion operation: The outputs of the upper branch and the lower branch are fused through the addition operation to obtain the final fusion feature map

[0077]

[0078] Among them, SNN represents the spiking neuron layer, BN represents the normalization layer, GAP represents the global average pooling layer, DConv represents the dilated convolution layer, and Sigmoid represents the Sigmoid activation function.

[0079] Step 2.3: Feature Map After the normalization layer, and then through three convolutional layers to obtain M represents the size of the receptive field. The appropriate M value is designed and selected based on the computational requirements and efficiency requirements of the model.

[0080] Step 2.4: Construct a fixed attention matrix. This is used to limit each query to focus on certain pixel positions in the key or value, specifically pixels at even coordinate positions. The core design idea of ​​this matrix is ​​to selectively mask certain pixels by controlling their relative positions, thereby effectively simulating the effect of dilated convolution while also meeting certain "blind spot" requirements. First, define a smaller binary matrix Used to determine whether attention is allowed. The value of this matrix depends on the relative position (x i -x j ,y i -y j ), and only allows queries to focus on pixel locations with even coordinate differences in the key value. The specific definition is as follows:

[0081]

[0082] Here, x and y are relative coordinate differences, which are assigned 0 when the conditions are met, and 1 otherwise. Next, using this binary matrix Can be further constructed A bigger M 2 ×M 2 Matrix. The specific construction rules are as follows:

[0083]

[0084] Among them, (x i ,y i ) and (x j ,y j ) are the positions of the query and key value respectively. If the relative position difference between the query and key value on some coordinate axis is an even number (i.e., it satisfies the condition), then Indicates that the attention values ​​between these positions will not be affected; otherwise, When calculated through softmax, the corresponding attention value will be masked and become 0. The purpose of this design is to make the query only focus on the pixel positions with even coordinate differences in the key, thereby mimicking the behavior of dilated convolution. Dilated convolution itself captures more contextual information by increasing the spacing between convolution kernels, but at the same time retains a certain "blind spot" - that is, it does not pay attention to all positions. In this way, the network model can control local relationships while maintaining global perception capabilities, avoiding excessive interference and noise. This design is very useful in certain visual tasks (especially image processing tasks) because it can enhance the model's selective processing of local and global information. This M 2 ×M 2 The matrix is ​​obtained by This process ensures that the model's queries focus only on pixels at even-numbered key coordinates, mimicking the behavior of dilated convolutions while maintaining a certain degree of localization and selective attention. This design helps improve the model's efficiency and enhances its ability to learn specific spatial structures.

[0085] Step 2.5: Weighted calculation of attention matrix and feature map. After constructing the attention matrix, perform matrix multiplication on it with query (Q') and key (K'), and multiply the result with value (V') to get the adjusted feature map. This process simulates the effect of dilated convolution through the design of the attention matrix, allowing the model to effectively learn the relationship between local and global features while avoiding excessive interference and noise.

[0086] Step 2.6: Residual connection and output feature map. Finally, the feature map obtained by the attention mechanism With the fused feature map Perform residual connections. The purpose of this step is to preserve the flow of information through skip connections while improving the expressiveness of the final features.

[0087]

[0088] Final feature map This feature map is used in subsequent remote sensing image segmentation tasks. It contains multi-scale information and local context features, which can effectively improve segmentation accuracy, especially when processing complex remote sensing image scenes.

[0089] Through the above steps, the present invention constructs a dilated multiscale fusion module. This module utilizes dilated convolution and an attention mechanism to combine local details with global contextual information. It also designs an attention matrix to simulate the sparse perception properties of dilated convolution. These innovative designs enable the module to more effectively process complex features in remote sensing images, improving segmentation performance and computational efficiency. It is particularly suitable for efficient feature extraction and image structure modeling in remote sensing image segmentation tasks.

[0090] Step 3: From feature map to remote sensing image target segmentation result, the details are as follows:

[0091] Step 3.1: Multi-scale feature extraction and fusion. The feature map obtained from step 2 Extract information of different scales to enhance the network's ability to distinguish, especially when processing target images with diverse scales. Use convolution kernels of different sizes to extract multi-scale features. Perform multi-scale convolution on the feature map, using different convolution kernel sizes {k1, k2, ..., k n}, get feature maps of multiple scales:

[0092]

[0093] in, Indicates the application of convolution kernel size k i Next, in order to fuse these multi-scale feature maps, weighted averaging or splicing is used:

[0094]

[0095] Among them, w i is the weight of each scale feature map, which can be learned through training or set based on experience.

[0096] Step 3.2: In order to further capture the contextual information of the target area in the remote sensing image, a global information modeling method is used to improve the model's ability to distinguish between background and target. Global average pooling is used to extract the global contextual information of the image. Perform global average pooling to obtain global features:

[0097]

[0098] in, is a global feature vector that contains the global context information of the image. Next, the global context information is fused with the local features to enhance the model’s ability to segment objects of different scales and types. After processing, it is compared with the To splice:

[0099]

[0100] Among them, Concat represents the concatenation operation.

[0101] Step 3.3: Predict the target segmentation result. The final feature map is decoded Convert the result of target segmentation to the same resolution as the input image. Use a 1x1 convolution layer to map the feature map to the number of channels corresponding to the number of target categories (for example, road and background segmentation, the number of categories is 2). Perform softmax or sigmoid activation on the output to obtain the probability of each pixel belonging to the target (for example, road). Where (i, j) represents the probability that the (i, j)th pixel in the image belongs to the target.

[0102] Step 3.4: Train the network. The present invention also combines cross-entropy loss and Dice loss to optimize the model. Cross-entropy loss is used to measure the difference between the predicted target segmentation probability and the true label. This loss function calculates the error between the predicted probability of each pixel category and the true label at the pixel level, and is particularly suitable for class imbalance problems (such as the background occupies a large part and the target area is small). Dice loss is used to measure the overlap between the predicted segmentation result and the true label. Dice loss is calculated based on the Dice coefficient, which is an indicator commonly used to evaluate image segmentation accuracy. By optimizing the Dice loss, the network can pay more attention to the accuracy of the segmented area, especially when the target area is small or has a complex shape, which can effectively improve the segmentation quality. Finally, weighted loss is used. Taking into account the role of different loss functions, cross-entropy loss and Dice loss are usually combined in a weighted manner. Cross-entropy loss is responsible for adjusting the overall classification ability of the model, while Dice loss ensures accurate segmentation of the target area. The weighting coefficient can be adjusted through experiments. Usually, we will give a higher weight to Dice loss to ensure the accuracy of the segmented area. The final loss function is a weighted combination of cross-entropy loss and Dice loss:

[0103]

[0104] in, Indicates total loss; represents the cross entropy loss; Represents Dice loss; α and β represent the weight coefficients of cross entropy loss and Dice loss respectively

[0105] During training, the segmentation results are gradually optimized by minimizing this total loss. By optimizing the total loss function, the model learns more accurate object segmentation results, especially in complex backgrounds and object edge areas. Through this step, the network's decoding process and the total loss function work together to enable the network to accurately segment objects in remote sensing images, improving the accuracy and robustness of object segmentation.

[0106] From the description of the above embodiments, those skilled in the art can know that: the present invention provides a remote sensing image target segmentation method based on dilated multi-scale fusion, the focus is on further improving the segmentation accuracy and multi-scale feature extraction capability of the network model by combining dilated convolution with the attention mechanism through step 2. Through the dilated convolution operation, the network can capture long-range features in remote sensing images within a larger receptive field, thereby improving the perception of complex backgrounds and low-contrast areas. And through the introduction of multi-scale feature fusion, image features of different scales can be effectively and efficiently fused, enhancing the perception of complex terrain and diverse targets, thereby improving segmentation accuracy and robustness.

[0107] Furthermore, this invention optimizes computational resource requirements. Through efficient parameter fine-tuning, it reduces computational overhead during training and deployment, enabling efficient operation in diverse computing environments. Compared to traditional deep learning methods, this invention not only improves segmentation accuracy but also demonstrates greater efficiency in computational resource utilization, better adapting to the computational demands of practical applications.

[0108] In summary, this invention can effectively improve the performance of remote sensing image segmentation in complex scenarios, especially in the segmentation of low-contrast areas and small objects. Furthermore, through efficient computational strategies, this invention can reduce the demand for computing resources while maintaining segmentation accuracy, thus possessing broad application prospects.

[0109] Furthermore, the present invention also provides a remote sensing image target segmentation network based on dilation multi-scale fusion, which applies a remote sensing image target segmentation method based on dilation multi-scale fusion in the above embodiment to achieve remote sensing image target segmentation. The network includes:

[0110] Image preprocessing and feature map extraction module, used for preprocessing and feature map extraction of original remote sensing images;

[0111] The dilated multi-scale fusion module is used to process the extracted feature maps using dilated convolution, attention mechanism and attention matrix to obtain enhanced feature maps;

[0112] The target segmentation module is used to extract and fuse multi-scale features of the enhanced feature map, and output the target segmentation result with the same resolution as the original remote sensing image through the decoder.

[0113] The network provided by the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference can be made to the corresponding content in the aforementioned method embodiment, which will not be repeated here.

[0114] In addition, refer to Figure 3 As shown, an embodiment of the present invention further provides an electronic device, which may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and run on the processor 10. The processor executes the computer program to implement a remote sensing image target segmentation method based on expansion multi-scale fusion in the above method embodiment.

[0115] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the memory 11, as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.

[0116] The memory 11 may be, for example, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (RAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination thereof.

[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, network systems, electronic devices, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] It should be noted that the word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0120] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image target segmentation method based on dilation multi-scale fusion, characterized in that: The method comprises the following steps: Step 1: Preprocess the original remote sensing image and extract feature maps; Step 2: Process the extracted feature map using dilated convolution, attention mechanism, and attention matrix to obtain an enhanced feature map; Step 3: Perform multi-scale feature extraction and fusion on the enhanced feature map, and output the target segmentation result with the same resolution as the original remote sensing image through the decoder; In step 1, the original remote sensing image is preprocessed and feature maps are extracted. The specific process includes: Step 1.1: Preprocess the original remote sensing image, including normalization, image enhancement and denoising, contrast enhancement and detail enhancement; Step 1.2: Combine the convolutional neural network and the transformer, and take advantage of the diffusion model and pre-trained model to extract features from the pre-processed remote sensing image. Step 1.3: The extracted features are fused along the channel dimension, and the spatial resolution is restored to generate a feature map F with spatial details and global context information suitable for image analysis tasks. In step 2, the extracted feature map is processed using dilated convolution, attention mechanism, and attention matrix to obtain an enhanced feature map. The specific process includes: Step 2.1: The feature map F obtained in step 1 is passed through the normalization layer and then expanded through three dilated convolution layers to obtain Then multiply Q and K to get Where H is the height of the image, W is the width of the image, C is the number of channels of the image, and g represents the grouping factor of the number of channels; then Multiplying with V, we get the feature map of the dilated self-attention, which is then fused with F through jump links to enhance the fusion of local and global information, and get the feature map after the dilated convolution operation. in, represents matrix multiplication, represents matrix addition; Step 2.2: Perform multi-path feature fusion. This process includes upper and lower branches, where: Upper branch: First, the feature map Perform global average pooling to obtain global context information; then perform nonlinear transformation through the spiking neuron layer, and enhance the expressiveness of the feature map through the activation function; Lower branch: contains two layers of dilated convolution, after which activation function is applied to maintain the validity and nonlinear representation of the output; Finally, the outputs of the upper and lower branches are fused through addition operations to obtain the fused feature map Step 2.3: Fuse the feature map After processing through a normalization layer and three convolutional layers, we obtain the query, key, and value respectively; Step 2.4: Construct a fixed attention matrix; Step 2.5: Perform weighted calculation on the attention matrix and feature map to obtain the adjusted feature map Step 2.6: Adjust the feature map and fusion feature map Perform residual connection to obtain the final enhanced feature map Used for subsequent remote sensing image target segmentation tasks; In step 3, multi-scale feature extraction and fusion are performed on the enhanced feature map, and the decoder outputs the target segmentation result with the same resolution as the original remote sensing image. The specific process includes: Step 3.1: Enhance the feature map obtained in step 2 Perform multi-scale convolution operations and use different convolution kernel sizes to obtain feature maps of multiple scales: and fuse these multi-scale feature maps by weighted averaging or splicing to obtain a feature map containing local features Step 3.2: Feature Map Perform global average pooling to obtain global features Then the global features and feature maps containing local features Splice to obtain the final feature map Step 3.3: Pass the final feature map through the decoder Converted into target segmentation results with the same resolution as the original remote sensing image.

2. The remote sensing image target segmentation method based on expansion multi-scale fusion according to claim 1, characterized in that: In step 3, a total loss function is constructed by combining cross entropy loss and Dice loss in a weighted manner to perform remote sensing image target segmentation training.

3. The method for remote sensing image target segmentation based on dilation multi-scale fusion according to claim 2, characterized in that: The total loss function constructed is: in, Indicates total loss; represents the cross entropy loss; represents Dice loss; α and β represent the weight coefficients of cross entropy loss and Dice loss, respectively.

4. A remote sensing image target segmentation network based on expansion multi-scale fusion, characterized in that: Applying a remote sensing image target segmentation method based on dilation multi-scale fusion according to any one of claims 1 to 3 to implement remote sensing image target segmentation, the network includes: Image preprocessing and feature map extraction module, used for preprocessing and feature map extraction of original remote sensing images; The dilated multi-scale fusion module is used to process the extracted feature maps using dilated convolution, attention mechanism and attention matrix to obtain enhanced feature maps; The target segmentation module is used to extract and fuse multi-scale features of the enhanced feature map, and output the target segmentation result with the same resolution as the original remote sensing image through the decoder.

5. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement a remote sensing image target segmentation method based on dilation multi-scale fusion as described in any one of claims 1 to 3.

Images