Improved remote sensing image segmentation method and system of segmer network model

By improving the Segformer network model, combining the fusion of near-infrared images and RGB images, and the multi-scale pyramid convolution structure and channel attention mechanism, the problems of low efficiency and insufficient segmentation accuracy in semantic segmentation of remote sensing images are solved, achieving more efficient and accurate image segmentation effect.

CN119068185APending Publication Date: 2024-12-03NANJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202410943561.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art has problems of low efficiency and insufficient segmentation accuracy in semantic segmentation of remote sensing images, especially in the processing of high-resolution remote sensing images, and the loss of segmentation due to feature loss.

Method used

By improving the Segformer network model, the near-infrared image and RGB image fusion of the GID dataset is adopted, the EPSA module and SKNet are introduced, and the multi-scale pyramid convolution structure and channel attention mechanism are combined to optimize the image processing flow to improve segmentation accuracy.

Benefits of technology

It significantly improves the efficiency and accuracy of remote sensing image segmentation, solves the problems of segmentation inaccuracy and detail loss caused by feature loss, and provides more accurate context semantic feature merging capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068185A_ABST
    Figure CN119068185A_ABST
Patent Text Reader

Abstract

The invention discloses an improved remote sensing image segmentation method and system of a sepform network model, and relates to the technical field of image processing, and the method comprises the steps: collecting data, constructing a GID data set, and carrying out the preprocessing; the method comprises the following steps of: improving an original segform network model; and inputting the training set into the improved sepform network model for training, adjusting parameters to optimize the model, outputting the trained segmentation network model, and applying the improved segmentation network model to image segmentation. According to the method, the near infrared image and the RGB image in the GID remote sensing data set are subjected to image fusion by adopting a deep learning method, the semantic information of the data set is enriched, and the model segmentation efficiency is improved. According to the invention, the EPSA module is introduced to replace the original Segform decoder design, and the EPSA module processes input on multiple scales. Information of an input feature map is integrated through a multi-scale pyramid convolution structure.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an improved remote sensing image segmentation method and system of a segformer network model. Background Art

[0002] Semantic segmentation is a basic task in computer vision, which refers to the pixel-level classification of images according to the semantic labels of each pixel. Semantic segmentation supports many downstream applications, such as medical image segmentation, video background replacement, and intelligent tasks such as unmanned driving.

[0003] Since the full convolutional neural network was proposed and applied to the field of semantic segmentation with remarkable success, convolutional neural networks have become the mainstream choice for dense prediction. Segmentation methods based on convolutional neural networks have good generalization capabilities. Due to their excellent performance, convolutional neural networks and full convolutional neural network architectures have become the basic components of segmentation research. Typical algorithms of this type include SegNet, U-Net, DeepLab, etc.

[0004] Due to the success of the Transformer model in natural language processing, the Transformer structure was introduced into the field of computer vision. The Transformer model contains a novel self-attention mechanism design that can establish associations between different positions, allowing the model to simultaneously consider all position information in the input sequence. This mechanism enables the Transformer model to capture long-distance dependencies and improves the performance of the model. On large image datasets, the Transformer network has achieved results that surpass the optimal CNN model, while on smaller remote sensing datasets, the complexity and diversity of the landforms make the remote sensing image semantic segmentation task challenging. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is to enhance the efficiency of semantic segmentation of remote sensing images and improve the segmentation accuracy of the original model network.

[0007] In order to solve the above technical problems, the present invention provides the following technical solution: an improved remote sensing image segmentation method of a segformer network model, which comprises the following steps:

[0008] Collect data, construct the GID dataset, and perform preprocessing; improve the original segformer network model; input the training set into the improved segformer network model for training, adjust the parameters to optimize the model, output the segmentation network model after training, and use the improved segmentation network model for image segmentation.

[0009] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, wherein: the construction of the GID data set includes fusing the near-infrared image and the RGB image in the GID data set.

[0010] The preprocessing includes rotation, mirror translation, shearing, expanding the data set, and dividing the training set and the test set of the data set into a ratio of 8:2.

[0011] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, the GID data set includes RGB images with a spatial resolution of 1m and near-infrared images of 4m.

[0012] The image fusion is performed through three parts: encoder stage, fuser stage and decoder stage.

[0013] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, the encoder stage includes two convolutional layers and a dense module, wherein the convolutional layer extracts rough features, and the dense module includes cascaded convolutional blocks.

[0014] The fuser stage adopts the l1-norm strategy. The feature map obtained by the encoder is calculated using the l1 norm and the block-based average operator. The formula is as follows:

[0015]

[0016] Among them, A i represents the minimum absolute error of the feature map, i represents the index of the feature map, The l1 norm is a measure of the absolute error of the feature map.

[0017] The average operator calculates the final feature map It is expressed as,

[0018]

[0019] The calculation formula of the fused feature map is as follows:

[0020]

[0021] Among them, f n represents the fused feature map, i.e., the input of the decoder, w i Represents the average value of the feature map, r is a custom parameter that depends on the size of the block.

[0022] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, the improvement of the original segformer network model includes improving the decoder layer of the original segformer network to an EPSA layer, fusing the output features of each layer of the encoder layer, and improving the channel attention of the EPSA module.

[0023] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, the improvement is that the EPSA layer includes a multi-scale feature map of the channel obtained by the SPC module, and the SPC module extracts the spatial information of the input feature map in a multi-branch manner, and the input channel dimension of each branch is And C is divisible by S, C represents the number of image channels, and S represents the number of custom branches.

[0024] As a preferred solution of the improved remote sensing image segmentation method of the segformer network model described in the present invention, the channel attention of the improved EPSA module includes using SKNet to extract the channel attention of the EPSA module, fusing multiple branch feature maps of the SPC module, first adding and fusing the receptive field information, and then using global average pooling to embed the global channel information:

[0025]

[0026] Among them, s c Represents the statistical information of the cth channel, U C Represents the result of the cth channel after the fusion of multi-branch feature maps.

[0027] Construct the feature descriptor z through the fully connected layer:

[0028] z=F fc (s) = δ(B(W s ))

[0029] Among them, δ represents the ReLU function, B represents batch normalization, Ws represents the weight matrix, and F fc (s represents a fully connected layer.

[0030] Under the guidance of the feature descriptor z, the attention vector is obtained through the softmax function, the channel attention is established, and the channel attention is applied to multiple feature maps, and selectively reorganized to construct a new feature map:

[0031]

[0032] Among them, AB are two matrices obtained by 1*1 convolution of branch feature maps, Ac and Bc represent the c-th row of matrix A and the c-th row of matrix B respectively, and a c and b c Satisfy a c +b c =1.

[0033] Use a c and b c The branch feature maps are weighted, and then the tensor elements of the weighted branch feature maps are added to obtain the final feature map.

[0034] Another object of the present invention is to provide an improved remote sensing image segmentation system of the segformer network model, which can optimize the image processing process by integrating advanced feature fusion technology and multi-scale information processing, thereby solving the problems of inaccurate segmentation and detail loss caused by feature loss when processing high-resolution remote sensing images in the prior art.

[0035] To solve the above technical problems, the present invention provides the following technical solutions: an improved remote sensing image segmentation system of a segformer network model, comprising a GID data set construction module, an image fusion module and an SPC module.

[0036] The GID dataset construction module is used to fuse the near infrared image and the RGB image in the GID dataset.

[0037] The image fusion module includes an encoder stage, a fuser stage and a decoder stage.

[0038] The encoder stage consists of two convolutional layers that extract coarse features and a dense module that includes cascaded convolutional blocks.

[0039] The fuser stage adopts the l1-norm strategy, and the feature map obtained by the encoder uses the l1 norm and the block-based average operator to calculate the final feature map.

[0040] The decoder stage uses the fused feature map as input.

[0041] The SPC module extracts spatial information of the input feature map in a multi-branch manner.

[0042] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the improved remote sensing image segmentation method of the segformer network model as described above are implemented.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the improved remote sensing image segmentation method of the segformer network model as described above.

[0044] Beneficial effects of the present invention: The present invention uses a deep learning method to fuse near-infrared images and RGB images in the GID remote sensing dataset, which greatly enriches the semantic information of the dataset and improves the model segmentation efficiency.

[0045] The present invention introduces the EPSA module to replace the original Segformer decoder design. The EPSA module can process inputs at multiple scales. The information of the input feature map is integrated by using a multi-scale pyramid convolution structure. At the same time, by compressing the channel dimension of the input, spatial information of different scales can be effectively extracted from each channel feature map. By doing so, contextual semantic features can be more accurately merged.

[0046] The present invention also improves the channel attention part of EPSA and introduces SKNet, which has a dynamic selection mechanism that allows each image to adaptively adjust its receptive field size according to multiple scales of the input information and aggregate feature maps of kernels of different sizes according to weights.

[0047] The design method of the present invention is universal, especially for the semantic segmentation method based on Segformer model, which provides a new idea for the related detection method based on Segformer. It can easily integrate the method mentioned in the present invention into its own model to improve the convergence speed and accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0049] Figure 1 An overall flow chart of the improved remote sensing image segmentation method of the segformer network model provided in the first embodiment of the present invention.

[0050] Figure 2 The structure of the DenseFusion network of the improved remote sensing image segmentation method of the segformer network model provided in the first embodiment of the present invention.

[0051] Figure 3An improved Segformer segmentation result diagram of the improved remote sensing image segmentation method of the segformer network model provided in the first embodiment of the present invention.

[0052] Figure 4 The overall framework diagram of the improved remote sensing image segmentation system of the segformer network model provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0054] Example 1, reference Figure 1 to Figure 3 , which is an embodiment of the present invention, provides an improved remote sensing image segmentation method based on a segformer network model, characterized in that:

[0055] S1: Collect data, construct GID dataset, and perform preprocessing.

[0056] Constructing the GID dataset includes fusing the near-infrared images and RGB images in the GID dataset.

[0057] Preprocessing includes rotation, mirror translation, shearing, expanding the data set, and dividing the training set and test set of the data set into a ratio of 8:2.

[0058] GID is a large-scale high-resolution remote sensing image land cover dataset built based on my country's Gaofen-2 satellite data. The GID dataset is divided into two parts: a large-scale classification set (GID-5) and a fine land cover set (GID-15). The large-scale classification set (GID-5) contains five land cover categories, including buildings, farmland, forests, grasslands and waters, with a total of 150 Gaofen-2 satellite remote sensing images annotated at the pixel level. Among them, the training set is 120 images and the validation set is 30 images. The size of the Gaofen-2 satellite remote sensing image is 6800×7200, and it is pixel-level annotated by experts in the field of remote sensing interpretation. The fine land cover set (GID-15) contains 15 categories, including rice fields, irrigated land, dry land, gardens, arbor forests, shrubs, natural grasslands, artificial grasslands, industrial land, urban residential, rural residential, transportation land, rivers, lakes, ponds, etc. This application adopts the fine land cover set (GID-15) part.

[0059] Since the GID data includes RGB images with a spatial resolution of 1m and near-infrared images with a spatial resolution of 4m, in order to improve the training effect of the model and obtain richer image information, the images in the GID dataset are fused using the DenseFusion network. First, the images in the GID dataset are cut into 512×512 pixels, and then the cut images are input into the trained DenseFusion network.

[0060] The structure of the DenseFusion network is as follows Figure 2 As shown in Figure 2. The DenseFusion network fuses two images through three parts: encoder, fuser and decoder. The encoder stage includes two convolutional layers and a dense module. The convolutional layer is used to extract rough features. The dense module contains several cascaded convolutional blocks. Each convolutional block is directly connected to all subsequent layers to retain as many deep features of the image as possible. This operation ensures that all significant features are used for the fusion strategy.

[0061] Image fusion goes through three stages: encoder stage, fuser stage and decoder stage.

[0062] The encoder stage consists of two convolutional layers and a dense module, where the convolutional layers extract coarse features and the dense module consists of cascaded convolutional blocks.

[0063] The fuser stage adopts the l1-norm strategy. The feature map obtained by the encoder is calculated using the l1 norm and the block-based average operator. The formula is as follows:

[0064]

[0065] Among them, A i represents the minimum absolute error of the feature map, i represents the index of the feature map, The l1 norm is a measure of the absolute error of the feature map.

[0066] The average operator calculates the final feature map It is expressed as,

[0067]

[0068] The calculation formula of the fused feature map is as follows:

[0069]

[0070] Among them, f n represents the fused feature map, i.e., the input of the decoder, w i Represents the average value of the feature map, r is a custom parameter that depends on the size of the block.

[0071] In the decoder stage, several cascaded 3×3 convolutional layers are used to restore the fused feature maps layer by layer in a simple and efficient manner.

[0072] In order to improve the training effect, generalization ability and robustness of the model, the data set is enhanced in advance, and the image enhancement method is used on the data set. The built-in functions of pytorch are used for image enhancement: RandomResize function is used for random scaling, RandomSizeCrop function is used for cropping, and RandomHorizontalFlip function is used for horizontal flipping. These methods are used to increase the diversity and robustness of the data and expand the data set to 10,000 images.

[0073] S2: Improve the original segformer network model.

[0074] Use the Segformer encoder as the backbone, use the pre-trained model parameters, and freeze all layers of the Segformer encoder.

[0075] The decoder stage of Segformer adopts a simple full MLP structure to fuse the feature maps of each layer obtained in the encoder stage. In the present invention, the decoder of Segformer is improved and the EPSA module is introduced.

[0076] First, the multi-scale feature map of the channel is obtained through the SPC module. The SPC module extracts the spatial information of the input feature map in a multi-branch manner. The input channel dimension of each branch is And C is divisible by S. By doing so, more abundant position information of the input tensor can be obtained and processed in parallel at multiple scales. The present invention chooses to extract spatial information by a dual-branch approach.

[0077] In order to process input tensors at different kernel scales without increasing the computational cost, a group convolution method is introduced and applied to the convolution kernel. The weights of the channels are then extracted through the channel attention module and aggregated into a PSA module. The PSA module can integrate multi-scale spatial information and cross-channel attention into the block of each feature group. At the corresponding position of the bottleneck block, the 3x3 convolution is replaced by the PSA module to obtain a new ESPA module.

[0078] Improving the channel attention of the EPSA module includes using SKNet to extract the channel attention of the EPSA module and fusing multiple branch feature maps of the SPC module. First, the receptive field information is fused by adding them, and then the global channel information is embedded by global average pooling:

[0079]

[0080] Among them, s c Represents the statistical information of the cth channel, U C Represents the result of the cth channel after the fusion of multi-branch feature maps, H and W represent the height and width of the feature map respectively.

[0081] Then, the feature descriptor z is constructed through the fully connected layer:

[0082] z=F fc (s) = δ(B(W s ))

[0083] Among them, δ represents the ReLU function, B represents batch normalization, and Ws represents the weight matrix.

[0084] Under the guidance of the feature descriptor z, the attention vector is obtained through the softmax function, the channel attention is established, and it is applied to the multiple feature maps previously split, and they are selectively recombined to construct a new feature map:

[0085]

[0086] Among them, AB are two matrices obtained by 1*1 convolution of branch feature maps, Ac and Bc represent the c-th row of matrix A and the c-th row of matrix B respectively, and a c and b c Satisfy a c +b c =1.

[0087] Use a c and b c The branch feature maps are weighted, and then the tensor elements of the weighted branch feature maps are added to obtain the final feature map.

[0088] S3: Input the training set into the improved segformer network model for training, adjust the parameters to optimize the model, and output the segmentation model after training.

[0089] Debug the modified model, select appropriate training parameters and train it on the data set. The weight files with the highest evaluation index on the test set are saved as follows:

[0090] Lr=1e -4

[0091] Init_Epoch = 0

[0092] Freeze_Epoch = 50

[0093] UnFreeze_Epoch = 100

[0094] lr_decay_type = 'cos'

[0095] Among them, lr represents the learning rate, Init_Epoch, Freeze_Epoch, and UnFreeze_Epoch are the training parameters in the freezing stage. Init_Epoch indicates the start of the training round; Freeze_Epoch is the round of model freezing training, at which the backbone of the model is frozen, the feature extraction network does not change, and the video memory occupied is small, and only the network is fine-tuned; UnFreeze_Epoch is the round of unfreezing training. lr_decay_type indicates that this application chooses to use cosine annealing for learning rate reduction.

[0096] Choose an appropriate model optimization algorithm:

[0097] The present invention uses the Adam optimization algorithm, with the momentum set to 0.9 and the weight decay set to 1e -2 Adam uses momentum and adaptive learning rate to speed up convergence, and uses first-order momentum and second-order momentum at the same time. The formula is as follows:

[0098] First, use the estimate of the first-order moment of SGD, that is, the mean:

[0099] m t =β1·m t-1 +(1-β1)·g t

[0100] With the addition of AdaDelta's second-order momentum, the estimate of the second-order moment, variance, is similar to variance in that it is a type of second-order moment:

[0101]

[0102] Correct the mean and variance because their initial values ​​are 0. Since they will be biased toward 0, this processing will reduce the impact of this bias:

[0103]

[0104] Adam parameter update formula is as follows:

[0105]

[0106] Where η is the learning rate, g tis the gradient of the current parameter, β1 is the exponential decay rate of the first-order moment estimate (such as 0.9), and β2 is the exponential decay rate of the second-order moment estimate (such as 0.999). The former controls the first-order moment estimate, and the latter controls the second-order moment estimate. This hyperparameter should be set to a number close to 1 in sparse gradients. and It is β1 and β2 raised to the power of t.

[0107] The trained semantic segmentation model is compared with the original Segformer network model on the test set, and the pixel accuracy (PA), mean pixel accuracy (MPA), and mean intersection-over-union (MioU) are used as evaluation indicators to compare and evaluate the Segformer network model before and after improvement. The calculation of these evaluation indicators is based on the confusion matrix.

[0108] The confusion matrix is ​​the classification result of the statistical classification model, including the number of samples classified into the right and wrong categories, and then the results are displayed in a table, which is the confusion matrix. The following table is the definition of the confusion matrix.

[0109] Table 1: Definition of confusion matrix

[0110]

[0111] Pixel accuracy is the ratio of the number of pixels with the correct predicted category to the total number of pixels; category average pixel accuracy is to calculate the ratio of the number of correctly classified pixels for each category separately, and then add up the average; intersection-over-union is the intersection of the true value and the predicted value of the pixel divided by the union of the true value and the predicted value of the pixel; average intersection-over-union is to calculate the intersection-over-union ratio for each category separately, and then average the IOU of all categories.

[0112] S4: The trained segmentation model is compared with the original segformer network model on the test set, and the improved segformer model is compared and evaluated with the original segformer network model through evaluation indicators.

[0113] In this embodiment, the performance of the original Segformer network model and the improved Segformer algorithm are compared in this embodiment. Some test results are as follows: Figure 3 As shown, the model comparison data is as follows:

[0114] Table 2: Performance comparison of Segformer and improved Segformer algorithms

[0115]

[0116] Example 2, reference Figure 4 , which is an embodiment of the present invention, provides a system for improving the remote sensing image segmentation method of the segformer network model, characterized by: including a GID data set construction module, an image fusion module and an SPC module.

[0117] The GID dataset construction module is used to fuse the near-infrared images and RGB images in the GID dataset.

[0118] The image fusion module includes the encoder stage, the fuser stage and the decoder stage.

[0119] The encoder stage consists of two convolutional layers that extract coarse features and a dense module consisting of cascaded convolutional blocks.

[0120] The fuser stage adopts the l1-norm strategy, and the feature map obtained by the encoder uses the l1 norm and the block-based average operator to calculate the final feature map.

[0121] The decoder stage uses the fused feature maps as input.

[0122] The SPC module extracts the spatial information of the input feature map in a multi-branch manner.

[0123] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0125] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0126] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. The improved remote sensing image segmentation method based on the segformer network model is characterized by: include: Collect data, construct GID dataset, and perform preprocessing; Improve the original segformer network model; The training set is input into the improved segformer network model for training, and the parameters of the model are adjusted to optimize the model. The segmentation network model after training is output, and the improved segmentation network model is used for image segmentation.

2. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 1, characterized in that: The constructing of the GID data set includes fusing the near infrared image and the RGB image in the GID data set; The preprocessing includes rotation, mirror translation, shearing, expanding the data set, and dividing the training set and the test set of the data set into a ratio of 8:

2.

3. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 2, characterized in that: The GID dataset includes RGB images with a spatial resolution of 1m and near-infrared images of 4m; The image fusion is performed through three parts: encoder stage, fuser stage and decoder stage.

4. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 3, characterized in that: The encoder stage includes two convolutional layers and a dense module, wherein the convolutional layer extracts coarse features and the dense module includes cascaded convolutional blocks; The fuser stage adopts the l1-norm strategy. The feature map obtained by the encoder is calculated using the l1 norm and the block-based average operator. The formula is as follows: Among them, A i represents the minimum absolute error of the feature map, i represents the index of the feature map, The l1 norm of is a measure of the absolute error of the feature map; The average operator calculates the final feature map It is expressed as, The calculation formula of the fused feature map is as follows: Among them, f n represents the fused feature map, i.e., the input of the decoder, w i Represents the average value of the feature map, r is a custom parameter that depends on the size of the block.

5. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 4, characterized in that: The improvement to the original segformer network model includes improving the decoder layer of the original segformer network to an EPSA layer, fusing the output features of each layer of the encoder layer, and improving the channel attention of the EPSA module.

6. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 5, characterized in that: The improvement is that the EPSA layer includes a multi-scale feature map of the channel obtained by the SPC module. The SPC module extracts the spatial information of the input feature map in a multi-branch manner. The input channel dimension of each branch is And C is divisible by S, C represents the number of image channels, and S represents the number of custom branches.

7. The improved remote sensing image segmentation method of the segformer network model as claimed in claim 6, characterized in that: The improved channel attention of the EPSA module includes using SKNet to extract the channel attention of the EPSA module, fusing multiple branch feature maps of the SPC module, first fusing the receptive field information by adding, and then using global average pooling to embed the global channel information: Among them, s c Represents the statistical information of the cth channel, U C Represents the result of the cth channel after the fusion of multi-branch feature maps, H and W represent the height and width of the feature map respectively; Then, the feature descriptor z is constructed through the fully connected layer: z=F fc (s)=δ(B(W s )) Among them, δ represents the ReLU function, B represents batch normalization, Ws represents the weight matrix, and F fc (s) represents a fully connected layer; Under the guidance of the feature descriptor z, the attention vector is obtained through the softmax function, the channel attention is established, and the channel attention is applied to multiple feature maps, and selectively reorganized to construct a new feature map: Among them, AB are two matrices obtained by 1*1 convolution of branch feature maps, Ac and Bc represent the c-th row of matrix A and the c-th row of matrix B respectively, and a c and b c Satisfy a c +b c =1; Use a c and b c The branch feature maps are weighted, and then the tensor elements of the weighted branch feature maps are added to obtain the final feature map.

8. A system for improving remote sensing image segmentation method using the segformer network model according to any one of claims 1 to 7, characterized in that: Includes GID dataset construction module, image fusion module and SPC module; The GID data set construction module is used to fuse the near infrared image and the RGB image in the GID data set; The image fusion module includes an encoder stage, a fuser stage and a decoder stage; The encoder stage consists of two convolutional layers and a dense module, the convolutional layer extracts coarse features, and the dense module includes cascaded convolutional blocks; The fuser stage adopts the l1-norm strategy, and the feature map obtained by the encoder uses the l1 norm and the block-based average operator to calculate the final feature map; The decoder stage uses the fused feature map as input; The SPC module extracts spatial information of the input feature map in a multi-branch manner.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the improved remote sensing image segmentation method of the segformer network model described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the improved remote sensing image segmentation method of the segformer network model described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Knowledge distillation-based lightweight remote sensing image semantic segmentation method and device

    CN116740344A

  • Systems and methods for fusing color image and near-infrared image

    WO2021184029A1

Cited By

  • Method for detecting farmland satellite remote sensing image in real time

    CN119964019A

  • A method for real-time detection of satellite remote sensing images of farmland

    CN119964019B