Mangrove forest unmanned aerial vehicle remote sensing super-resolution image reconstruction method based on SwinIR

Abstract

The invention discloses a mangrove forest unmanned aerial vehicle remote sensing super-resolution image reconstruction method based on SwinIR. According to the technical scheme, an efficient channel attention ECA mechanism and a plurality of convolutional layers are combined together, important features are effectively highlighted and unimportant features are inhibited through self-adaptive distribution of channel weights, the feature extraction capability is enhanced, and the reconstruction efficiency is improved. According to the method, the accuracy of mangrove forest unmanned aerial vehicle remote sensing super-resolution image feature representation can be improved, and the species identification effect and efficiency of the forest unmanned aerial vehicle remote sensing image are improved.

Description

Technical Field

[0001] The invention relates to computer image super-resolution technology, in particular to a mangrove unmanned aerial vehicle remote sensing super-resolution image reconstruction method based on SwinIR. Background Art

[0002] Mangroves are salt-tolerant evergreen trees and other plant communities growing in tropical and subtropical intertidal zones, which provide important ecosystem services such as nutrient cycling, carbon sequestration and coastal disaster mitigation. In the past few decades, the ecological functions of mangroves are being degraded due to the impact of climate change, natural disasters and human interference. Therefore, accurate classification and real-time monitoring of mangrove species are crucial for the protection and restoration of mangroves. Traditionally, obtaining information on mangrove species requires expensive, laborious and time-consuming field surveys, and it is often difficult for investigators to access mangrove areas. In recent years, remote sensing technology has been widely used in mangrove monitoring and evaluation with its excellent spatial and texture characteristics and high-resolution multispectral images. These technologies include spectrometer measurement, high-resolution aerial imaging, medium- and high-spatial-resolution satellite remote sensing imaging, and hyperspectral imaging. Most remote sensing technologies are used for large-scale forest resource surveys and cannot capture the detailed distribution of mangrove species. In recent years, the rapid development of drone technology has provided a new source of data for the classification of mangrove communities. The flexibility, cloud-based operation capability, low cost, and centimeter-level high spatial resolution of drones have made drones a supplement to satellite remote sensing and have broad development prospects. However, accurate classification of mangrove species remains challenging due to the diversity of species and canopy structures, and even with high spectral and spatial resolution remote sensing data, it is difficult to distinguish different tree species.

[0003] Super-resolution reconstruction technology provides a new method. Based on this technology, research on mangrove species identification based on super-resolution reconstruction of UAV remote sensing images is carried out. In recent years, in the field of image super-resolution, the effectiveness of the single image super-resolution (SISR) method based on the Transformer model has gradually surpassed the traditional convolutional neural network (CNN) method. SwinTransformer is a visual model based on the Transformer structure. It has good characteristics such as global perception and dynamic weighting, and can complete various visual tasks well. SwinIR, as a super-resolution model based on Swin Transformer, also shows excellent performance. Summary of the invention

[0004] The purpose of the present invention is to provide a method for reconstructing super-resolution images of mangroves using UAV remote sensing technology based on SwinIR in order to address the deficiencies of the prior art. This method can improve the accuracy of the feature representation of super-resolution images of mangroves using UAV remote sensing technology and improve the species recognition effect and efficiency of UAV remote sensing images of forests.

[0005] The technical solution for achieving the purpose of the present invention is:

[0006] A method for acquiring super-resolution images of mangroves by UAV remote sensing based on SwinIR comprises the following steps:

[0007] 1) Establish a mangrove UAV remote sensing image reconstruction network based on improved SwinIR and segment the super-resolution image. The mangrove UAV remote sensing image reconstruction network based on improved SwinIR includes:

[0008] 1-1) Shallow feature extraction: In the shallow feature extraction stage, a convolutional layer is used to extract shallow features similar to SwinIR to solve the problem of edge and texture feature extraction in remote sensing images, which provides more effective input for subsequent feature extraction processing;

[0009] 1-2) Deep feature extraction: In the deep feature extraction stage, a residual mixed attention module RMAM is designed to extract the shallow features F S Extract deep features F D ∈R H×W×C , the formula is as follows:

[0010] F D =H D (F S ), (1)

[0011] Among them, H D (·) represents the deep feature extraction module, and the final deep feature F D It uses each residual mixed attention module and the final convolutional layer extraction, the formula is as follows:

[0012] F i =H RMAM (F i-1 ), i=1, 2, 3,..., 6, (2)

[0013] Among them, H RMAM (·) represents the i-th RMAM module. RMAM is mainly composed of a hybrid attention module MAM, which is composed of a new convolutional enhanced channel attention module (CEA) and a neighborhood attention NAT module. Given an input feature F k , the formula is as follows:

[0014] F D =H D (F S ), (3)

[0015] F m =NAT(F n )+CEA(F n )+F k , (4)

[0016] F i =MLP(LN(F m ))+F m , (5)

[0017] where F n and F m It is an intermediate feature. The convolution enhanced channel attention module CEA combines the efficient channel attention ECA, adopts adaptive allocation of channel weights, and adds convolutional layers and activation layers before the ECA attention mechanism. The formula is as follows:

[0018] F1=Conv(F0), (3)

[0019] F2=Activation(F1), (4)

[0020] F3=Conv(F2), (5)

[0021] F4=BN(F3),

[0022] CEA(F n )=ECA(F5),

[0023] Where F0 is the initial feature of the input convolution enhanced channel attention module CEA;

[0024] 1-3) Image reconstruction: In the image reconstruction stage, the Pixel-Shuffle method is used to fuse the shallow and deep features. Through the Pixel-Shuffle operation, the information between the shallow and deep features can be fully utilized to achieve the reconstruction of the original image.

[0025] The convolution enhanced channel attention CEA module in this technical solution combines the efficient channel attention ECA mechanism with multiple convolutional layers. This module effectively highlights important features and suppresses unimportant features by adaptively allocating channel weights, thereby enhancing the feature extraction capability. This technical solution adds some convolutional layers and activation layers before the ECA attention mechanism, which are mainly used to extract features and perform nonlinear transformations. This method can improve the accuracy of feature representation of super-resolution images of mangrove drone remote sensing, and improve the species recognition effect and efficiency of forest drone remote sensing images.

[0026] The NAT module is introduced in this technical solution mainly to enhance its feature extraction and detail capture capabilities. Through the innovative neighborhood attention mechanism, NAT can process each pixel in the image more flexibly while maintaining computational efficiency, enabling the model to better capture and reconstruct image details.

[0027] This technical solution not only provides an effective upsampling strategy, but also retains information that helps restore image details by rearranging channels, providing strong support for performance optimization of the image reconstruction layer.

[0028] This method can improve the accuracy of feature representation of mangrove UAV remote sensing super-resolution images and improve the species recognition effect and efficiency of forest UAV remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a method flow of an embodiment;

[0030] Figure 2 Schematic diagram of the convolution enhanced channel attention CEA module in the embodiment. DETAILED DESCRIPTION

[0031] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.

[0032] Example:

[0033] Reference Figure 1 , a mangrove UAV remote sensing super-resolution image reconstruction method based on SwinIR, comprising the following steps:

[0034] 1) Establish a mangrove UAV remote sensing image reconstruction network based on improved SwinIR and segment the super-resolution image. The mangrove UAV remote sensing image reconstruction network based on improved SwinIR includes:

[0035] 1-1) Shallow feature extraction: In the shallow feature extraction stage, a convolutional layer is used to extract shallow features similar to SwinIR to solve the problem of edge and texture feature extraction in remote sensing images, which provides more effective input for subsequent feature extraction processing;

[0036] 1-2) Deep feature extraction: In the deep feature extraction stage, a residual mixed attention module RMAM is designed to extract the shallow features F S Extract deep features F D ∈R H×W×C , the formula is as follows:

[0037] F D =H D (FS ), (1)

[0038] Among them, H D (·) represents the deep feature extraction module, and the final deep feature F D It uses each residual mixed attention module and the final convolutional layer extraction, the formula is as follows:

[0039] F i =H RMAM (F i-1 ), i=1, 2, 3,..., 6, (2)

[0040] Among them, H RMAM (·) represents the i-th RMAM module. RMAM is mainly composed of a hybrid attention module MAM, which is composed of a new convolution-enhanced channel attention module CEA and a neighborhood attention NAT module. Given an input feature F k , the formula is as follows:

[0041] F D =H D (F S ), (3)

[0042] F m =NAT(F n )+CEA(F n )+F k , (4)

[0043] F i =MLO(LN(F m ))+F m , (5)

[0044] where F n and F m It is an intermediate feature. The convolution enhanced channel attention module CEA combines the efficient channel attention ECA, adopts adaptive allocation of channel weights, and adds convolutional layers and activation layers before the ECA attention mechanism. The formula is as follows:

[0045] F1=Conv(F0), (3)

[0046] F2=Activation(F1), (4)

[0047] F3=Conv(F2), (5)

[0048] F4=BN(F3),

[0049] CEA(F n )=ECA(F5),

[0050] Where F0 is the initial feature of the input convolution enhanced channel attention module CEA;

[0051] 1-3) Image reconstruction: In the image reconstruction stage, the Pixel-Shuffle method is used to fuse shallow and deep features. Through the Pixel-Shuffle operation, the information between shallow and deep features can be fully utilized to achieve reconstruction of the original image. This method not only provides an effective upsampling strategy, but also retains information that helps restore image details through the rearrangement of channels, providing strong support for the performance optimization of the image reconstruction layer.

[0052] In this example: Table 1 shows the comparison between SwinNET and other super-resolution methods on the drone mangrove remote sensing image dataset. The experimental results show that under the scaling factors of ×2, ×3 and ×4, this example achieves 34.04 / 94.43, 30.59 / 88.55 and 28.52 / 83.35 respectively. Compared with SwinIR, this example achieves performance improvements of +1.39 / 3.19, +0.18 / 0.10 and +0.04 / 1.66:

[0053] Table 1

[0054]

[0055] Table 2 shows the segmentation results of the original mangrove drone images and the super-resolution reconstructed mangrove remote sensing images. The experimental results show that the use of x4-enlarged images significantly improves the performance of the FPN network in the mangrove species segmentation task, especially in the segmentation of Rhizophora mangrovea and Bruguiera gymnorrhiza. Especially for Bruguiera gymnorrhiza, super-resolution processing significantly improves the segmentation effect, indicating that reasonable super-resolution processing can significantly improve the segmentation performance of mangrove images:

[0056] Table 2

[0057]

Claims

1. A method for reconstructing super-resolution images of mangroves using UAV remote sensing based on SwinIR, characterized in that: The steps include: 1) Establish a mangrove UAV remote sensing image reconstruction network based on improved SwinIR and segment the super-resolution image. The mangrove UAV remote sensing image reconstruction network based on improved SwinIR includes: 1-1) Shallow feature extraction: In the shallow feature extraction stage, similar to SwinIR, a convolutional layer is used to extract shallow features; 1-2) Deep feature extraction: In the deep feature extraction stage, a residual mixed attention module RMAM is designed to extract the shallow features F S Extract deep features F D ∈R H×W×C , the formula is as follows: F D =H D (F S ), (1) Among them, H D (·) represents the deep feature extraction module, and the final deep feature F D It uses each residual mixed attention module and the final convolutional layer extraction, the formula is as follows: F i =H RMAM (F i-1 ),i=1,2,3,…,6, (2) Among them, H RMAM (·) represents the i-th RMAM module. RMAM is mainly composed of a hybrid attention module MAM, which consists of a new convolutional enhanced channel attention module (CEA) and a neighborhood attention NAT module. Given an input feature F k , the formula is as follows: F D =H D (F S ), (3) F m =NAT(F n )+CEA(F n )+F k , (4) F i =MLP(LN(F m ))+F m , (5) Among them, F n and F m It is an intermediate feature. The convolution enhanced channel attention module CEA combines the efficient channel attention ECA, adopts adaptive allocation of channel weights, and adds convolutional layers and activation layers before the ECA attention mechanism. The formula is as follows: F1=Conv(F0), (3) F2=Activation(F1), (4) F3=Conv(F2), (5) F4=BN(F3), CEA(F n )=ECA(F5), Where F0 is the input feature of the input convolution enhanced channel attention module CEA; 1-3) Image reconstruction: In the image reconstruction stage, the Pixel-Shuffle method is used to fuse shallow and deep features.

Images