A method for detecting cultivated land in spring and winter based on feature highlighting

Abstract

A method for detecting spring and winter cultivated land based on feature highlighting includes the following steps: Step 1: Data set preparation, panchromatic image and multispectral image are merged, cultivated land is marked to generate training labels, and then the image is cropped and preprocessed as a training set; Step 2: Optimized residual feature extraction model is constructed, feature extraction is performed on the training set image, and feature map is generated; Step 3: The generated feature map is used as input feature highlighting module, feature maps of different scales are extracted, different weight features are assigned to the acquired feature elements, sample feature information is highlighted, and learning is performed to generate recognition model; Step 4: The validation set image is cropped and put into the recognition model for recognition. Step 5: A recognition result mask is generated to obtain a segmentation result. The purpose of the present invention is to solve the problem that the pixels between cultivated land objects and other objects in spring and winter are similar, which makes it difficult to segment with other objects, and a robust method for segmenting cultivated land through drone images is proposed.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic segmentation in remote sensing images, and in particular to a method for segmenting cultivated land in spring and winter. Background Art

[0002] Object extraction plays an important role in land management, resource planning, agricultural yield assessment, environmental protection, etc. In the past, pixel-based methods were mostly used to extract remote sensing objects. For example, random forest, k-nearest neighbor algorithm, Bayesian model and other methods were applied to object classification. The essence of this method is to distinguish different types of objects according to the pixel values ​​and multi-feature information in the remote sensing image. Such methods require a lot of human labeling to obtain the sample features of the objects, and most of the features are not represented enough, which directly leads to unsatisfactory segmentation effects. Therefore, such methods are less practical. In recent years, many researchers have applied deep learning to remote sensing image object segmentation to achieve high-precision object segmentation.

[0003] In the prior art, a paper on cultivated land change detection technology based on deep learning attempts to use convolutional neural networks to extract and analyze features in remote sensing images. The difference between the present invention and the paper is that the cultivated land in spring and winter is difficult to segment, and the cultivated land in spring and winter can be extracted more accurately to avoid misclassification.

[0004] The present invention mainly studies the similar pixels between cultivated land and other land objects in spring and winter, and the shape of cultivated land varies greatly, the target is small, and the background of remote sensing images is complex. It is difficult for the network to concentrate the weights of neurons in the land object categories to be extracted, which leads to misclassification in the cultivated land recognition results.

[0005] In view of the above situation, the present invention provides a farmland segmentation algorithm based on feature highlighting to improve the accuracy of farmland segmentation. Summary of the invention

[0006] The purpose of the present invention is to solve the problem that the pixels between cultivated land objects and other objects in spring and winter are similar, which makes it difficult to segment them from other objects. A robust method for segmenting cultivated land using drone images is proposed.

[0007] A method for extracting spring and winter cultivated land based on feature highlighting includes the following steps:

[0008] Step 1: Dataset creation: Fusing panchromatic and multispectral images, marking cultivated land to generate training labels, and then cropping and preprocessing the images as training sets;

[0009] Step 2: Build an optimized residual feature extraction model, extract features from the training set images, and generate feature maps;

[0010] Step 3: Construct a feature highlighting module, use the generated feature map as input to the feature highlighting module, extract feature maps of different scales, assign different weight features to the acquired feature elements, and highlight the sample feature information.

[0011] Step 4: Crop the validation set images and put them into the recognition model for recognition.

[0012] Step 5: Generate recognition result mask and obtain segmentation result.

[0013] In step 2, an optimized residual feature extraction model is constructed, specifically using the following steps:

[0014] 1): First, build a residual model. The residual model prevents gradient diffusion by establishing an identical mapping relationship during the iteration process, which can improve the accuracy of network training and speed up training. Assume that the input of the neural network layer l is x l , under the condition that the activation function is an identity mapping, the formula is:

[0015] x l =F(x l ,w l )+x l

[0016] Where F(x,w) is the residual function, w l is the weight function. By continuously stacking residual blocks to pass the network feature information system, the output of any layer L and the expression of the l layer are:

[0017]

[0018] where x L represents the output of any L layer.

[0019] 2): Optimize the residual model and add a BN layer before entering the activation function to make the model more regularized.

[0020] In step 3, the construction process of the feature highlighting module is as follows:

[0021] 1) First, construct the main branch, which can be divided into X blocks. Each block consists of a feature extraction layer and a pooling layer. The pooling layer is after the feature extraction layer. The feature extraction layer consists of a convolution kernel of size w*w;

[0022] 2) Construct a mask branch. The mask branch consists of X downsampling layers and Y upsampling layers. The downsampling layer and the upsampling layer are a pair of symmetrical u*u pooling layers. After the feature map is input into the mask branch, it first passes through the u1*u1 downsampling layer, then enters the u2*u2 downsampling layer, and after reaching the lowest resolution, it enters the u2*u2 upsampling layer. Finally, it passes through the u1*u1 upsampling layer to enlarge the feature back to the input size to ensure that the input and output sizes are consistent. The final output feature map of the mask branch is M(x);

[0023] 3) The feature map finally output by the feature highlighting module is the feature map of the fusion of the trunk branch and the mask branch, which can be expressed as:

[0024] H(x)=M(x)*T(x)

[0025] Where H(x) represents the feature map of the fusion of the trunk branch and the mask branch, M(x) is the feature map output by the mask branch, and T(x) is the feature map output by the trunk branch;

[0026] 4) Construct a semantically guided extraction module.

[0027] In step 1), the size of the convolution kernel of the feature extraction layer changes according to the different components of the plate. For example, the feature extraction layer in plate 1 uses m convolution kernels and the number of output channels is b, while plate 2 uses 2m convolution kernels and the number of output channels is 2b, and so on until plate X. The feature map processed by the feature extraction layer is passed to the pooling layer. Assuming that the input feature matrix is ​​h*h, the pooling layer is a u*u matrix, then the h*h input is split into u*u small matrices, and the maximum value of the small matrix in u*u is taken to recombine into a u*u feature matrix. The feature map finally output by the trunk branch is T(x).

[0028] In step 4), the semantic guidance extraction module averages different channels of the same pixel of the fusion feature map. Assuming that the input is a feature map H(x) of H*W*C, the semantic guidance extraction module first performs a maximum pooling and an average pooling to obtain two feature maps of H*W*1. The two feature maps are spliced, and finally the activation function Sigmoid is used for normalization to obtain the weight coefficient Ms(x). Finally, the weight coefficient Ms(x) is multiplied by the input feature map F to obtain the scaled new feature. The formula of the semantic guidance extraction module of the present invention can be expressed as follows:

[0029] F i =(1+Ms(x))*H(x)

[0030] where F i It is the feature map processed by the semantic guidance extraction module, Ms(x) is the weight coefficient, and H(x) is the fused feature map.

[0031] Compared with the prior art, the present invention has the following technical effects:

[0032] 1) The present invention is a semantic segmentation model based on the attention mechanism, which improves the problem of difficulty in segmenting different categories of objects due to the small pixel differences between different objects caused by the influence of climate and seasons through the attention mechanism;

[0033] 2) In the construction of the recognition model, the present invention has designed an optimized residual feature extraction model to more effectively prevent and treat gradient explosion and gradient diffusion, and designed a feature highlighting module to more effectively obtain global information and adjust the weight ratio, increase the weight of the target object, and effectively improve the situation in which the segmentation network has little difference in pixel values ​​in different seasons and the network is difficult to divide the objects. The above improvement methods can greatly improve the accuracy of the detection framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the present invention;

[0035] Figure 2 This is a structural diagram of the optimized residual feature extraction model in the present invention;

[0036] Figure 3 This is a structural diagram of a module that highlights the features of the present invention;

[0037] Figure 4 This is the overall network structure diagram of the present invention. DETAILED DESCRIPTION

[0038] like Figure 1 As shown, a method for extracting spring and winter cultivated land based on the fusion residual attention mechanism includes the following steps:

[0039] Step 1: Dataset creation: Fusing panchromatic and multispectral images, marking cultivated land to generate training labels, and then cropping and preprocessing the images as training sets;

[0040] Step 2: Build an optimized residual feature extraction model, extract features from the training set images, and generate feature maps;

[0041] Step 3: Construct a feature highlighting module, use the generated feature map as input to the residual channel attention module, extract feature maps of different scales, assign different weight features to the acquired feature elements, and highlight the sample feature information;

[0042] Step 4: Crop the validation set images and put them into the recognition model for recognition.

[0043] Step 5: Generate recognition result mask and obtain segmentation result.

[0044] like Figure 2 As shown, in step 2, an optimized residual feature extraction model is constructed, and the following steps are specifically adopted:

[0045] 1) First, build a residual model. The residual model prevents gradient diffusion by establishing an identical mapping relationship during the iteration process, which can improve the accuracy of network training and speed up training. Assume that the input of the neural network layer l is x l , under the condition that the activation function is an identity mapping, the formula is:

[0046] x l =F(x l ,W l )+x l

[0047] Where F(x) is the residual function and W is the weight function. By continuously stacking residual blocks to pass the network feature information system, the output of any layer L and the expression of the l layer are:

[0048]

[0049] The identity mapping is generally represented by stacked residual blocks, which ensures the propagation path and method of the entire network, as well as the integrity of the parameters in back propagation, thereby speeding up network training.

[0050] 2) The residual block is composed of a 1*1, 3*3, and 1*1 feature extraction layer. The feature extraction layer mainly extracts features through convolution operations.

[0051] 3) Build an optimized residual model. After adding the trunk unit and the jump connection, it is an identity mapping, which makes the gradient propagation faster. Therefore, the deeper the network, the more obvious the effect. The trunk unit and the jump connection in the residual model need to be activated by the Relu activation function after adding. Some negative elements in the input matrix will be affected. When there are more residual units, this effect will be more obvious. Therefore, before activation, add a BN layer to normalize the feature map image, so that the model has more regularization ability, and then the residual module structure performs identity mapping to fuse the residual, which makes the segmentation effect better.

[0052] like Figure 3 As shown, the specific steps of constructing the feature highlighting module in step 3 are as follows:

[0053] 1) Since crops in spring and winter cultivated land have not yet grown, the features in drone images are similar to those of bare soil, which can easily lead to misclassification during detection. The present invention proposes a feature highlighting method to enhance the feature information of spring and winter cultivated land, making it less likely to be misclassified with similar objects such as bare soil. The feature highlighting method is based on the idea of ​​identity mapping, and achieves the effect of enhancing features by combining the feature information of the trunk branch and the mask branch to obtain complementary information.

[0054] 2) First, construct the main branch. The main branch can be divided into 5 sections. Each section consists of a feature extraction layer and a pooling layer. The pooling layer is after the feature extraction layer. The feature extraction layer consists of 3*3 convolution kernels. The number of convolution kernels in the feature extraction layer varies according to the different components of the section. As the number of layers increases, the number of convolution kernels used in each section doubles. For example, the feature extraction layer in section 1 uses 64 convolution kernels and the number of output channels is 64. The number of convolution kernels used in section 2 is 128 and the number of output channels is 128. This is analogous to section 5. The feature map processed by the feature extraction layer is passed to the pooling layer. Assuming that the input feature matrix is ​​4*4 and the pooling layer is a 2*2 matrix, the 4*4 input is split into 2*2 small matrices, and the maximum value of the 2*2 small matrices is taken to recombine into a 2*2 feature matrix. The feature map finally output by the main branch is T(x).

[0055] 3) Then construct the mask branch, which consists of 2 downsampling layers and 2 upsampling layers. The downsampling layer and upsampling layer are a pair of symmetrical 2*2 and 4*4 pooling layers. After the feature map is input into the mask branch, it first passes through the 4*4 downsampling layer, and then enters the 2*2 downsampling layer, which can quickly increase the receptive field. After reaching the lowest resolution, it is input into the 2*2 upsampling layer, and finally the 4*4 upsampling layer is used to enlarge the feature back to the input size to ensure that the input and output sizes are consistent. The final output feature map of the mask branch is M(x).

[0056] 4) Construct a semantic guidance extraction module. The semantic guidance extraction module averages different channels of the same pixel of the fusion feature map. Assuming that the input is a feature map H(x) of H*W*C, the semantic guidance extraction module first performs a maximum pooling and an average pooling to obtain two feature maps of H*W*1. These two feature maps are spliced, and finally the activation function Sigmoid is used for normalization to obtain the weight coefficient Ms(x). Finally, the weight coefficient Ms(x) is multiplied by the input feature map F to obtain the scaled new feature. The formula of the semantic guidance extraction module of the present invention can be expressed as follows:

[0057] F i =(1+Ms(x))*H(x)

[0058] where F i It is the feature map processed by the semantic guidance extraction module, Ms(x) is the weight coefficient, and H(x) is the fused feature map.

Claims

1. A method for detecting spring and winter cultivated land based on feature highlighting, characterized in that: The following steps are involved: Step 1: Dataset creation: panchromatic and multispectral images are fused, cultivated land is marked to generate training labels, and then the images are cropped and preprocessed as training sets; Step 2: Build an optimized residual feature extraction model, extract features from the training set images, and generate feature maps; Step 3: Construct a feature highlighting module, input the generated feature map into the feature highlighting module, extract feature maps of different scales, assign different weight features to the acquired feature elements, and highlight the sample feature information; Step 4: Crop the validation set images and put them into the recognition model for recognition; Step 5: Generate recognition result mask and obtain segmentation result; In step 2, an optimized residual feature extraction model is constructed, specifically using the following steps: 1): First, build a residual model. The residual model prevents gradient diffusion by establishing an identical mapping relationship during the iteration process, which can improve the accuracy of network training and speed up training. Assume that the input of the neural network layer l is x l , under the condition that the activation function is an identity mapping, the formula is: x l =F(x l ,w l )+x l Where F(x,w) is the residual function, w l is a weight function, which transmits the network feature information system by continuously stacking residual blocks. The output of any layer L and the expression of the l layer are: where x L represents the output of any L layer; 2): Optimize the residual model and add a BN layer before entering the activation function to make the model more regularized; In step 3, the construction process of the feature highlighting module is as follows: 1) First, construct the main branch, which can be divided into X blocks. Each block consists of a feature extraction layer and a pooling layer. The pooling layer is after the feature extraction layer, and the feature extraction layer consists of a convolution kernel of size w*w; 2) Construct a mask branch. The mask branch consists of X downsampling layers and Y upsampling layers. The downsampling layer and the upsampling layer are a pair of symmetrical u*u pooling layers. After the feature map is input into the mask branch, it first passes through the u1*u1 downsampling layer, then enters the u2*u2 downsampling layer, and after reaching the lowest resolution, it enters the u2*u2 upsampling layer. Finally, it passes through the u1*u1 upsampling layer to enlarge the feature back to the input size to ensure that the input and output sizes are consistent. The final output feature map of the mask branch is M(x); 3) The feature map finally output by the feature highlighting module is the feature map of the fusion of the trunk branch and the mask branch, which can be expressed as: H(x)=M(x)*T(x) Where H(x) represents the feature map of the fusion of the trunk branch and the mask branch, M(x) is the feature map output by the mask branch, and T(x) is the feature map output by the trunk branch; 4) Construct a semantically guided extraction module.

2. The method according to claim 1, characterized in that In step 1), the number of convolution kernels in the feature extraction layer changes according to the different compositions of the plates. The number of convolution kernels used in the nth plate is k times more than that used in the n-1th plate. The feature map processed by the feature extraction layer is passed to the pooling layer. Assuming that the input feature matrix is ​​h*h, the pooling layer is a u*u matrix. The h*h input is split into u*u small matrices, and the maximum value of the u*u small matrices is taken to recombine into a u*u feature matrix. The feature map finally output by the trunk branch is T(x).

3. The method according to claim 1, characterized in that In step 4), the semantic guidance extraction module averages different channels of the same pixel of the fusion feature map. Assuming the input is the feature map H(x) of H*W*C, the semantic guidance extraction module first performs a maximum pooling and an average pooling to obtain two feature maps of H*W*1, and then concatenates the two feature maps. Finally, the activation function Sigmoid is used for normalization to obtain the weight coefficient Ms(x). Finally, the weight coefficient Ms(x) is multiplied by the input feature map F to obtain the scaled new feature. The formula of the semantic guidance extraction module can be expressed as follows: F i =(1+Ms(x))*H(x); where F i It is the feature map processed by the semantic guidance extraction module, Ms(x) is the weight coefficient, and H(x) is the fused feature map.

Images