A method for extracting rapeseed from high-resolution images based on deep learning algorithm

Abstract

The present invention provides a method for extracting rapeseed from high-resolution images based on a deep learning algorithm. The model integrates DRN and an improved U-Net semantic segmentation model, and includes two core modules of Encoder and Decoder, which realize the autonomous learning of spatial and spectral features of GF-2 images and the automatic extraction of rapeseed at the pixel level. The model uses GF-2 images during the rapeseed flowering period (late February to mid-April) as the data source, is trained and tested through labeled rapeseed samples, and increases rapeseed samples through a data enhancement algorithm and obtains an optimized rapeseed extraction model by adjusting model parameters. When applying the model, a data input clipping strategy and a result output clipping strategy are proposed to realize the extraction of rapeseed from input images of any size. Compared with traditional classification analysis methods based on pixels and statistical theories and object-oriented classification algorithms, this method makes full use of the spatial and spectral features of GF-2 images, realizes the autonomous learning of semantic features used for rapeseed extraction, and improves the accuracy of rapeseed extraction and the generalization ability of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and more specifically relates to a method for extracting rapeseed from high-resolution images based on a deep learning algorithm. Background Art

[0002] Rapeseed is the first of my country's five major oil crops (including rapeseed, soybean, peanut, sunflower and sesame), and the only winter oil crop. It is also an important source of edible oil and protein feed. Timely and effective acquisition of winter rapeseed spatial distribution information is of great significance to the adjustment of planting industry structure, food security, national economy and social development.

[0003] Rapeseed production in China is affected by the geographical environment and farmers' planting behavior, and is characterized by fragmented planting. Conventional survey methods are time-consuming and labor-intensive, and it is difficult to meet the needs of routine monitoring of the spatial distribution of rapeseed over a large area; and remote sensing images with medium and low spatial resolutions can hardly meet the resolution requirements for fragmented rapeseed extraction; the successful launch of the Gaofen-2 satellite has provided basic data support for the rapid and accurate monitoring of the spatial distribution of crops and area estimation, but the widespread existence of the "same object, different spectrum" and "different objects, same spectrum" phenomena limits the extraction accuracy of traditional classification and analysis methods based on pixels and statistical theory, and although object-oriented classification algorithms can effectively improve the classification accuracy of remote sensing images, they use limited spatial features such as texture and shape, and have obvious regional differences, which limits the scalability and generalization ability of a single model. Summary of the invention

[0004] In view of the above problems, the present invention provides a method for extracting rapeseed from high-resolution images based on a deep learning algorithm, which has high accuracy and strong generalization ability. The core module of the method is a semantic segmentation model based on deep learning. The model is based on data and self-learns the characteristics of rapeseed in different regions through training to achieve normalized monitoring of the spatial distribution of rapeseed in a large range.

[0005] The specific steps of the present invention to solve the above technical problems are:

[0006] A. Data collection and preprocessing:

[0007] (1) GF-2 image acquisition: Acquire GF-2 images during the rapeseed flowering period (late February to mid-April);

[0008] (2) GF-2 image data preprocessing: The acquired GF-2 images are subjected to radiometric calibration, orthorectification, image registration, atmospheric correction, and image fusion preprocessing in sequence;

[0009] B. Deep learning model training and testing:

[0010] (1) Model training and test sample library construction: After preprocessing, rapeseed is annotated on the GF-2 image and cropped into a 600×600 rapeseed sample library. Three-quarters of the sample data in the sample library are randomly selected as training samples, and one-quarter of the sample data are used as test samples.

[0011] (2) Deep learning model construction: Based on the Pytorch open source deep learning framework, Python programming is used to build a semantic segmentation model that integrates DRN and improved U-Net;

[0012] (3) Deep learning model training: Use the He weight initialization value, Adam parameter optimization algorithm, and set hyperparameters such as batch size, learning rate, and epochs. Input the training samples into the model to perform autonomous learning of the model.

[0013] (4) Deep learning model testing: Use F1-Score as the model accuracy evaluation indicator. If the average F1-Score of the model trained in step (3) for the test samples is ≥ 90%, skip step (5) and save the model; if the average F1-Score of the model trained in step (3) for the test samples is < 90%, execute step (5);

[0014] (5) Deep learning model optimization: Model optimization is achieved by increasing training samples and adjusting model hyperparameters: a. Increasing training samples: Collect more GF-2 images and repeat steps (1)-(5); If the collected GF-2 images cannot meet the accuracy requirements of the model application, increase the training samples through data enhancement and repeat steps (3)-(5); b. Adjusting model hyperparameters: Adjust hyperparameters including batch size, learning rate, and epoch, and repeat steps (3)-(5).

[0015] C. Model application:

[0016] (1) Collect Gaofen-2 images from March to April in the application area;

[0017] (2) Perform data preprocessing on the collected GF-2 images;

[0018] (3) Using the vector layer of the region of interest to crop the GF-2 image obtained in step (2);

[0019] (4) Input the high-resolution image of the area of ​​interest obtained in step (3) into the deep learning model saved in step B to obtain the rapeseed extraction result.

[0020] Preferably, the GF-2 images collected in step A above need to be acquired during the rapeseed flowering period (late February to mid-April). Rapeseed during this period has characteristics that are significantly different from other landforms, and is the best period for remote sensing rapeseed monitoring.

[0021] Preferably, the collected GF-2 image data is preprocessed in the above step A to eliminate the influence of terrain, sensor imaging process, atmospheric influence, etc., and realize the fusion of multispectral image and panchromatic image, so that the obtained image has both multispectral and high spatial resolution characteristics, which is specifically achieved in the following steps:

[0022] (1) Radiometric calibration: Convert the DN value of the GF-2 image into the apparent radiance (L λ ):

[0023] L λ =Gain λ ×DN+Offset λ

[0024] Where Gain λ and Offset λ They represent the gain and offset at different wavelengths (λ), respectively, and are provided by China Resources Satellite Application Center.

[0025] (2) Orthorectification: Using the Digital Elevation Model (DEM), the RPC (Rational Polynomial Coefficient) orthorectification model provided by ENVI software is used to eliminate the parallax (coordinate offset) caused by ground elevation.

[0026] (3) Image registration: GF-2 image registration is used to eliminate the problem that the same features in the overlapping area of ​​(a) GF-2 multispectral and panchromatic images, and (b) GF-2 and reference images may not overlap. Image registration is achieved through the Image Registration Workflow tool provided by ENVI, which includes two processes: tie point generation and resampling. In particular, it is sometimes difficult to meet the accuracy requirements through the automatic tie point generation algorithm. It is necessary to remove the points with large errors in the automatically generated tie points and manually add tie points to meet the requirement that there are enough tie points randomly distributed in the overlapping area of ​​the two images.

[0027] (4) Atmospheric correction: The FLASSH atmospheric correction model is used to perform atmospheric correction on the GF-2 multispectral image to eliminate the influence of scattering such as water vapor and aerosol in the atmosphere on the surface reflectivity, and obtain the surface reflectivity data of the GF-2 image.

[0028] (5) Image fusion: Using the Gram-Schmidt Pan Sharpening image fusion algorithm built into ENVI, with the GF-2 multispectral image and panchromatic image as input, multispectral image data with high spatial resolution is obtained.

[0029] Preferably, the deep learning model established in step B above integrates DRN and the improved U-Net semantic segmentation model to achieve automatic extraction of multi-layer semantic features and output rapeseed extraction results with the same size as the input image. The model includes two modules: Encoder and Decoder:

[0030] (1) The Encoder module implements the extraction of multi-layer semantic features with reference to the DRN-C-26 model. This model is implemented by improving the ResNet-18 model. The pooling layer in Layer 0-4 of the original model is replaced by a convolutional layer with a stride of 2 to achieve the downsampling effect. At the same time, the pooling layer in Layer 5 and Layer 6 is removed, and the dilation convolution is introduced to achieve the effect of expanding the receptive field. In addition, in order to eliminate the stripe feature error that may be caused by the dilation convolution, two layers of neural networks, Layer 7 and Layer 8, are added. Each layer of the neural network contains two Conv-BN-Relu combination operations, where Conv (Convolution) represents convolution operation; BN (BatchNormalization) represents batch normalization processing; and Relu represents the Relu activation function.

[0031] (2) The Decoder module is implemented through an improved U-Net semantic segmentation model. The feature map output by Layer 8 of the Encoder module is upsampled once and fused with the feature map output by Layer 3 in the channel dimension. After two convolutions, the upsampling-fusion-convolution process is repeated twice. Finally, the prediction results of two channels (representing the prediction probabilities of rapeseed and background, respectively) are output through two layers of 64-dimensional 3×3 convolution and one layer of 2-dimensional 1×1 convolution. The dimension with the maximum value of the channel dimension is calculated to realize the extraction of the spatial distribution of rapeseed (1 represents rapeseed; 0 represents background).

[0032] Preferably, the deep learning model training in step B above uses the He weight initial value. The He initial value uses a standard deviation of Gaussian distribution of , where n is the number of nodes in the previous layer of neural network.

[0033] Preferably, the deep learning model training in step B above uses the Adam parameter optimization algorithm. The Adam optimization algorithm is an extension of the stochastic gradient descent algorithm, which uses momentum and adaptive learning rate to accelerate the model convergence speed.

[0034] Preferably, the model in step B above uses F1-Score for accuracy assessment. F1-Score is the harmonic mean of precision (P, Precision) and recall (R, Recall):

[0035] Right now

[0036] In the formula, the precision rate P represents the proportion of pixels predicted as rapeseed that are actually rapeseed pixels; the recall rate R represents the proportion of all correct rapeseed pixels that are correctly predicted as rapeseed. P and R are calculated based on the confusion matrix:

[0037]

[0038] Where TP (True Positive) is the number of pixels in the confusion matrix that are actually rapeseed but predicted to be rapeseed; FP (False Positive) is the number of pixels that are actually background but predicted to be rapeseed; FN (False Negative) is the number of pixels that are actually rapeseed but predicted to be background.

[0039] Preferably, during the deep learning model optimization process in the above step B, if the collected GF-2 images are difficult to meet the accuracy requirements of the model application, data enhancement methods are used to increase training samples, such as flipping, rotating, and setting overlapping areas for adjacent samples.

[0040] Preferably, when applying the deep learning model in step C above, since the deep learning model input requires a size of 600×600, it is necessary to discuss separately according to the size (Width×Height) of the GF-2 image of the area of ​​interest:

[0041] (1) If Width<600 and Height<600, fill the original GF-2 image with 0 to make it a 600×600 image as the input of the deep learning model, and crop the model output result to the size of Width×Height;

[0042] (2) If Width<600, Height=600, fill the original GF-2 image with 0 to a 600×600 image as the input of the deep learning model, and crop the model output result to a size of Width×600;

[0043] (3) If Width<600, Height>600:

[0044] a. Fill the original GF-2 image with 0 to make it 600×(Height+40) in size;

[0045] b. Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0046] c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result consistent with the size of interest (Width×Height).

[0047] (4) If Width = 600 and Height < 600, fill the original GF-2 image with 0 to a 600 × 600 image as the input of the deep learning model, and crop the model output result to a size of 600 × Height;

[0048] (5) If Width = 600, Height = 600, the GF-2 image is directly input into the model to obtain the rapeseed extraction result;

[0049] (6) If Width = 600, Height > 600, the method is the same as (3);

[0050] (7) If Width>600, Height<600:

[0051] a. Fill the original GF-2 image with 0 to make it (Width+40)×600;

[0052] b. Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0053] c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result consistent with the size of interest (Width×Height).

[0054] (8) If Width>600, Height=600, the method is the same as (7);

[0055] (9) If Width>600, Height>600:

[0056] a. Fill the original GF-2 image with 0 to make it (Width+40)×(Height+40);

[0057] b. Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0058] c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result consistent with the size of interest (Width×Height).

[0059] Preferably, in the above step C, when applying the deep learning model, for any input image, if its Width or Height is less than 600, a padding method is used to meet the model input requirements;

[0060] Preferably, in the above step C, when applying the deep learning model, for any input image, if its Width or Height is greater than 600, the data input cropping strategy and the result output cropping strategy are adopted to meet the model input requirements and output the rapeseed extraction result of the same size as the original input image, and eliminate the mutation of the extraction result at the boundary of adjacent samples. The following takes the input image of size Width×Height (Width>600, Height>600) as an example (denoted as image_ori) to illustrate the data input cropping strategy and the result output cropping strategy:

[0061] (1) Data input clipping strategy: After clipping, adjacent small samples are overlapped by 40 rows or columns. The specific steps are as follows:

[0062] a. First, add 20 rows or columns around the input image and fill them with 0, so that the image size becomes (Width+40)×(Height+40), and the resulting image is recorded as image_expand;

[0063] b. Calculate the four important parameters that determine the number of small samples after pruning and the sampling range of the last row and the last column of small samples:

[0064]

[0065] m_rem=Width\560

[0066] n_rem=Height\560

[0067] In the formula Indicates rounding up operation, \ indicates remainder operation, m determines the number of columns of the original image cropped into small samples, n determines the number of rows of the original image cropped into small samples, m_rem and n_rem determine the cropping range of the last column or row in the original image.

[0068] c. Determine the sampling range of each small sample based on the values ​​of m, n, m_rem and n_rem:

[0069] ① Determine the sampling range (range_x_min, range_x_max) of the i-th column sample (i=1,2,3,...,m) in image_expand:

[0070] If i=1,2,3,...,m-1, then range_x_min=600×(i-1)+1, range_x_max=600×(i-1)+600;

[0071] If i=m, then range_x_min=Width+40-599, range_x_max=Width+40.

[0072] ② Determine the sampling range (range_y_min, range_y_max) of the j-th row of samples (j=1,2,3,...,n) in image_expand:

[0073] If j=1,2,3,...,n-1, then range_y_min=600×(j-1)+1, range_y_max=600×(j-1)+600;

[0074] If j=n, then range_y_min=Height+40-599, range_y_max=Height+40.

[0075] (2) Result output clipping strategy: Determine the sampling range of the model for each small sample output result based on the values ​​of m, n, m_rem and n_rem:

[0076] ① Determine the sampling range (out_x_min, out_x_max) of the model's output results for the i-th column of small samples (i = 1, 2, 3, ..., m):

[0077] If i=1,2,3,……,m-1, then out_x_min=21,out_x_max=580;

[0078] If i=m and m_rem=0, then out_x_min=21, out_x_max=580;

[0079] If i=m and m_rem≠0, then out_x_min=580-m_rem+1, out_x_max=580.

[0080] ② Determine the sampling range (out_y_min, out_y_max) of the model's output results for the j-th row of small samples (j = 1, 2, 3, ..., n):

[0081] If j=1,2,3,……,n-1, then out_y_min=21,out_y_max=580;

[0082] If j=m and n_rem=0, then out_y_min=21, out_y_max=580;

[0083] If j=m and n_rem≠0, then out_y_min=580-n_rem+1, out_y_max=580.

[0084] The beneficial effects of the present invention are:

[0085] The present invention provides a method for extracting rapeseed from high-resolution images based on a deep learning algorithm. The model integrates DRN and an improved U-Net semantic segmentation model. The semantic segmentation model based on CNN realizes the autonomous learning of image spatial features and spectral features. The deep learning model including two core modules of Encoder and Decoder realizes the automatic extraction of rapeseed at the pixel level. The model uses the GF-2 image during the rapeseed flowering period (late February to mid-April) as the data source, is trained and tested through labeled rapeseed samples, and increases rapeseed samples through a data enhancement algorithm and obtains an optimized rapeseed extraction model by adjusting hyperparameters. When applying the model, a data input clipping strategy and a result output clipping strategy are proposed to realize the extraction of rapeseed from input images of any size. Compared with the traditional classification analysis method based on pixels and statistical theory and the object-oriented classification algorithm, this method makes full use of the spatial and spectral features of the GF-2 image, realizes the autonomous learning of semantic features that are conducive to rapeseed extraction, and improves the accuracy of rapeseed extraction and the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0087] Figure 1 It is the overall technical flow chart of the present invention;

[0088] Figure 2 This is a deep learning model diagram in the present invention;

[0089] Figure 3 This is the data input clipping strategy and result output clipping strategy used when applying the deep learning model in the present invention. DETAILED DESCRIPTION

[0090] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0091] The implementation of this case is as follows:

[0092] Step 1: Data collection and preprocessing:

[0093] (1) GF-2 image acquisition: Acquire GF-2 images during the rapeseed flowering period (late February to mid-April);

[0094] (2) GF-2 image data preprocessing: The acquired GF-2 images are preprocessed in sequence by radiometric calibration, orthorectification, image registration, atmospheric correction and image fusion. The specific steps are as follows:

[0095] a. Radiometric calibration: Convert the DN value of the GF-2 image into the apparent radiance (L λ ):

[0096] L λ =Gain λ ×DN+Offset λ

[0097] Where Gain λ and Offset λ They represent the gain and offset at different wavelengths (λ), respectively, and are provided by China Resources Satellite Application Center.

[0098] b. Orthorectification: Using the Digital Elevation Model (DEM), the RPC (Rational Polynomial Coefficient) orthorectification model provided by ENVI software is used to eliminate the parallax (coordinate offset) caused by ground elevation.

[0099] c. Image registration: GF-2 image registration is used to eliminate the problem that the same features in the overlapping area of ​​(a) GF-2 multispectral and panchromatic images, and (b) GF-2 and reference images may not overlap. Image registration is achieved through the Image Registration Workflow built into ENVI, which includes two processes: tie point generation and resampling. In particular, it is sometimes difficult to meet the accuracy requirements through the automatic tie point generation algorithm. It is necessary to remove the points with large errors in the automatically generated tie points and manually add tie points to meet the requirement that there are enough tie points randomly distributed in the overlapping area of ​​the two images.

[0100] d. Atmospheric correction: Use the FLASSH atmospheric correction model to perform atmospheric correction on the GF-2 multispectral image to eliminate the influence of scattering such as water vapor and aerosol in the atmosphere on the surface reflectivity, and obtain the surface reflectivity data of the GF-2 image.

[0101] e. Image fusion: Using the Gram-Schmidt Pan Sharpening image fusion algorithm built into ENVI, with GF-2 multispectral image and panchromatic image as input, multispectral image data with high spatial resolution is obtained.

[0102] Step 2: Deep learning model training and testing:

[0103] (1) Model training and test sample library construction: After preprocessing, rapeseed is annotated on the GF-2 image and cropped into a 600×600 rapeseed sample library. Three-quarters of the sample data in the sample library are randomly selected as training samples, and one-quarter of the sample data are used as test samples.

[0104] (2) Deep learning model construction: Based on the Pytorch open source deep learning framework, Python programming is used to build a semantic segmentation model that integrates DRN and improved U-Net. The model includes two modules: encoder and decoder.

[0105] a. The Encoder module refers to the DRN-C-26 model to extract multi-layer semantic features. This model is implemented by improving the ResNet-18 model. The convolution layer with a step size of 2 replaces the pooling layer in Layer 0-4 of the original model to achieve the downsampling effect. At the same time, the pooling layer in Layer 5 and Layer 6 is removed, and the dilation convolution is introduced to achieve the effect of expanding the receptive field. In addition, in order to eliminate the stripe feature error that may be caused by the dilation convolution, two layers of neural networks, Layer 7 and Layer 8, are added. Each layer of the neural network contains two Conv-BN-Relu combination operations, where Conv (Convolution) represents convolution operation; BN (Batch Normalization) represents batch normalization processing; Relu represents Relu activation function.

[0106] b. The Decoder module is implemented through an improved U-Net semantic segmentation model. The feature map output by Layer 8 of the Encoder module is upsampled once and fused with the feature map output by Layer 3 in the channel dimension. After two convolutions, the upsampling-fusion-convolution process is repeated twice. Finally, the prediction results of two channels (representing the prediction probabilities of rapeseed and background, respectively) are output through 2 layers of 64-dimensional 3×3 convolution and 1 layer of 2-dimensional 1×1 convolution. The dimension with the maximum value of the channel dimension is calculated to realize the extraction of the spatial distribution of rapeseed (1 represents rapeseed; 0 represents background).

[0107] (3) Deep learning model training: Use the He weight initialization value, Adam parameter optimization algorithm, and set hyperparameters such as batch size, learning rate, and epochs. Input the training samples into the model to perform autonomous learning of the model.

[0108] (4) Deep learning model testing: Use F1-Score as the model accuracy evaluation indicator. If the average F1-Score of the model trained in step (3) for the test samples is ≥ 90%, skip step (5) and save the model; if the average F1-Score of the model trained in step (3) for the test samples is < 90%, execute step (5). F1-Score is the harmonic mean of precision (P, Precision) and recall (R, Recall):

[0109] Right now

[0110] In the formula, the precision rate P represents the proportion of pixels predicted as rapeseed that are actually rapeseed pixels; the recall rate R represents the proportion of all correct rapeseed pixels that are correctly predicted as rapeseed. P and R are calculated based on the confusion matrix:

[0111]

[0112] Where TP is the number of pixels in the confusion matrix that are actually rapeseed but predicted to be rapeseed; FP is the number of pixels that are actually background but predicted to be rapeseed; and FN is the number of pixels that are actually rapeseed but predicted to be background.

[0113] (5) Deep learning model optimization: Model optimization is achieved by increasing training samples and adjusting model hyperparameters: a. Increasing training samples: Collect more GF-2 images and repeat steps (1)-(5); If the collected GF-2 images cannot meet the accuracy requirements of the model application, increase training samples through data enhancement methods (such as flipping, rotation, setting overlapping areas for adjacent samples, etc.), and repeat steps (3)-(5); b. Adjusting model hyperparameters: Adjust hyperparameters including batch size, learning rate, and epoch number, and repeat steps (3)-(5).

[0114] Step 3: Model application:

[0115] (1) Collect Gaofen-2 images from March to April in the application area;

[0116] (2) Perform data preprocessing on the collected GF-2 images;

[0117] (3) Using the vector layer of the region of interest to crop the GF-2 image obtained in step (2);

[0118] (4) Input the GF-2 image of the area of ​​interest obtained in step (3) into the deep learning model saved in step B to obtain the rapeseed extraction result. Since the deep learning model input requires a size of 600×600, it is necessary to discuss separately according to the size (Width×Height) of the GF-2 image of the area of ​​interest:

[0119] a. If Width<600, Height<600, fill the original GF-2 image with 0 to make it a 600×600 image as the input of the deep learning model, and crop the model output result to the size of Width×Height;

[0120] b. If Width<600, Height=600, fill the original GF-2 image with 0 to a 600×600 image as the input of the deep learning model, and crop the model output result to a size of Width×600;

[0121] c. If Width<600, Height>600:

[0122] ① Fill the original GF-2 image with 0 to make it 600×(Height+40) in size;

[0123] ② Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0124] ③ Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result with the same size of interest (Width×Height).

[0125] d. If Width = 600 and Height < 600, fill the original GF-2 image with 0 to a 600×600 image as the input of the deep learning model, and crop the model output result to a size of 600×Height;

[0126] e. If Width = 600, Height = 600, directly input the GF-2 image into the model to obtain the rapeseed extraction result;

[0127] f. If Width=600, Height>600, the method is the same as c.;

[0128] g. If Width>600, Height<600:

[0129] ① Fill the original GF-2 image with 0 to make it (Width+40)×600 size;

[0130] ② Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0131] ③ Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result with the same size of interest (Width×Height).

[0132] h. If Width>600, Height=600, the method is the same as g.;

[0133] i. If Width>600, Height>600:

[0134] ①. Fill the original GF-2 image with 0 to make it (Width+40)×(Height+40) size image;

[0135] ②. Use the data input cropping strategy (increasing the overlap area) to crop the image into a small sample of 600×600 as the input of the deep learning model;

[0136] ③. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result consistent with the size of interest (Width×Height).

[0137] In the above discussion, for images of size Width×Height (Width or Height greater than 600), the data input cropping strategy and the result output cropping strategy are used to meet the model input requirements and output rapeseed extraction results of the same size as the original input image, and eliminate the mutation of the extraction results at the boundaries of adjacent samples. The following takes the input image of size Width×Height (Width>600, Height>600) as an example (denoted as image_ori) to illustrate the data input cropping strategy and the result output cropping strategy:

[0138] Data input trimming strategy: After trimming, adjacent small samples are overlapped by 40 rows or columns. The specific steps are as follows:

[0139] a. First, add 20 rows or columns around the input image and fill them with 0, so that the image size becomes (Width+40)×(Height+40), and the resulting image is recorded as image_expand;

[0140] b. Calculate the four important parameters that determine the number of small samples after pruning and the sampling range of the last row and the last column of small samples:

[0141]

[0142] m_rem=Width\560

[0143] n_rem=Height\560

[0144] In the formula Indicates rounding up operation, \ indicates remainder operation, m determines the number of columns of the original image cropped into small samples, n determines the number of rows of the original image cropped into small samples, m_rem and n_rem determine the cropping range of the last column or row in the original image.

[0145] c. Determine the sampling range of each small sample based on the values ​​of m, n, m_rem and n_rem:

[0146] ① Determine the sampling range (range_x_min, range_x_max) of the i-th column sample (i=1,2,3,...,m) in image_expand:

[0147] If i=1,2,3,...,m-1, then range_x_min=600×(i-1)+1, range_x_max=600×(i-1)+600;

[0148] If i=m, then range_x_min=Width+40-599, range_x_max=Width+40.

[0149] ② Determine the sampling range (range_y_min, range_y_max) of the j-th row of samples (j=1,2,3,...,n) in image_expand:

[0150] If j=1,2,3,...,n-1, then range_y_min=600×(j-1)+1, range_y_max=600×(j-1)+600;

[0151] If j=n, then range_y_min=Height+40-599, range_y_max=Height+40.

[0152] Result output clipping strategy: Determine the sampling range of the model's output results for each small sample based on the values ​​of m, n, m_rem, and n_rem:

[0153] ① Determine the sampling range (out_x_min, out_x_max) of the model's output results for the i-th column of small samples (i = 1, 2, 3, ..., m):

[0154] If i=1,2,3,……,m-1, then out_x_min=21,out_x_max=580;

[0155] If i=m and m_rem=0, then out_x_min=21, out_x_max=580;

[0156] If i=m and m_rem≠0, then out_x_min=580-m_rem+1, out_x_max=580.

[0157] ② Determine the sampling range (out_y_min, out_y_max) of the model's output results for the j-th row of small samples (j = 1, 2, 3, ..., n):

[0158] If j=1,2,3,……,n-1, then out_y_min=21,out_y_max=580;

[0159] If j=m and n_rem=0, then out_y_min=21, out_y_max=580;

[0160] If j=m and n_rem≠0, then out_y_min=580-n_rem+1, out_y_max=580.

Claims

1. A high-resolution image rapeseed extraction method based on deep learning algorithm, with semantic segmentation model as the core, integrating DRN and improved U-Net semantic segmentation model, including two modules of encoding (Encoder) and decoding (Decoder), characterized in that: The following steps are involved: A. Data collection and preprocessing: (1) GF-2 image acquisition: Obtain GF-2 images from the end of February to mid-April during the rapeseed flowering period; (2) GF-2 image data preprocessing: The acquired GF-2 images are subjected to radiometric calibration, orthorectification, image registration, atmospheric correction and image fusion preprocessing in sequence; B. Deep learning model training and testing: (1) Model training and test sample library construction: After preprocessing, rapeseed is annotated on the GF-2 image and cropped into a 600×600 rapeseed sample library. Three-quarters of the sample data in the sample library are randomly selected as training samples, and one-quarter of the sample data are used as test samples. (2) Deep learning model construction: Based on the Pytorch open source deep learning framework, Python programming is used to build a semantic segmentation model that integrates DRN and improved U-Net; (3) Deep learning model training: Use the He weight initialization value, Adam parameter optimization algorithm, and set the batch size, learning rate, and epoch hyperparameters. Input the training samples into the model to perform autonomous learning of the model. (4) Deep learning model testing: Use F1-Score as the model accuracy evaluation indicator. If the average F1-Score of the model trained in step (3) for the test samples is ≥ 90%, skip step (5) and save the model; if the average F1-Score of the model trained in step (3) for the test samples is < 90%, execute step (5); (5) Deep learning model optimization: Model optimization is achieved by increasing training samples and adjusting model hyperparameters: a. Increasing training samples: Collect more GF-2 images and repeat steps (1)-(5); If the collected GF-2 images cannot meet the accuracy requirements of model application, increase training samples through data enhancement and repeat steps (3)-(5); b. Adjusting model hyperparameters: Adjust hyperparameters including batch size, learning rate, and epochs, and repeat steps (3)-(5); C. Model application: (1) Collect Gaofen-2 images from March to April in the application area; (2) Perform data preprocessing on the collected GF-2 images; (3) Using the vector layer of the region of interest to crop the GF-2 image obtained in step (2); (4) inputting the GF-2 image of the area of ​​interest obtained in step (3) into the deep learning model saved in step B to obtain the rapeseed extraction result; The deep learning model in step B above integrates DRN and the improved U-Net semantic segmentation model to achieve automatic extraction of multi-layer semantic features and output rapeseed extraction results with the same size as the input image; the model includes two modules: encoding (Encoder) and decoding (Decoder): (1) The Encoder module implements the extraction of multi-layer semantic features with reference to the DRN-C-26 model. This model is implemented by improving the ResNet-18 model. The pooling layer in Layer 0-4 of the original model is replaced by a convolution layer with a step size of 2 to achieve the downsampling effect. At the same time, the pooling layer in Layer 5 and Layer 6 is removed, and the dilation convolution is introduced to achieve the effect of expanding the receptive field. In addition, in order to eliminate the stripe feature error that may be caused by the dilation convolution, two layers of neural networks, Layer 7 and Layer 8, are added. Each layer of the neural network contains two Conv-BN-Relu combination operations, where Conv (Convolution) represents the convolution operation; BN (BatchNormalization) represents the batch normalization processing. Relu represents the Relu activation function; (2) The Decoder module is implemented through an improved U-Net semantic segmentation model. The feature map output by Layer 8 of the Encoder module is upsampled once and fused with the feature map output by Layer 3 in the channel dimension. After two convolutions, the upsampling-fusion-convolution process is repeated twice. Finally, two layers of 64-dimensional 3×3 convolutions and one layer of 2-dimensional 1×1 convolution are used to output the prediction results of two channels, which represent the prediction probabilities of rapeseed and background respectively. The dimension with the maximum value of the channel dimension is calculated to realize the extraction of the spatial distribution of rapeseed, where 1 represents rapeseed and 0 represents background.

2. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that The period for acquiring the GF-2 image in step A above is from the end of February to mid-April, the rapeseed flowering season. The rapeseed in this period has characteristics that are significantly different from other landforms, making it the best period for remote sensing rapeseed monitoring.

3. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that The GF-2 image data preprocessing in step A above eliminates the influence of terrain, sensor imaging process, and atmospheric influence, and realizes the fusion of multispectral image and panchromatic image. The obtained image has both multispectral and high spatial resolution characteristics, which is specifically implemented in the following steps: (1) Radiometric calibration: Convert the DN value of the GF-2 image into the apparent radiance (L λ ): L λ =Gain λ ×DN+Offset λ Where Gain λ and Offset λ They represent the gain and offset at different wavelengths (λ), respectively, and are provided by China Resources Satellite Application Center; (2) Orthorectification: Using the Digital Elevation Model (DEM), the RPC (Rational Polynomial Coefficient) orthorectification model provided by ENVI software is used to eliminate the parallax caused by ground elevation. (3) Image registration: GF-2 image registration is used to eliminate the problem that the same features in the overlapping area of ​​(a) GF-2 multispectral and panchromatic images and (b) GF-2 and reference images may not overlap. Image registration is achieved through the Image Registration Workflow tool provided by ENVI, which includes two processes: tie point generation and resampling. The automatic tie point generation algorithm is sometimes difficult to meet the accuracy requirements. It is necessary to remove the tie points with large errors in the automatically generated tie points and manually add tie points to ensure that there are enough tie points randomly distributed in the overlapping area of ​​the two images. (4) Atmospheric correction: The FLASSH atmospheric correction model is used to perform atmospheric correction on the GF-2 multispectral image to eliminate the influence of water vapor and aerosol scattering in the atmosphere on the surface reflectivity, and obtain the surface reflectivity data of the GF-2 image; (5) Image fusion: Using the Gram-Schmidt Pan Sharpening image fusion algorithm built into ENVI, with the GF-2 multispectral image and panchromatic image as input, multispectral image data with high spatial resolution is obtained.

4. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that In the above step B, the model training uses the initial value of He weight; the standard deviation of the initial value of He is Gaussian distribution of , where n is the number of nodes in the previous layer of neural network.

5. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that In the above step B, the deep learning model training uses the Adam parameter optimization algorithm; the Adam optimization algorithm is an extension of the stochastic gradient descent algorithm, which uses momentum and adaptive learning rate to accelerate the model convergence speed.

6. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that In step B above, F1-Score is used to evaluate the accuracy of the model; F1-Score is the harmonic mean of precision (P, Precision) and recall (R, Recall): Right now In the formula, the precision rate P represents the proportion of pixels predicted as rapeseed that are actually rapeseed pixels; the recall rate R represents the proportion of all correct rapeseed pixels that are correctly predicted as rapeseed; P and R are calculated based on the confusion matrix: Where TP (True Positive) is the number of pixels in the confusion matrix that are actually rapeseed but predicted to be rapeseed; FP (False Positive) represents the number of pixels that are actually background but predicted to be rapeseed; FN (False Negative) represents the number of pixels that are actually rapeseed but predicted to be background.

7. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that In the deep learning model optimization process in step B above, if the collected GF-2 images are difficult to meet the accuracy requirements of the model application, the data enhancement method is used to increase the training samples.

8. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 1, characterized in that When the model is applied in step C above, since the deep learning model input requires a size of 600×600, it is necessary to discuss separately according to the size (Width×Height) of the GF-2 image in the area of ​​interest: (1) If Width<600 and Height<600, fill the original GF-2 image with 0 to make it a 600×600 image as the input of the deep learning model, and crop the model output result to the size of Width×Height; (2) If Width<600, Height=600, fill the original GF-2 image with 0 to a 600×600 image as the input of the deep learning model, and crop the model output result to a size of Width×600; (3) If Width<600, Height>600: a. Fill the original GF-2 image with 0 to make it 600×(Height+40) in size; b. Use the data input cropping strategy to crop the image into a small sample of 600×600 size as the input of the deep learning model; c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result of the same size (Width×Height) as the interested size; (4) If Width = 600 and Height < 600, fill the original GF-2 image with 0 to a 600 × 600 image as the input of the deep learning model, and crop the model output result to a size of 600 × Height; (5) If Width = 600, Height = 600, the GF-2 image is directly input into the model to obtain the rapeseed extraction result; (6) If Width = 600, Height > 600, the method is the same as (3); (7) If Width>600, Height<600: a. Fill the original GF-2 image with 0 to make it (Width+40)×600; b. Use the data input cropping strategy to crop the image into a small sample of 600×600 size as the input of the deep learning model; c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result of the same size (Width×Height) as the interested size; (8) If Width>600, Height=600, the method is the same as (7); (9) If Width>600, Height>600: a. Fill the original GF-2 image with 0 to make it (Width+40)×(Height+40); b. Use the data input cropping strategy to crop the image into a small sample of 600×600 size as the input of the deep learning model; c. Input the cropped small sample into the model, adopt the result output cropping strategy and merge the output results to obtain the rapeseed extraction result consistent with the size of interest (Width×Height).

9. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 8, characterized in that For any input image, if its Width or Height is less than 600, the filling method is used to meet the model input requirements.

10. The method for extracting rapeseed from high-resolution images based on a deep learning algorithm according to claim 9, characterized in that For any input image, if its Width or Height is greater than 600, the data input cropping strategy and the result output cropping strategy are used to meet the model input requirements and output the rapeseed extraction results that are consistent with the original input image size, and eliminate the mutation of the extraction results at the boundaries of adjacent samples; assuming that the size of the input image is Width×Height, Width>600, Height>600, recorded as image_ori, the data input cropping strategy and the result output cropping strategy are explained: (1) Data input clipping strategy: After clipping, adjacent small samples are overlapped by 40 rows or columns. The specific steps are as follows: a. First, add 20 rows or columns around the input image and fill them with 0, so that the image size becomes (Width+40)×(Height+40), and the resulting image is recorded as image_expand; b. Calculate the four important parameters that determine the number of small samples after trimming and the sampling range of each small sample: m_rem=Width\560 n_rem=Height\560 In the formula Indicates rounding up operation, \ indicates remainder operation, m determines the number of columns of the original image cropped into small samples, n determines the number of rows of the original image cropped into small samples, and m_rem and n_rem determine the cropping range of the last column or row in the original image; c. Determine the sampling range of each small sample based on the values ​​of m, n, m_rem and n_rem: ① Determine the sampling range (range_x_min, range_x_max) of the i-th column sample (i=1,2,3,...,m) in image_expand: If i=1,2,3,...,m-1, then range_x_min=600×(i-1)+1, range_x_max=600×(i-1)+600; If i=m, then range_x_min=Width+40-599, range_x_max=Width+40; ② Determine the sampling range (range_y_min, range_y_max) of the j-th row of samples (j=1,2,3,...,n) in image_expand: If j=1,2,3,...,n-1, then range_y_min=600×(j-1)+1, range_y_max=600×(j-1)+600; If j=n, then range_y_min=Height+40-599, range_y_max=Height+40; (2) Result output clipping strategy: Determine the sampling range of the model for each small sample output result based on the values ​​of m, n, m_rem and n_rem: ① Determine the sampling range (out_x_min, out_x_max) of the model's output results for the i-th column of small samples (i = 1, 2, 3, ..., m): If i=1,2,3,……,m-1, then out_x_min=21,out_x_max=580; If i=m and m_rem=0, then out_x_min=21, out_x_max=580; If i=m and m_rem≠0, then out_x_min=580-m_rem+1, out_x_max=580; ② Determine the sampling range (out_y_min, out_y_max) of the model's output results for the j-th row of small samples (j = 1, 2, 3, ..., n): If j=1,2,3,……,n-1, then out_y_min=21,out_y_max=580; If j=m and n_rem=0, then out_y_min=21, out_y_max=580; If j=m and n_rem≠0, then out_y_min=580-n_rem+1, out_y_max=580.

Images