A rapid detection method for plastic greenhouses based on remote sensing images

Abstract

The present invention discloses a rapid detection method for plastic greenhouses based on remote sensing images, which can intelligently detect plastic greenhouses in remote sensing images and obtain the length, width, area and location information of each plastic greenhouse in the remote sensing images. Based on remote sensing images, a deep learning method is used to extract and detect plastic greenhouses in the image, and a vector file of its spatial distribution is generated to obtain the information and total number of each plastic greenhouse. This method can show that deep learning is an efficient method in the detection of plastic greenhouses in remote sensing images, thereby realizing the monitoring and mapping of plastic greenhouses in the study area. It is of great significance to optimize the agricultural structure, promote the development of modern agriculture, and promote the implementation of the rural revitalization strategy.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing images and deep learning technology, and in particular to a method for detecting and extracting ground objects based on target detection technology in computer vision. Background Art

[0002] As a kind of agricultural facility with a protective nature, plastic greenhouses can overcome the contradiction between the growth requirements of crops and the local natural conditions in agricultural production. Greenhouses have completely changed agriculture around the world and greatly improved the productivity of vegetable production. As of 2016, the area of ​​plastic greenhouses in China has increased to 981,000 hectares, an increase of 110% compared with 2006. Although plastic greenhouses have made great contributions to agricultural development, they will bring some environmental problems. Plastic waste will also cause soil pollution, biodiversity degradation and other problems. Therefore, the monitoring and mapping of plastic greenhouses is of great significance for the development planning of modern agriculture and understanding the impact of plastic landscapes on the environment.

[0003] Remote sensing has the visibility and frequency of a wide geographical area, and has become a key technology for capturing surface information, providing the possibility for rapid and large-scale detection of plastic greenhouses. In recent years, remote sensing images have been widely used to extract plastic greenhouses. Some mainstream methods use new plastic greenhouse indices to distinguish greenhouses from other geographical objects. Some scholars have proposed a new spectral index based on Sentinel-2 images to distinguish land covers with similar spectral characteristics to plastic greenhouses and obtain greenhouse masks. Some scholars have also proposed a decision tree classifier for extracting greenhouses in Xinjiang, China based on the spectral characteristics of plastic greenhouses on Landsat TM images. However, there are spectral differences between different materials, and isolated noise in the spectrum will reduce the reliability of the classification results. Excessive reliance on spectral information has an adverse effect on the extraction of plastic greenhouses.

[0004] The development of deep learning technology has brought new ideas to greenhouse detection. Some scholars use the U-net model to perform semantic segmentation on remote sensing images and obtain the area of ​​plastic greenhouses. However, in the case of multi-span greenhouses, the boundaries of each greenhouse cannot be distinguished well. At the same time, some scholars established a target detection model based on Faster-RCNN to detect the study area, but only the total number of greenhouses was obtained. Although these methods can work well, the pixel-based method can only obtain the area of ​​the greenhouse, while the object-based method can only obtain the number of greenhouses. There are still unresolved problems in extracting greenhouses from remote sensing images, namely the problem of simultaneously extracting the number and area of ​​greenhouses and the problem of dense spatial distribution of greenhouses.

[0005] We observed that plastic greenhouses in China have regular rectangular features, and oriented frames can well represent regular rectangular greenhouses. At the same time, oriented frames avoid boundary intersection problems during detection and perform well in dense scenes. Therefore, based on remote sensing images and using deep learning technology, we provide a method to quickly detect the number and area of ​​greenhouses in one task. Summary of the invention

[0006] In view of the problems in the related art, the present invention proposes a rapid detection method for plastic greenhouses based on remote sensing images to overcome the above technical problems existing in the existing related art.

[0007] To this end, the specific technical solution adopted by the present invention is as follows: a rapid detection method for plastic greenhouses based on remote sensing images, the process of which includes the following steps:

[0008] A. Select the image of the sample area and perform sliding cutting to obtain several image tiles. Visually interpret and label each image tile to obtain the corresponding label file. The label file contains the following information: the coordinates of the center point of the plastic greenhouse (x, y), the length of the long side of the plastic greenhouse h, the length of the short side of the plastic greenhouse w, and the angle between the short side of the plastic greenhouse and the x-axis angle;

[0009] B. Divide the image tiles and corresponding label files into training sets and validation sets in proportion, and perform data augmentation on the training set;

[0010] C. Put the training set into the fast detection model based on deep learning for training, and use the validation set to evaluate the fast detection model to obtain the optimal network parameter model; in this step, select the YOLOX model as the fast detection model, select kld as the loss function to realize the rotation box detection; add the attention mechanism to the network backbone to optimize the extraction process; and add the angle detection branch to the detection head to decouple the curvature parameters of the candidate box;

[0011] Among them, kld is expressed as a loss function as follows:

[0012]

[0013] Among them, N p represents the prediction box, N t represents the sample frame, (μ p ,Σ p ) is the two-dimensional Gaussian distribution value of the prediction box rectangle coordinate transformation generated by the model, (μ t ,Σ t ) is the two-dimensional Gaussian distribution value converted from the five parameters (x, y, h, w, angle) of the plastic greenhouse label, Tr is the trace of the moment;

[0014] The attention distribution function formula is as follows:

[0015]

[0016] Among them, a i is the attention distribution value of the i-th feature value, q is the query vector, and h i is the i-th original input vector, s(h i ,q) is the i-th original input vector h i The hyperbolic tangent function value of the sum of the query vector q, exp is the exponential function with the natural constant e as the base, and n is the total number of eigenvalues;

[0017] Add an angle detection branch to the detection head as follows:

[0018] The feature map established in the YOLOX model is transformed into a 20*20*128 dimension through a 1*1 convolution kernel. Feature extraction is performed through two 3*3 convolution kernels, and each convolution passes through a silu activation function. Finally, the result is reduced to a 20*20*1 dimension through a 3*3 convolution kernel and a sigmod activation function to obtain the detection angle value.

[0019] D. Use the optimal network parameter model to detect the remote sensing images of the study area and obtain preliminary detection results;

[0020] E. Post-process the test results to eliminate the plastic greenhouses that are repeated on multiple slices and obtain the final test results of the study area.

[0021] The present invention extracts shallow and deep features in remote sensing images through a residual network, constructs a feature pyramid, and screens candidate boxes corresponding to the target box through the detection head and simOTA, thereby updating the model hyperparameters to achieve detection in other research areas. At the same time, after obtaining the detection results of the model, the overall NMS is used to eliminate the problem of repeated extraction of the same target caused by sliding cutting. This paper reduces the amount of network training parameters and improves the extraction speed by approximating rather than directly generating anchor boxes. In addition, this paper obtains information through a deep residual network, making the detection results of the model more convincing in complex scenes.

[0022] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0023] The present invention extracts plastic greenhouses based on remote sensing images, changing the traditional extraction method. The present invention overcomes the defect that the area and quantity cannot be extracted simultaneously in the previous plastic greenhouse extraction method, and can detect the number of greenhouses in the study area and the area and longitude and latitude of the center point of each plastic greenhouse. The present invention builds a model based on the deep learning pytorch framework, and selects kld as the loss function to achieve the change from the horizontal frame to the rotation frame. The network backbone adds an attention mechanism to optimize the extraction process and improve the accuracy of the feature pyramid establishment. An angle detection branch is added to the detection head to decouple the arc parameters of the candidate frame and generate a directional frame to wrap the plastic greenhouse. The kld loss and category score are used as evaluation indicators, which are more accurate and objective. The model detection results are processed by overall non-maximum suppression, which eliminates repeated detection of the same target caused by sliding cutting, and effectively improves the reliability of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a flow chart of the method of the present invention.

[0026] Figure 2 It is the geographical location, zoning and some greenhouse characteristics of the study area.

[0027] Figure 3 It is the residual network structure used in the present invention.

[0028] Figure 4 These are the labels of some of the training sets and test sets that were created.

[0029] Figure 5 It is the changing trend of the loss values ​​of the training set and the test set validation set during the deep learning network training process.

[0030] Figure 6 These are the detection results of plastic greenhouses in some scenes in the study area.

[0031] Figure 7 It is the spatial distribution map of the final detection results in the study area. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0033] This embodiment takes Zhoukou City, Henan Province as an example to explain in detail the process of rapid detection of plastic greenhouses using the present invention.

[0034] according to Figure 1 As shown in the method flow chart, the rapid detection method of plastic greenhouse based on remote sensing images in this embodiment includes the following steps:

[0035] Step A: Select Figure 2 The image of the area shown is cut by sliding to obtain several image tiles, and some image tiles are selected for visual interpretation and labeling of greenhouses to obtain the corresponding label file, which contains the following information: the coordinates of the center point of the plastic greenhouse (x, y), the length of the long side of the plastic greenhouse h, the length of the short side of the plastic greenhouse w, and the angle between the short side of the plastic greenhouse and the x-axis angle. Specifically, this step includes:

[0036] A1. Select areas with plastic greenhouses with different distribution characteristics and construction types as sample areas;

[0037] A2, slide and cut the image of the sample area into image tiles with a side length of 640 in steps of 320;

[0038] A3. Use visual interpretation to label the plastic greenhouse on each image tile, and extract the coordinates of the center point of the plastic greenhouse (x, y), the length of the long side of the plastic greenhouse h, the length of the short side of the plastic greenhouse w, and the angle between the short side of the plastic greenhouse and the x-axis. This information is recorded in VOC format and saved as an xml file. Figure 4 shown.

[0039] Step B: Divide the image tiles and the corresponding label files into training set and validation set in a ratio of 8.5:1.5, and perform data enhancement on the training set. Step B specifically includes:

[0040] B1, divide the image tiles and corresponding label files into training set and validation set in a ratio of 8.5:1.5;

[0041] B2. Use Mosaic and Mixup algorithms to perform data enhancement on the image tiles and corresponding label files that are classified into the training set;

[0042] B3. Perform flipping, color enhancement, and random scaling operations on the data-augmented training set.

[0043] Step C: Put the training set into the fast detection model based on deep learning for training, and use the validation set to evaluate the fast detection model to obtain the optimal network parameter model; in this step, select the YOLOX model as the fast detection model, according to Figure 3As shown in the figure, a residual network is built; kld is selected as the loss function to realize rotation box detection; an attention mechanism is added to the network backbone to optimize the extraction process; and an angle detection branch is added to the detection head to decouple the curvature parameters of the candidate box;

[0044] Among them, kld is expressed as a loss function as follows:

[0045]

[0046] Among them, N p represents the prediction box, N t represents the sample frame, (μ p ,Σ p ) is the two-dimensional Gaussian distribution value of the prediction box rectangle coordinate transformation generated by the model, (μ t ,Σ t ) is the two-dimensional Gaussian distribution value converted from the five parameters (x, y, h, w, angle) of the plastic greenhouse label, Tr is the trace of the moment;

[0047] The attention distribution function formula is as follows:

[0048]

[0049] Among them, a i is the attention distribution value of the i-th feature value, q is the query vector, and h i is the i-th original input vector, s(h i ,q) is the i-th original input vector h i The hyperbolic tangent function value of the sum of the query vector q, exp is the exponential function with the natural constant e as the base, and n is the total number of eigenvalues;

[0050] Add an angle detection branch to the detection head as follows:

[0051] The feature map established in the YOLOX model is transformed into a 20*20*128 dimension through a 1*1 convolution kernel. Feature extraction is performed through two 3*3 convolution kernels, and each convolution passes through the silu activation function. Finally, the result is reduced to a 20*20*1 dimension through a 3*3 convolution kernel and a sigmoid activation function to obtain the detection angle value.

[0052] Specifically, step C is performed according to the following steps:

[0053] C1. Build a rapid detection model;

[0054] C2. Put the training set into the fast detection model for iterative training and update the network parameters. A total of 140 generations of training are performed, of which the first 35 generations use frozen training and the data enhancement algorithm is turned off for the last 35 generations.

[0055] C3, using kld loss and category score as evaluation indicators, evaluate the model through the validation set, and select the best fast detection model on the training set and validation set as the optimal network parameter model for saving. The model loss value changes during training are as follows: Figure 5 As shown, the parameter model with the smallest loss value is selected as the optimal parameter model.

[0056] Step D: Use the optimal network parameter model to detect the remote sensing image of the study area and obtain preliminary detection results. Step D specifically includes:

[0057] D1. Download Figure 2 Images of all study areas are shown and saved in tiff format;

[0058] D2, slide and cut the image of the study area into image tiles with a side length of 640 with a step length of 320;

[0059] D3. All image tiles are put into the optimal network parameter model for detection to obtain the preliminary detection results of the plastic greenhouse. The detection results are saved with x, y, h, w, angle, cls parameters, which represent the horizontal coordinate of the center point of the plastic greenhouse, the vertical coordinate of the center point, the length of the long side, the length of the short side, the radian of the angle between the short side and the x-axis, and the confidence level.

[0060] Step E: Post-process the test results to eliminate the plastic greenhouses that are repeated on multiple slices and obtain the final test results of the study area. Step E specifically includes:

[0061] E1. Perform overall non-maximum suppression on the preliminary detection results to eliminate repeated detection caused by sliding cutting. In this step, a non-maximum suppression algorithm is written to perform overall non-maximum suppression on the preliminary detection results. Repeated detection caused by sliding cutting will be suppressed according to the confidence level to obtain accurate detection results. The preliminary detection results are as follows: Figure 6 shown.

[0062] E2. By installing the gadl function package in Python, the results after non-maximum suppression are merged to obtain the final plastic greenhouse detection results of the study area. The detection results are converted into shp files for output. The file database records the detailed information of each greenhouse. The shp file is visualized as follows Figure 7 shown.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A rapid detection method for plastic greenhouses based on remote sensing images, the process comprising the following steps: A. Select the image of the sample area and perform sliding cutting to obtain several image tiles. Visually interpret and label each image tile to obtain the corresponding label file. The label file contains the following information: the coordinates of the center point of the plastic greenhouse (x, y), the length of the long side of the plastic greenhouse h, the length of the short side of the plastic greenhouse w, and the angle between the short side of the plastic greenhouse and the x-axis angle; B. Divide the image tiles and corresponding label files into training sets and validation sets in proportion, and perform data augmentation on the training set; C. Put the training set into the fast detection model based on deep learning for training, and use the validation set to evaluate the fast detection model to obtain the optimal network parameter model; in this step, select the YOLOX model as the fast detection model, select kld as the loss function to realize the rotation box detection; add the attention mechanism to the network backbone to optimize the extraction process; and add the angle detection branch to the detection head to decouple the curvature parameters of the candidate box; Among them, kld is expressed as a loss function as follows: Among them, N p represents the prediction box, N t represents the sample frame, (μ p ,Σ p ) is the two-dimensional Gaussian distribution value of the prediction box rectangle coordinate transformation generated by the model, (μ t ,Σ t ) is the two-dimensional Gaussian distribution value converted from the five parameters (x, y, h, w, angle) of the plastic greenhouse label, Tr is the trace of the moment; The attention distribution function formula is as follows: Among them, a i is the attention distribution value of the i-th feature value, q is the query vector, and h i is the i-th original input vector, s(h i ,q) is the i-th original input vector h i The hyperbolic tangent function value of the sum of the query vector q, exp is the exponential function with the natural constant e as the base, and n is the total number of eigenvalues; Add an angle detection branch to the detection head as follows: The feature map established in the YOLOX model is transformed into a 20*20*128 dimension through a 1*1 convolution kernel. Feature extraction is performed through two 3*3 convolution kernels, and each convolution passes through a silu activation function. Finally, the result is reduced to a 20*20*1 dimension through a 3*3 convolution kernel and a sigmod activation function to obtain the detection angle value. The step C specifically comprises: C1. Build a rapid detection model; C2. Put the training set into the fast detection model for iterative training and update the network parameters. A total of 140 generations of training are performed, of which the first 35 generations use frozen training and the data enhancement algorithm is turned off for the last 35 generations. C3, using kld loss and category score as evaluation indicators, evaluate the model through the validation set, and select the best fast detection model on the training set and validation set as the optimal network parameter model for saving; D. Use the optimal network parameter model to detect the remote sensing images of the study area and obtain preliminary detection results; E. Post-process the test results to eliminate the plastic greenhouses that are repeated on multiple slices and obtain the final test results of the study area; The step E specifically comprises: E1. Perform overall non-maximum suppression on the preliminary detection results to eliminate repeated detection caused by sliding cutting; E2. Merge the results after non-maximum suppression to obtain the final plastic greenhouse detection results of the study area.

2. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 1, characterized in that: The step A specifically comprises: A1. Select areas with plastic greenhouses with different distribution characteristics and construction types as sample areas; A2, slide and cut the image of the sample area into image tiles with a side length of 640 in steps of 320; A3. Mark the plastic greenhouse on each image tile through visual interpretation, and extract the coordinates of the center point of the plastic greenhouse (x, y), the length of the long side of the plastic greenhouse h, the length of the short side of the plastic greenhouse w, and the angle between the short side of the plastic greenhouse and the x-axis angle.

3. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 1, characterized in that: The step B specifically comprises: B1, divide the image tiles and corresponding label files into training set and validation set; B2. Use Mosaic and Mixup algorithms to perform data enhancement on the image tiles and corresponding label files that are classified into the training set; B3. Perform flipping, color enhancement, and random scaling operations on the data-augmented training set.

4. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 1, characterized in that: The step D specifically comprises: D1. Download the area to be studied; D2, slide and cut the image of the study area into image tiles with a side length of 640 with a step length of 320; D3. All image tiles are put into the optimal network parameter model for detection to obtain preliminary detection results of the plastic greenhouse.

5. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 2, characterized in that: In step A3, the extracted data is saved as an xml file in VOC format.

6. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 3, characterized in that: In step B1, the ratio of training set to validation set is 8.5:1.

5.

7. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 2, characterized in that: In step B2, the Mosaic algorithm cuts the three image tiles and their corresponding labels into one picture, and the proportion of each image tile in the spliced ​​image tiles is random; the Mixup algorithm weightedly merges the three channel values ​​of the two image tiles, with weights of 0.4 and 0.6 respectively, and the labels corresponding to the two image tiles are directly merged.

8. A rapid detection method for plastic greenhouses based on remote sensing images as claimed in claim 2, characterized in that: In step C1, the target detection model is written in Python based on the pytorch framework; in step D1, the area to be studied is downloaded and saved in tiff format.

Images