An intelligent nested recognition method for plant scene targets based on UAV remote sensing

Abstract

The present invention discloses an intelligent nested recognition method for plant scene targets based on UAV remote sensing. The method is as follows: S1. In the UAV remote sensing image, the scene target is composed of plant sample targets and similar plants within a certain range around them, as a whole piece of land, and a rectangular box is used as a marker to mark, and a scene target detection sample set CJSet is made. Within the scene range, a plant semantic segmentation sample set YYSet is made based on actual plant vector data and UAV data in the scene; S2. Plant targets are usually few, and it is necessary to perform geometric space and color space enhancement processing on the scene target sample set CJSet and the semantic segmentation sample set YYSet, and expand the sample set scale. The predetermined scene range where the plant is located is automatically determined for the UAV remote sensing data with complex backgrounds, and then the plant targets in the UAV image located in the scene are detected to obtain a higher target recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of plant target detection in UAV remote sensing images, and specifically relates to an intelligent nesting-intelligent identification method based on UAV remote sensing plant scene targets. The method has strong adaptability and robustness to various interference factors in target identification in UAV remote sensing data in key areas across the country. The method is specifically divided into two steps: first, identifying the scene range where the plants are located, and then secondary identifying the plant targets based on the scene range, forming an intelligent identification nesting strategy, and then obtaining higher target identification accuracy. Background Art

[0002] It is of great significance to quickly and accurately obtain the geographic spatial information of plants and monitor them in the first place. Remote sensing data has the advantages of wide coverage and strong timeliness, especially the higher resolution and more flexible data acquisition of drone remote sensing images, which can play an important role in plant growth, discovery and monitoring. Using remote sensing images to identify plants, especially remote sensing detection of plants, has become a research hotspot in recent years.

[0003] Using remote sensing images to identify plant targets belongs to the field of remote sensing target detection algorithms. After the traditional manually designed target feature pattern recognition algorithm, the target detection algorithm based on deep learning has rapidly emerged and has been successfully applied to the field of remote sensing target detection. Deep learning target detection algorithms are usually divided into one-stage and two-stage detection algorithms based on whether candidate boxes are generated. The most famous one-stage typical models are SSD and YOLO series algorithms, and the two-stage typical models are R-CNN series algorithms (including SPP-net, Fast R-CNN, Faster R-CNN). "Single-stage" usually has a higher detection speed, while "two-stage" has a higher detection accuracy. Because deep neural networks have excellent feature learning capabilities and do not rely on manual subjective design, they have greatly improved the accuracy of remote sensing target detection.

[0004] Since some plants are often planted in very hidden areas and often interplanted with other similar plants, their features are not obvious in remote sensing images. Although drone images have high image resolution, the interplanted plants and other crop image features are similar during the growth stage. The feature targets are affected by large redundant and interfering information, and the same spectrum and different objects and the same object and different spectrum phenomena are more obvious, which increases the difficulty of plant extraction in remote sensing images over a large range. Currently, the commonly used plant target detection algorithms are all performed in small areas, and only one deep learning target detection framework is used. Even in 1.5 cm drone images, the Faster R-CNN algorithm with the best two-stage accuracy can only achieve a recognition accuracy of about 30%. Therefore, for the target detection of plants in drone remote sensing images over a large range, a more effective detection algorithm is needed. Summary of the invention

[0005] In order to overcome the above technical problems, the purpose of the present invention is to provide an intelligent nested recognition method for plant scene targets based on UAV remote sensing. The present invention targets UAV remote sensing images of a larger range. First, the UAV images containing plant target scene information are used to make larger samples to implement Faster R-CNN model training, and preliminarily detect the scene range containing plants; secondly, a deep semantic segmentation model is trained using smaller samples within the scene range to obtain specific semantic information of plants; finally, all plant information within the scene range is geographically stitched to complete the extraction of all plant information in the entire image. The overall algorithm process solves the problems of low accuracy and slow speed in UAV remote sensing plant target detection by nesting deep target detection model training and in-scene deduction.

[0006] The present invention provides an intelligent nesting recognition method for plant scene targets based on UAV remote sensing, comprising the following steps:

[0007] S1. In the UAV remote sensing image, the scene target is composed of plant sample targets and similar plants within a certain range around them, which can usually be a whole piece of land, and a rectangular box is used as a marker to mark it, and a scene target detection sample set CJSet is produced. At the same time, within each scene range, a plant semantic segmentation sample set YYSet is produced based on the actual plant vector data and the UAV data in the scene;

[0008] S2. Since there are usually fewer plant targets, it is necessary to perform random geometric transformations including rotation, symmetry transformation, affine transformation, and color space enhancement processing including blur, color, brightness, contrast, sharpening, and adding noise on the scene target sample set CJSet and the semantic segmentation sample set YYSet to expand the sample set size;

[0009] S3. Use the enhanced scene target detection sample set CJSet to train the plant scene range target detection model M based on Faster R-CNN CJ And deduce to obtain the plant scene target result set CJOBJSet;

[0010] S4, spatially vectorizing the CJOBJSet scene result set obtained in S3 to form multiple rectangular scene frame data sets CJOBJRectSet with spatial geographic information;

[0011] S5. Use the enhanced semantic segmentation sample set YYSet to train the plant semantic segmentation model M based on DLinkNet YY, and based on the CJOBJRectSet obtained in S4, traverse and deduce the unmanned remote sensing images corresponding to each scene frame one by one, and then obtain the semantic information YYOBJSet of the plant target in each scene range i , where i∈(1,Num CJOBJSet ), Num CJOBJSet The number of CJOBJSets.

[0012] S6. Traverse all YYOBJSets within each scene i The plant semantic target in the , calculate the intersection and union ratio η between the plant semantic polygons IOU , if η IOU >∈, then delete one of them, otherwise keep both, where ∈ is the judgment threshold, which is usually set to 0.5, until all scenes in an image are traversed, and finally the plant vector result set YSOBJSet consistent with the remote sensing image coordinate system is obtained.

[0013] Preferably, the drone images are all in true color band combination mode.

[0014] Preferably, the size of the scene rectangular frame in the scene sample set in step S2 is usually based on the range of objects with uniform image features, the sample image size is 2000pi*2000pi, and the remote sensing image size in the plant semantic segmentation sample set is 256pi*256pi.

[0015] Preferably, the sample enhancement algorithm in step S2 is divided into two aspects, respectively for the geometric space and the color space, wherein the rotation transformation principle in the geometric space is to randomly rotate counterclockwise and rotate the sample to retain the original size at intervals of 15 degrees, then 360 degrees is divided into 24 parts, and each enhancement will randomly select one of the angles for enhancement processing, and the calculation formula is as follows:

[0016]

[0017] Among them, A is the sample image matrix to be processed, and α is the rotation angle.

[0018] Preferably, the backbone feature extraction network of the Faster R-CNN used in step S3 is ResNet101. Since the scene range of plants is large and the sizes are different, the candidate boxes generated in the RPN stage are 5 scales, namely [64, 128, 256, 512, 1024], and the unit is pixel value.

[0019] Preferably, in step S4, spatial vectorization is performed on the target detection result set CJOBJSet obtained in S3 to obtain a series of vector rectangles with geographic coordinate information, and the file type is saved in ESRI Shapefile (.shp) file format.

[0020] Preferably, the backbone feature extraction network of DLinkNet used in step S5 is ResNet34. When the drone image is deduced in each scene rectangular frame, it is traversed in a size range of 256pi*256pi. To ensure the integrity of the target, it moves in a step size of 128 to complete the traversal and deduction of the complete image range in the scene frame.

[0021] Preferably, the polygon intersection ratio η in step S6 is IOU The calculation formula is as follows

[0022]

[0023] Among them, P A-area and P B-area They are the areas of two semantic polygons of plants respectively. The main difference from the conventional IOU is that A and B are not rectangles but general polygons.

[0024] The beneficial effects of the present invention are: 1. The present invention adopts a deep nested double-layer network of target detection and semantic segmentation for automatic plant recognition, which has higher plant recognition accuracy than the conventional single-model detection network; 2. The present invention first uses the target detection framework to identify the predetermined scene range where the plant is located, and then extracts specific information in the scene. The model deduction only needs to be performed in the scene, thereby speeding up the plant extraction speed; 3. The present invention adopts a variety of image enhancement algorithms, expands the number of small plant samples, and adds multiple types of interference factors, so it has strong robustness in dealing with various complex scenes in drone remote sensing data; 4. The present invention can directly obtain target information with geographic coding, realize rapid positioning of plants and better assist field surveys. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a method for intelligent plant identification based on a predetermined scene of a drone in an embodiment of the present invention.

[0026] Figure 2 are the training samples used in the embodiment of the present invention, where A is the plant scene training sample and B is the semantic segmentation sample.

[0027] Figure 3 This is the training sample result of rotation enhancement in geometric enhancement in an embodiment of the present invention.

[0028] Figure 4Schematic diagram of the algorithm for polygon intersection and union ratio in an embodiment of the present invention.

[0029] Figure 5 It is a partial result of the model deduction in the embodiment of the present invention, wherein the red box represents the recognition of the predetermined scene information and the red polygon information represents the recognition of the plants in the scene. DETAILED DESCRIPTION

[0030] In order to better illustrate the technical solution and advantages of the present invention, plant target extraction is performed using drone image data with a spatial resolution better than 2 cm. It should be understood that the specific implementation described herein is only used to explain the present invention and is not used to limit the present invention.

[0031] like Figure 1 As shown, this embodiment provides an intelligent nested recognition method for plant scene targets based on UAV remote sensing, and the plant target recognition method includes the following steps:

[0032] S1, select a 2 cm resolution true color UAV remote sensing image I containing plant targets, determine the scene target rectangular frame according to the scene environment around the plant, and create a scene target detection sample set GJSet. The sample sample is as follows: Figure 2 As shown in A, the basic scene information includes "yard", "vegetable field", "crop field", etc. The sample size is 2000pi*2000pi, and the number of samples is 1256. At the same time, the real plant vector data is combined with the drone image in the scene to produce the semantic segmentation sample set YYSet. The sample samples are as follows: Figure 2 As shown in B, the sample size is 256pi*256pi and the number of samples is 10686.

[0033] S2. Perform sample enhancement operations on CJSet and YYSet in step S1, including rotation, symmetry transformation, affine transformation, blur, color transformation, brightness, contrast enhancement, sharpening, and noise addition for geometric space enhancement and color space enhancement. The sample sizes of both are expanded to 10 times the original ones. The number of samples of CJSet is increased to 12560, and the number of samples of YYSet is 106860. Figure 3 The rotation enhancement of a sample in CJSet and YYSet.

[0034] S3, using S2 to obtain the enhanced scene target detection sample set CJSet to train the plant target detection model M based on Faster R-CNN CJ And execute the deduction process to obtain the plant scene target result set CJOBJSet. In this example, 8 plant scene ranges are obtained in one drone image;

[0035] S4, spatially vectorizing the CJOBJSet scene result set obtained in S3 to form multiple rectangular scene frame data sets CJOBJRectSet with spatial geographic information. At this time, the scene result storage format is ESRI Shapefile (.shp) file format and is saved on the disk;

[0036] S5. Use the enhanced semantic segmentation sample set YYSet to train the plant semantic segmentation model M based on DLinkNet YY , and based on the CJOBJRectSet obtained in S4, since there are 8 elements (i.e., 8 scene rectangles), the drone image data in the 8 scene ranges are traversed one by one and plant semantic deduction is performed to obtain the plant target semantic information set YYOBJSet in each scene. i , we can see that j∈(1,8).

[0037] S6. Since there are overlapping areas in the pixel-by-pixel block traversal mode, in order to remove duplicate plant semantics, it is necessary to calculate the intersection and union ratio η of each pair of semantic target polygons within each plant scene. IOU (Specific principles such as Figure 4 As shown), according to the principle: if η IOU >0.5, one of them is deleted, otherwise all are retained until all scenes in the entire scene are traversed, and finally the plant vector result set YSOBJSet consistent with the remote sensing image coordinate system is obtained, and 13 locations of all plants in the 8 scenes are obtained. Figure 5 This is the plant identification in two of the scenes. A total of 2 plant targets were found. Since this method is based on pre-determined scene identification and then inferring target information, it is more efficient and faster than directly using inference in the entire scene data. The time statistics are as follows:

[0038]

[0039] By adopting the above technical solution disclosed in the present invention, the following beneficial effects are obtained:

[0040] The present invention provides an intelligent nested recognition method for plant scene targets based on unmanned aerial vehicle remote sensing. The method firstly recognizes the scene targets where plants are located in complex unmanned aerial vehicle images, and then automatically recognizes the plant information in the scene, which not only improves the plant recognition speed, but also improves the recognition accuracy. The present invention proposes a double-layer nested recognition network of deep target detection and semantic segmentation, which improves the accuracy of the plant target algorithm and the robustness against complex background interference. In view of the disadvantage of small number of plant target samples, the present invention proposes a strategy of using multiple enhancement algorithms for effective expansion, which effectively avoids the problem of model overfitting.

[0041] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. An intelligent nested recognition method for plant scene targets based on UAV remote sensing, characterized by: The method comprises the following steps, S1. In the UAV remote sensing image, the scene target is composed of plant sample targets and similar plants within a certain range around them, which are regarded as a whole piece of land. The scene targets containing plants are marked with rectangular boxes to produce a scene target detection sample set CJSet. At the same time, within each scene range, a plant semantic segmentation sample set YYSet is produced based on the actual plant vector data and the UAV data in the scene. S2, perform geometric space and color space enhancement processing on the scene target sample set CJSet and the semantic segmentation sample set YYSet to expand the sample set size; S3. Use the enhanced scene target detection sample set CJSet to train the plant scene range target detection model M based on FasterR-CNN CJ And deduce to obtain the plant scene target result set CJOBJSet; S4, spatially vectorizing the CJOBJSet scene result set obtained in S3 to form multiple rectangular scene frame data sets CJOBJRectSet with spatial geographic information; S5. Use the enhanced semantic segmentation sample set YYSet to train the plant semantic segmentation model M based on DLinkNet YY , and based on the CJOBJRectSet obtained in S4, traverse and deduce the unmanned remote sensing images corresponding to each scene frame one by one, and then obtain the semantic information YYOBJSet of the plant target in each scene range i , where i∈(1,Num CJOBJSet ), Num CJOBJSet The number of scene targets in CJOBJSet; S6. Traverse all YYOBJSets within each scene i The plant semantic target in the , calculate the intersection and union ratio η between the plant semantic polygons IOU , if η IOU >∈, then delete one of them, otherwise keep both, where ∈ is the judgment threshold, until all scenes in a scene are traversed, and finally obtain the plant vector result set YSOBJSet consistent with the remote sensing image coordinate system.

2. The method for intelligent nested recognition of plant scene targets by UAV remote sensing according to claim 1 is characterized by: The UAV remote sensing images are all true color band combination modes.

3. The intelligent nested recognition method for UAV remote sensing plant scene targets according to claim 2 is characterized by: The size of the scene rectangle frame in the scene sample set is based on the range of objects with uniform image features. The size of the remote sensing image is 2000pi*2000pi, and the size of the remote sensing image in the plant semantic segmentation sample set is 256pi*256pi.

4. The intelligent nested recognition method for UAV remote sensing plant scene targets according to claim 2 is characterized by: The sample enhancement algorithm is divided into two aspects, namely, the geometric space and the color space. The principle of rotation transformation in the geometric space is to randomly rotate counterclockwise and rotate the sample at intervals of 15 degrees to retain the original size. Then, 360 degrees is divided into 24 parts, and each enhancement will randomly select one of the angles for enhancement. The calculation formula is as follows: Among them, A is the sample image matrix to be processed, and α is the rotation angle.

5. The intelligent nested recognition method for UAV remote sensing plant scene targets according to claim 3 is characterized by: The scene object detection result set is a vector rectangle with geographic coordinate information, and the file type is saved in the ESRI Shapefile file format.

6. The method for intelligent nested recognition of plant scene targets by UAV remote sensing according to claim 3 is characterized by: The backbone feature extraction network of the Faster R-CNN used in step S3 is ResNet101. Since the scene range of plants is large and the sizes are different, the candidate boxes generated in the RPN stage are 5 scales, namely [64, 128, 256, 512, 1024], and the unit is pixel value.

7. The intelligent nested recognition method for UAV remote sensing plant scene targets according to claim 4 is characterized by: When the drone image is deduced in each scene rectangular frame, it is traversed in a range of 256pi*256. To ensure the integrity of the target, it moves in a step length of 128 to complete the traversal and deduction of the complete image range in the scene frame.

8. The method for intelligent nested recognition of plant scene targets by UAV remote sensing according to claim 5 is characterized by: Polygon Intersection Ratio η IOU The calculation formula is as follows: Among them, P A-area and P B-area Two semantic polygons for plants respectively.

9. The method for intelligent nested recognition of plant scene targets by UAV remote sensing according to claim 6 is characterized by: After the spatial vectorization of the plant target result in step S6, the coordinate system is consistent with the drone image, and the specific real geographical location information is saved in the ESRI Shapefile file format.

Images