A method for detecting invasive plant damage

Abstract

The present invention discloses a method for detecting invasive plant hazards, including image acquisition, image splicing, image sequential cropping, image super-resolution processing, neural network training, neural network import, image compression, sequential splicing, data analysis processing, and report generation steps. The present invention does not require manual intervention, and the image collected by the drone is processed by computer, which can reduce the labor intensity of people and perform accurate qualitative and quantitative analysis of invasive plants. The neural network used in the present invention includes an up-sampling layer up-conv, which expands the image size while extracting features; and includes three connection layers Concat1-Concat3, which splice the front feature image with the rear feature image to reduce the loss of effective features and improve the detection accuracy.

Description

Technical Field

[0001] The invention relates to a plant detection method, in particular to an invasive plant hazard detection method, and belongs to the technical field of plant protection. Background Art

[0002] With the continuous expansion of my country's import and export trade, invasive plants are introduced into China through mixing with imported grains, posing a threat to the balance of my country's ecosystem. The treatment of invasive plants needs to be guided by the invasive plant hazard level.

[0003] There are currently two methods for detecting the harm of alien invasive plants: the method of manually determining the number of plants per unit area and the method of subjectively judging the coverage. The first method is to randomly select several plots of 1m2 in a certain area, and then have researchers determine the number of plants. When calculating the number of plants per unit area, it requires a high labor intensity and is prone to subjective errors. The second method is to estimate the coverage area of ​​alien invasive plants within a certain range, and then roughly derive the coverage of invasive plants, which is very prone to human errors and difficult to quantify.

[0004] Both of the above methods have human errors and can only perform qualitative analysis but not quantitative analysis. The accuracy of the damage report and prescription report generated is worrying, and they cannot correctly guide the treatment of invasive alien plants. Therefore, a highly accurate method for invasive plant damage detection that can perform both quantitative and qualitative analysis is needed. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for detecting invasive plant damage.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for detecting invasive plant damage comprises the following steps:

[0008] Step 1: Image acquisition: The image is acquired by aerial photography with a drone camera; the drone flight altitude is set according to Table 1; the drone camera shooting angle is perpendicular to the ground, and the drone cruises and shoots at a fixed point according to the preset trajectory;

[0009] Step 2: Image stitching: stitch the images acquired by the drone into an aerial remote sensing image of the current area;

[0010] Step 3: Image sequential cropping: sequentially crop the aerial remote sensing images into images of preset sizes;

[0011] Step 4: Image super-resolution processing: The image cut in step 3 is subjected to super-resolution processing, and the resolution is increased to N times of the original image, where N>1;

[0012] Step 5: neural network training: the training set includes more than one set of training data; the training set is used to train network parameters; each set of training data consists of an original image and a binary image of the invasive plant canopy; the original image is an image obtained by aerial photography of a drone camera and is processed according to steps 1 to 4; the original image is input into the neural network, and the corresponding binary image of the invasive plant canopy is the desired output; the neural network includes convolution layers Conv1-Conv15, maximum pooling layers Max pool 1-Maxpool 3, connection layers Concat1-Concat4, up-sampling layers Up-conv 1, Up-conv 2 and Up-conv 3; the convolution layer Conv1, convolution layer Conv2, maximum pooling layer Max pool 1, convolution layer Conv3, convolution layer Conv4, maximum pooling layer Max pool 2, convolution layer Conv5, convolution layer Conv6, maximum pooling layer Max pool 3, convolution layer Conv7, convolution layer Conv8, up-sampling layer Up-conv 1. Connection layer Concat1, convolution layer Conv9, convolution layer Conv10, up-sampling layer Up-conv 2. Connection layer Concat2, convolution layer Conv11, convolution layer Conv12, up-sampling layer Up-conv 3. Connection layer Concat3, convolution layer Conv13, convolution layer Conv14, and convolution layer Conv15 are connected in sequence; the output end of the convolution layer Conv6 is connected to the input end of the connection layer Concat1; the output end of the convolution layer Conv4 is connected to the input end of the connection layer Concat2; the output end of the convolution layer Conv2 is connected to the input end of the connection layer Concat3;

[0013] Step 6: Importing the neural network: Import the image to be detected processed in step 4 into the neural network trained in step 5, and output the corresponding binary image of the invasive plant canopy;

[0014] Step 7: Pixel compression: compress the invasive plant canopy binary image output from step 6 to the image size before the image super-resolution processing in step 4;

[0015] Step 8: Sequential stitching: sequentially stitching the binary images of the invasive plant canopies compressed in step 7 to generate a binary image of the invasive plant canopies of the entire plot;

[0016] Step 9: Data analysis and processing: It consists of the following specific steps:

[0017] Step 9-1: Set the target object image pixels in the binary image of the invasive plant canopy of the entire plot to 1 and the background pixels to 0 to generate a one-dimensional matrix;

[0018] Step 9-2: Calculate the invasive plant coverage area S1 and its proportion e:

[0019]

[0020] Where S0 is the total area of ​​the region; S1 is the coverage area; e is the proportion; h is the cruising altitude of the drone; θ1 is the horizontal angle of view of the camera; θ2 is the vertical angle of view of the camera; w is the horizontal pixel of the camera; d is the vertical pixel of the camera; n0 is the total number of pixels in the region; n1 is the total number of target pixels;

[0021] The total number of target pixels n1 is the number of pixels with a value of 1 in the one-dimensional matrix, and the total number of regional pixels n0 is the total number of pixels in the one-dimensional matrix;

[0022] Step 10: Generate report: Generate a damaged area assessment report and a prescription report with geographic coordinates based on the GPS positioning of the drone.

[0023] Furthermore, the preset trajectory of the drone camera for aerial photography is a broken line, and the lateral interval b is:

[0024] b=h*tan(θ1 / 2)*(100%+100%-30%) (2)

[0025] Where h is the cruising altitude of the UAV, and θ1 is the lateral field of view angle of the UAV;

[0026] The fixed-point shooting interval s is:

[0027] s=h*tan(θ2 / 2)*(100%+100%-30%) (3)

[0028] Where h is the cruising altitude of the UAV, and θ2 is the longitudinal field of view angle of the UAV.

[0029] The beneficial effects of adopting the above technical solution are:

[0030] 1. The present invention does not require human intervention. The images collected by the drone are processed by computer, which can reduce the labor intensity of people and perform accurate qualitative and quantitative analysis of invasive plants.

[0031] 2. The neural network used in the present invention includes an up-sampling layer up-conv, which expands the image size while performing feature extraction and improves the detection accuracy.

[0032] 3. The neural network used in the present invention includes three connection layers Concat1-Concat3, which splice the front feature image with the back feature image to reduce the loss of effective features and improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the present invention;

[0034] Figure 2 is a structural diagram of a neural network according to Embodiment 1 of the present invention;

[0035] Figure 3 This is a schematic diagram of a preset trajectory shot by a drone camera according to Embodiment 1 of the present invention;

[0036] Figure 4 is the image to be detected in Example 1 of the present invention;

[0037] Figure 5 This is a binary image of the invasive plant canopy in Example 1 of the present invention. DETAILED DESCRIPTION

[0038] Embodiment 1:

[0039] A method for detecting invasive plant damage comprises the following steps:

[0040] Step 1: Image acquisition: The image is acquired by aerial photography with a drone camera; the drone flight altitude is set according to Table 1; the drone camera shooting angle is perpendicular to the ground, and the drone cruises and shoots at a fixed point according to the preset trajectory;

[0041] Step 2: Image stitching: stitch the images acquired by the drone into an aerial remote sensing image of the current area;

[0042] Step 3: Image sequential cropping: sequentially crop the aerial remote sensing images into images of preset sizes;

[0043] Step 4: Image super-resolution processing: The image cut in step 3 is subjected to super-resolution processing, and the resolution is increased to N times of the original image, where N>1;

[0044] Step 5: neural network training: the training set includes more than one set of training data; the training set is used to train network parameters; each set of training data consists of an original image and a binary image of the invasive plant canopy; the original image is an image obtained by aerial photography of a drone camera and is processed according to steps 1 to 4; the original image is input into the neural network, and the corresponding binary image of the invasive plant canopy is the desired output; the neural network includes convolution layers Conv1-Conv15, maximum pooling layers Max pool 1-Maxpool 3, connection layers Concat1-Concat4, up-sampling layers Up-conv 1, Up-conv 2 and Up-conv 3; the convolution layer Conv1, convolution layer Conv2, maximum pooling layer Max pool 1, convolution layer Conv3, convolution layer Conv4, maximum pooling layer Max pool 2, convolution layer Conv5, convolution layer Conv6, maximum pooling layer Max pool 3, convolution layer Conv7, convolution layer Conv8, up-sampling layer Up-conv 1. Connection layer Concat1, convolution layer Conv9, convolution layer Conv10, up-sampling layer Up-conv 2. Connection layer Concat2, convolution layer Conv11, convolution layer Conv12, up-sampling layer Up-conv 3. Connection layer Concat3, convolution layer Conv13, convolution layer Conv14, and convolution layer Conv15 are connected in sequence; the output end of the convolution layer Conv6 is connected to the input end of the connection layer Concat1; the output end of the convolution layer Conv4 is connected to the input end of the connection layer Concat2; the output end of the convolution layer Conv2 is connected to the input end of the connection layer Concat3;

[0045] Step 6: Importing the neural network: Import the image to be detected processed in step 4 into the neural network trained in step 5, and output the corresponding binary image of the invasive plant canopy;

[0046] Step 7: Pixel compression: compress the invasive plant canopy binary image output from step 6 to the image size before the image super-resolution processing in step 4;

[0047] Step 8: Sequential stitching: sequentially stitching the binary images of the invasive plant canopies compressed in step 7 to generate a binary image of the invasive plant canopies of the entire plot;

[0048] Step 9: Data analysis and processing: It consists of the following specific steps:

[0049] Step 9-1: Set the target object image pixels in the binary image of the invasive plant canopy of the entire plot to 1 and the background pixels to 0 to generate a one-dimensional matrix;

[0050] Step 9-2: Calculate the invasive plant coverage area S1 and its proportion e:

[0051]

[0052] Where S0 is the total area of ​​the region; S1 is the coverage area; e is the proportion; h is the cruising altitude of the drone; θ1 is the horizontal angle of view of the camera; θ2 is the vertical angle of view of the camera; w is the horizontal pixel of the camera; d is the vertical pixel of the camera; n0 is the total number of pixels in the region; n1 is the total number of target pixels;

[0053] The total number of target pixels n1 is the number of pixels with a value of 1 in the one-dimensional matrix, and the total number of regional pixels n0 is the total number of pixels in the one-dimensional matrix;

[0054] Step 10: Generate report: Generate a damaged area assessment report and a prescription report with geographic coordinates based on the GPS positioning of the drone.

[0055] Furthermore, the preset trajectory of the drone camera for aerial photography is a broken line, and the lateral interval b is:

[0056] b=h*tan(θ1 / 2)*(100%+100%-30%) (2)

[0057] Where h is the cruising altitude of the UAV, and θ1 is the lateral field of view angle of the UAV;

[0058] The fixed-point shooting interval s is:

[0059] s=h*tan(θ2 / 2)*(100%+100%-30%) (3)

[0060] Where h is the cruising altitude of the UAV, and θ2 is the longitudinal field of view angle of the UAV.

[0061] In the neural network in this embodiment, the convolution kernel of the convolution layer Conv1-Conv12 is 3*3, and the ReLu module is a 3*3 convolution block, which is used to extract image features and deepen or reduce the image depth; the maximum pooling layer Max pool 1-3 is a 2*2 maximum pooling layer, which is used to reduce the image size and filter out the feature quantity irrelevant to the result; the up-sampling layer up-conv 1-3 module is a deconvolution, which is used to expand the image size while performing feature extraction; the connection layer Concat1-Concat3 module is an image splicing module, which is used to splice the front feature image with the rear feature image to reduce the loss of effective features; the convolution layer Conv13-Conv15 is used to process the effective image into a single-channel image.

[0062] Table 1

[0063]

Claims

1. A method for detecting invasive plant damage, characterized in that: The following steps are involved: Step 1: Image acquisition: The image is acquired by aerial photography using a drone camera. The flight altitude of the drone is set as follows: When the height of the invasive plant is less than 10cm, the invasive plant is a small plant, and the UAV's flight altitude is 5m; when the height of the invasive plant is greater than or equal to 10cm and less than 50cm, the invasive plant is a medium-sized plant, and the UAV's flight altitude is 10m; when the height of the invasive plant is greater than or equal to 50cm and less than 200cm, the invasive plant is a medium-to-large plant, and the UAV's flight altitude is 15m; when the height of the invasive plant is greater than or equal to 200cm, the invasive plant is a large plant, and the UAV's flight altitude is 20m; The drone camera takes pictures at a vertical angle to the ground, and it cruises and shoots at a fixed point according to a preset trajectory; Step 2: Image stitching: stitch the images acquired by the drone into an aerial remote sensing image of the current area; Step 3: Image sequential cropping: sequentially crop the aerial remote sensing images into images of preset sizes; Step 4: Image super-resolution processing: The image cut in step 3 is subjected to super-resolution processing, and the resolution is increased to N times of the original image, where N>1; Step 5: neural network training: the training set includes more than one set of training data; the training set is used to train network parameters; each set of training data consists of an original image and a binary image of the invasive plant canopy; the original image is an image obtained by aerial photography of a drone camera and is processed according to steps 1 to 4; the original image is input into the neural network, and the corresponding binary image of the invasive plant canopy is the desired output; the neural network includes convolution layers Conv1-Convolution layers Conv15, maximum pooling layers Max pool 1-Max pool3, connection layers Concat1-Concat4, up-sampling layers Up-conv 1, Up-conv 2 and Up-conv 3; the convolution layer Conv1, convolution layer Conv2, maximum pooling layer Max pool 1, convolution layer Conv3, convolution layer Conv4, maximum pooling layer Maxpool 2, convolution layer Conv5, convolution layer Conv6, maximum pooling layer Max pool 3, convolution layer Conv7, convolution layer Conv8, up-sampling layer Up-conv 1. Connection layer Concat1, convolution layer Conv9, convolution layer Conv10, up-sampling layer Up-conv 2. Connection layer Concat2, convolution layer Conv11, convolution layer Conv12, up-sampling layer Up-conv 3. Connection layer Concat3, convolution layer Conv13, convolution layer Conv14, and convolution layer Conv15 are connected in sequence; the output end of the convolution layer Conv6 is connected to the input end of the connection layer Concat1; the output end of the convolution layer Conv4 is connected to the input end of the connection layer Concat2; the output end of the convolution layer Conv2 is connected to the input end of the connection layer Concat3; Step 6: Importing the neural network: importing the image to be detected processed in step 4 into the neural network trained in step 5, and outputting the corresponding binary image of the invasive plant canopy; Step 7: Pixel compression: compress the invasive plant canopy binary image output from step 6 to the image size before the image super-resolution processing in step 4; Step 8: Sequential stitching: sequentially stitching the binary images of the invasive plant canopies compressed in step 7 to generate a binary image of the invasive plant canopies of the entire plot; Step 9: Data analysis and processing: It consists of the following specific steps: Step 9-1: Set the target object image pixels in the binary image of the invasive plant canopy of the entire plot to 1 and the background pixels to 0 to generate a one-dimensional matrix; Step 9-2: Calculate the invasive plant coverage area S1 and its proportion e: Where S0 is the total area of ​​the region; S1 is the coverage area; e is the proportion; h is the cruising altitude of the drone; θ1 is the horizontal angle of view of the camera; θ2 is the vertical angle of view of the camera; w is the horizontal pixel of the camera; d is the vertical pixel of the camera; n0 is the total number of pixels in the region; n1 is the total number of target pixels; The total number of target pixels n1 is the number of pixels with a value of 1 in the one-dimensional matrix, and the total number of regional pixels n0 is the total number of pixels in the one-dimensional matrix; Step 10: Generate report: Generate a damaged area assessment report and a prescription report with geographic coordinates based on the GPS positioning of the drone.

2. The method for detecting invasive plant damage according to claim 1, characterized in that: The preset trajectory of the drone camera aerial photography in step 1 is a broken line, and the lateral interval b is: b=h*tan(θ1 / 2)*(100%+100%-30%) (2) Where h is the cruising altitude of the UAV, and θ1 is the lateral field of view angle of the UAV; The fixed-point shooting interval s is: s=h*tan(θ2 / 2)*(100%+100%-30%) (3) Where h is the cruising altitude of the UAV, and θ2 is the longitudinal field of view angle of the UAV.

Images