A method, medium and system for identifying poppy using remote sensing by unmanned aerial vehicle

Abstract

The invention provides a method, medium and system for identifying poppies by remote sensing by unmanned aerial vehicles, belonging to the technical field of poppy identification by remote sensing by unmanned aerial vehicles, and comprising: collecting multispectral remote sensing images, including visible light, near infrared and thermal infrared images. Using a pre-trained leaf recognition model to identify leaf areas in visible light images, and marking them on near infrared and thermal infrared images. Using a pre-trained dew recognition model to mark dew areas in leaf areas of near infrared and thermal infrared images. Extracting features of near infrared and thermal infrared dew areas as training input. Using poppy positions as training outputs, training poppy recognition models. Extracting near infrared and thermal infrared dew features from farmland images to be tested, inputting them into a poppy recognition model, and obtaining poppy position coordinates. The invention solves the technical problem that when dew exists on the surface of plant leaves, the spectral features of plant leaves in the visible light band will be affected, resulting in a reduction in the ability of existing machine learning methods to judge poppies.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle remote sensing poppy identification, and in particular, relates to an unmanned aerial vehicle remote sensing poppy identification method, medium and system. Background Art

[0002] Opium poppy is an important and widely planted medicinal plant. Its flowers and seeds are rich in opium alkaloids and are the main source of raw materials for making drugs such as morphine and heroin. In order to control the illegal cultivation and abuse of medicinal poppy, many countries have formulated strict planting licenses and regulatory policies. However, since poppy is usually grown in remote rural areas, it is difficult to fully cover it by manual inspections and monitoring methods, and it is easy to miss some small-scale illegal planting sites. Therefore, the development of rapid identification and automatic monitoring methods for poppy based on remote sensing technology is of great significance to improving the control capabilities of medicinal poppy cultivation.

[0003] In existing research, many scholars have tried to use remote sensing technology to detect and identify poppies in crops. For example, using high-resolution satellite multispectral remote sensing images, according to the spectral characteristics of poppy leaves, a method based on maximum likelihood method is used to classify and identify poppies in farmland. Or use drone low-altitude remote sensing, combined with machine learning algorithms to achieve automatic detection of poppies. Since poppy identification is mostly about finding very few poppy plants in a large range of farmland, it will be difficult to obtain features by simply analyzing the color and shape characteristics of plants due to the occlusion between leaves of different types of plants. If the detailed features of the leaf surface can be collected, such as the texture, villi, stomata and other detailed features of the leaf surface, the recognition accuracy of poppies will be greatly improved. However, since it is difficult to capture such detailed features in remote sensing images, such detailed features of the leaf surface are often not considered in the field of poppy identification. These methods have achieved certain recognition effects, but there are also some limitations: when there are dewdrops on the surface of plant leaves, it will affect the spectral characteristics of plant leaves in the visible light band, resulting in a reduction in the judgment ability of existing machine learning methods for poppies. Summary of the invention

[0004] In view of this, the present invention provides a method, medium and system for identifying poppies by remote sensing by unmanned aerial vehicles, which can solve the technical problem that when dewdrops exist on the surface of plant leaves, the spectral characteristics of plant leaves in the visible light band will be affected, resulting in a decrease in the ability of existing machine learning methods to judge poppies.

[0005] The present invention is achieved in that:

[0006] A first aspect of the present invention provides a method for identifying poppies using remote sensing by an unmanned aerial vehicle, comprising the following steps:

[0007] S10, collecting multispectral remote sensing images of multiple located farmland areas sparsely planted with poppies in dewy weather;

[0008] S20, using a pre-trained leaf recognition model based on color recognition to identify a leaf area in the visible light image;

[0009] S30, marking leaf areas on the near infrared image and the thermal infrared image by aligning them with the visible light image;

[0010] S40, marking the dew area in the leaf area of ​​the near infrared image and the thermal infrared image using a pre-trained dew recognition model;

[0011] S50, extracting features of the dew drop areas in the near infrared image and the thermal infrared image to obtain near infrared dew drop features and thermal infrared dew drop features, respectively;

[0012] S60, using the near-infrared dewdrop features and the thermal infrared dewdrop features as training inputs and the marked poppy positions as training outputs, training a machine learning model to obtain a poppy recognition model;

[0013] S70, finally obtaining the image of the farmland to be tested of the poppy to be identified collected by the drone in dewy weather, and obtaining the near-infrared dewdrop features and thermal-infrared dewdrop features of the farmland image to be tested, recording them as the dewdrop features to be tested, inputting them into the poppy identification model, obtaining the coordinates of the poppy identified in the farmland image to be tested and outputting them.

[0014] The multispectral remote sensing image includes image data of visible light image, near infrared image and thermal infrared image.

[0015] The step S10 specifically includes:

[0016] Step 1: In the located farmland area with sparse poppy planting, use drone hovering photography to obtain multispectral remote sensing image data such as visible light images, near infrared images and thermal infrared images;

[0017] Step 2: geometrically correct and radiometrically correct the multispectral remote sensing image data to ensure that the spatial positions of the three spectral images are completely aligned;

[0018] Step 3: When collecting images, the geographical location information of the farmland area is recorded to establish the corresponding relationship between the multispectral remote sensing image and the actual farmland location;

[0019] Step 4: Store the corrected multispectral remote sensing image data in the database to provide data support for subsequent leaf identification and dew drop identification.

[0020] The farmland area in step S10 is a farmland sparsely planted with poppies, and the spectral characteristics are obviously different; the image acquisition in step S10 is performed in dewy weather to enhance the spectral characteristics of poppy leaves and surrounding vegetation.

[0021] Optionally, the flight speed of the drone hovering photography to collect images in step S10 is controlled between 5-10 m / s to ensure image clarity.

[0022] Wherein, the pre-trained leaf recognition model based on color recognition is a convolutional neural network model, and its structure includes:

[0023] Input layer: accepts visible light image data;

[0024] Convolutional layer: extracts the color and texture features of the image;

[0025] Pooling layer: reduces feature dimension and improves robustness;

[0026] Fully connected layer: comprehensive features, output segmentation results of leaf area;

[0027] Output layer: gives the probability of whether each pixel belongs to the leaf area;

[0028] Through supervised training with a large number of labeled leaf and non-leaf samples, a model that can accurately identify leaf areas is finally formed.

[0029] The specific steps of step S20 include:

[0030] Step 1: retrieve a visible light image from a database as input data;

[0031] Step 2: Use a pre-trained convolutional neural network model based on color recognition to perform semantic segmentation on the visible light image and identify the leaf area in the image;

[0032] Step 3: extracting visual features such as texture and shape of the leaf area as output results of leaf recognition;

[0033] Step 4: The leaf recognition result and the corresponding blank. The specific steps of step S30 include:

[0034] Step 1, retrieve the blade area coordinate information obtained in step S20 from the database;

[0035] Step 2: Map the leaf area coordinate information in step S20 onto the near infrared image and the thermal infrared image according to the correspondence between the multispectral remote sensing image recorded in step S10 and the actual farmland position;

[0036] Step 3: Mark the corresponding leaf areas on the near infrared image and thermal infrared image.

[0037] The leaf recognition model adopts a convolutional neural network model based on color recognition.

[0038] The image alignment method in step S30 adopts a geometric transformation registration algorithm based on feature points.

[0039] Furthermore, the pre-trained dew drop recognition model based on image segmentation is a convolutional neural network model, and its structure includes:

[0040] Input layer: accepts near infrared image and thermal infrared image data;

[0041] Convolutional layer: extracts the spectral and temperature features of the leaf area in the image;

[0042] Pooling layer: reduces feature dimension and improves robustness;

[0043] Fully connected layer: integrates spectral temperature features and outputs the segmentation result of the dew drop area;

[0044] Output layer: gives the probability of whether each pixel belongs to the dew area;

[0045] Through supervised training with a large number of labeled dew and non-dew samples, a model that can accurately identify the position of dew drops on the leaf surface is finally formed.

[0046] The specific steps of step S40 include:

[0047] Step 1, retrieve the leaf area of ​​the near infrared image and thermal infrared image marked in step S30 from the database;

[0048] Step 2: using a pre-trained convolutional neural network model based on image segmentation to perform semantic segmentation on the leaf area and identify the dew area on the leaf surface;

[0049] Step 3, extracting the spectrum, temperature and other characteristics of the dew drop area as the output result of dew drop recognition;

[0050] Step 4: storing the dew drop recognition result and the corresponding spatial position coordinate information in a database.

[0051] Among them, the near-infrared dew drop characteristics and thermal infrared dew drop characteristics are spectral and temperature characteristics such as average reflectivity, variance, texture characteristics, etc. of the dew drop area in the near-infrared or thermal-infrared.

[0052] The specific steps of step S50 include:

[0053] Step 1, retrieving the dew area coordinate information of the near infrared image and the thermal infrared image obtained in step S40 from the database;

[0054] Step 2: according to the coordinates of the dew drop area, extract the spectrum and temperature characteristics such as the average reflectivity, variance, and texture characteristics of the dew drop area in the near infrared image and the thermal infrared image respectively;

[0055] Step 3: Store the near-infrared dew drop features and thermal infrared dew drop features in a database as input data for subsequent machine learning model training.

[0056] The poppy recognition model uses a support vector machine as a basic model.

[0057] The specific steps of step S60 include:

[0058] Step 1: retrieve the near infrared dew drop features and thermal infrared dew drop features extracted in step S50 from the database as input data for the machine learning model;

[0059] Step 2: retrieve the annotation information of poppy samples at known locations from the database as the output label of the machine learning model;

[0060] Step 3, the poppy recognition model adopts a support vector machine as a basic model, specifically adopts a support vector machine algorithm, takes near infrared dew drop features and thermal infrared dew drop features as input, and the poppy position is marked as output, and the poppy recognition model is trained;

[0061] Optionally, in step 4, the model parameters are optimized by cross-validation to ensure that the recognition accuracy of the model on the test set reaches more than 95%.

[0062] The specific steps of step S70 include:

[0063] Step 1, collecting multispectral remote sensing image data of the farmland to be identified;

[0064] Step 2: According to the method of step S30 and step S40, the leaf area and the dew area are marked on the near infrared image and the thermal infrared image of the farmland to be identified;

[0065] Step 3: extract the feature extraction method in step S50 to obtain the near infrared and thermal infrared features of the dew drops on the leaves in the farmland image to be identified;

[0066] Step 4, inputting the dew feature of the farmland image to be identified into the poppy recognition model trained in step S60 for prediction;

[0067] Step 5: Output the spatial coordinate location information of the poppy plants in the farmland image predicted by the model.

[0068] In step S70, if the model detects multiple suspicious poppy locations, the coordinate information of the top three locations with the highest probability is output for reference.

[0069] The dewdrop recognition model also takes into account the leaf temperature information to further improve the recognition accuracy of the dewdrop area.

[0070] The near-infrared dew drop features extracted in step S50 also include shape parameters of the reflection spectrum curve, and the thermal infrared dew drop features also include dynamic features of temperature change rate.

[0071] Wherein, the multispectral remote sensing image also includes mid-infrared image to further enhance the recognition capability of poppy.

[0072] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the above-mentioned UAV remote sensing poppy identification method.

[0073] A third aspect of the present invention provides an unmanned aerial vehicle remote sensing poppy identification system, which includes the above-mentioned computer-readable storage medium.

[0074] Compared with the existing technology, the present invention proposes a method for identifying poppy by remote sensing using drones, which makes full use of the spectral and temperature characteristics of leaves in multispectral remote sensing images. Through the steps of leaf recognition, dewdrop recognition, feature extraction and machine learning model training, the present invention finally realizes accurate recognition of poppy plants in farmland images collected by drones. Compared with the existing remote sensing poppy recognition technology, the present invention has the following advantages:

[0075] 1. The use of drone low-altitude multispectral remote sensing can provide high-resolution real-time monitoring of local farmland, making up for the limitations of relying on satellite remote sensing data. The drone's hovering photography mode can obtain multispectral image data such as visible light, near infrared and thermal infrared, providing richer information input for subsequent feature extraction and recognition algorithms.

[0076] 2. In the leaf recognition and dewdrop recognition stages, the spectral and temperature characteristics of the vegetation growth status are fully utilized. The leaf recognition model based on color and texture can accurately locate the leaf area; the dewdrop recognition model based on image segmentation can detect the distribution of dewdrops on the leaf surface, which is important for distinguishing poppies from other similar plants.

[0077] 3. Extracting the spectrum, temperature and other features of leaf dew in near-infrared and thermal infrared images as input to the machine learning model can better characterize the physiological state of poppy plants, thereby improving the accuracy and robustness of recognition. Compared with the method that only uses the features of the visible light band, the solution of the present invention can still maintain a high recognition accuracy under poor lighting conditions or high crop coverage.

[0078] 4. Use supervised learning algorithms such as support vector machines (SVM) to train the poppy recognition model. Use known poppy sample annotation information as the training basis to learn more accurate poppy feature representations, thereby achieving better detection performance in practical applications. At the same time, optimize the model parameters by cross-validation to further improve the generalization ability.

[0079] In general, the UAV remote sensing poppy identification method proposed in the present invention fully utilizes the advantages of multispectral remote sensing technology and machine learning algorithms, and innovatively solves the problems existing in the existing methods. It solves the technical problem that when there are dewdrops on the surface of plant leaves, the spectral characteristics of plant leaves in the visible light band will be affected, resulting in the existing machine learning method's reduced ability to judge poppies. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0081] Figure 1 A flow chart of the method provided by the present invention; DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0083] like Figure 1 FIG. 1 is a flowchart of a method for identifying poppies by remote sensing using an unmanned aerial vehicle provided by the present invention. The method comprises the following steps:

[0084] S10, collecting multispectral remote sensing images of multiple located farmland areas sparsely planted with poppies in dewy weather;

[0085] S20, using a pre-trained leaf recognition model based on color recognition to identify a leaf area in the visible light image;

[0086] S30, marking the leaf area on the near infrared image and the thermal infrared image by aligning them with the visible light image;

[0087] S40, in the leaf area of ​​the near infrared image and the thermal infrared image, using a pre-trained dew drop recognition model to mark the dew drop area;

[0088] S50, extracting features of the dew drop areas in the near infrared image and the thermal infrared image to obtain near infrared dew drop features and thermal infrared dew drop features, respectively;

[0089] S60, using near infrared dew drop features and thermal infrared dew drop features as training inputs and using the marked poppy positions as training outputs, training a machine learning model to obtain a poppy recognition model;

[0090] S70, finally obtaining the image of the farmland to be tested of the poppy to be identified collected by the drone in dewy weather, and obtaining the near-infrared dewdrop features and thermal-infrared dewdrop features of the farmland image to be tested, recorded as the dewdrop features to be tested, inputting them into the poppy recognition model, obtaining the coordinates of the poppy identified in the farmland image to be tested and outputting them.

[0091] The specific implementation methods of the above steps are described in detail below:

[0092] Step S10: Collect multispectral remote sensing image data of multiple farmland areas where poppies are sparsely planted under dewy weather. The purpose of this step is to collect multispectral remote sensing image information required for identifying poppies, including visible light images, near infrared images, and thermal infrared images. The image data are collected under dewy weather because dew can enhance the spectral feature differences between poppy leaves and surrounding vegetation, which is conducive to subsequent identification.

[0093] Step S20: Use a pre-trained leaf recognition model based on color recognition to identify leaf areas in visible light images. The purpose of this step is to extract the leaf area information of vegetation from the visible light image, laying the foundation for the subsequent positioning of leaf areas on near-infrared and thermal infrared images. The leaf recognition model can adopt a supervised learning method based on support vector machine (SVM) or convolutional neural network (CNN), and train with a large number of labeled leaf samples to learn the color, texture and other features of the leaves in the visible light image, and finally form a model that can accurately identify the leaf area.

[0094] Step S30: On the near-infrared image and thermal infrared image, the leaf area is marked by aligning with the visible light image. This step is to accurately align and mark the leaf area identified on the visible light image in step S20 on the near-infrared and thermal infrared images. This requires registration processing of the three spectral images to ensure that the positions of the leaves on the three images are completely corresponding. In this way, during the subsequent feature extraction and machine learning model training, it can be ensured that the leaf areas on the three images are completely corresponding.

[0095] Step S40: In the leaf area of ​​the near infrared image and the thermal infrared image, a pre-trained dew drop recognition model is used to mark the dew drop area. The purpose of this step is to extract the dew drop area information on the leaf surface from the near infrared and thermal infrared images respectively. The dew drop recognition model can be trained to accurately locate the dew drop position by using the spectral and temperature characteristics of the leaf area in the near infrared and thermal infrared images based on an image segmentation method.

[0096] Step S50: Extract features of the dew drop areas in the near infrared image and the thermal infrared image to obtain near infrared dew drop features and thermal infrared dew drop features, respectively. The purpose of this step is to extract some representative spectral and temperature features from the dew drop areas in the near infrared and thermal infrared images, respectively, to provide input features for subsequent machine learning model training. The features that can be extracted include: average reflectance, variance, texture features, etc. of the dew drop area.

[0097] Step S60: Using near-infrared dew features and thermal infrared dew features as training inputs and the marked poppy positions as training outputs, a machine learning model is trained to obtain a poppy recognition model. This step uses the features extracted in step S50 and the known poppy position annotation information to train a machine learning model that can accurately identify poppies. The training of this model can adopt supervised learning algorithms such as support vector machines (SVM) or random forests. Different algorithms and parameter settings can be tried during training, and the best model can be selected by cross-validation.

[0098] Step S70: Finally, the near infrared dew features and thermal infrared dew features of the farmland image to be tested are obtained and input into the poppy recognition model trained in step S60 to obtain and output the poppy coordinates identified in the farmland image to be tested. This step is to input the actual farmland remote sensing image to be identified into the previously trained poppy recognition model. The model will predict the position coordinates of the poppy in the image based on the near infrared and thermal infrared dew features of the leaves in the image, and finally output the result.

[0099] In order to better understand the method of the present invention, the specific implementation of the method is explained in more detail below in conjunction with a specific formula:

[0100] Step S10: Multispectral remote sensing image acquisition

[0101] The purpose of this step is to collect multispectral remote sensing image data of farmland areas known to be planted with poppies under dew weather conditions, providing data support for subsequent image processing and machine learning model training. The specific steps are as follows:

[0102] Step 1: Use drone hovering photography to obtain visible light images I of the sparsely planted poppy fields that have been located.vis 、Near infrared image I nir and thermal infrared image I ir The drone’s hovering height h is controlled between 50-100 meters, the resolution can reach 0.1 meters / pixel, and the flight speed v is controlled between 5-10 meters / second.

[0103] Step 2: Perform geometric correction and radiation correction on the acquired multispectral remote sensing image data to ensure that the spatial positions of the three spectral images are completely aligned and eliminate the geometric distortion and light radiation differences that affect subsequent processing. Geometric correction can be achieved by using an affine transformation T based on feature points. geo :

[0104] x′=T geo (x) = Ax + b

[0105] Where x = [x, y] T and x′ = [x′, y′] T are the pixel coordinates before and after correction, A is a 2×2 affine transformation matrix, and b is a 2×1 translation vector. Radiation correction is performed using a model based on sensor parameters. The corrected image data satisfy:

[0106]

[0107] Where θ is the radiation correction parameter of the sensor.

[0108] Step 3: During the image acquisition process, the geographical location information p = [latitude, longitude] of the farmland area is synchronously recorded. T , establish the correspondence between multispectral remote sensing images and actual farmland locations, and provide support for subsequent spatial positioning.

[0109] Step 4: Corrected multispectral remote sensing image data And the corresponding geographic location information p is stored in the database for subsequent image processing and machine learning model training.

[0110] Step S20: Leaf recognition in visible light image

[0111] The purpose of this step is to use the pre-trained leaf recognition model based on color recognition to identify the leaf in the visible light image. The leaf area can be accurately identified in the image, laying the foundation for locating the leaf area in the near infrared and thermal infrared images. The specific steps are as follows:

[0112] Step 1: Retrieve the visible light image obtained in step S10 from the database as input.

[0113] Step 2: Use a pre-trained convolutional neural network (CNN) based leaf recognition model to Perform semantic segmentation to obtain the probability distribution P of whether each pixel belongs to the leaf area leaf The structure of the CNN model is as follows:

[0114] Input layer: accepts data,

[0115] Convolutional layer: extracts image features such as color and texture, h l =f(W l *h l-1 +b l )

[0116] Pooling layer: reduces feature dimension and improves model robustness. l+1 =pool(h l )

[0117] Fully connected layer: comprehensive features, output probability distribution of leaf area, P leaf =softmax(W fc h L +b fc )

[0118] Output layer: Gives the probability of whether each pixel belongs to the leaf area

[0119] The CNN model is obtained through supervised training with a large number of labeled leaf and non-leaf samples, and can accurately identify leaf areas in visible light images.

[0120] Step 3: Extract the leaf area probability distribution P obtained in step 2 leaf The central moment, texture features and other visual features are used as the output results of leaf recognition.

[0121] Step 4: The leaf recognition result and the corresponding spatial position coordinate information p leaf Stored in the database, it provides a basis for subsequent positioning of leaf areas on near-infrared and thermal infrared images.

[0122] Step S30: Labeling leaf regions in near infrared and thermal infrared images

[0123] The purpose of this step is to accurately align and annotate the leaf area identified on the visible light image in step S20 to the near infrared image. and thermal infrared images This provides accurate leaf area information for subsequent feature extraction and machine learning model training. The specific steps are as follows:

[0124] Step 1: Retrieve the blade area coordinate information p obtained in step S20 from the database leaf .

[0125] Step 2: Based on the correspondence between the multispectral remote sensing image recorded in step S10 and the actual farmland location T geo , the blade area coordinate information p in step S20 leaf Mapping to near infrared images and thermal infrared images On the top, get the corresponding leaf area coordinates and This requires applying the following transformations:

[0126] pleaf nir =Tgeo(p leaf )

[0127] pleaf ir =Tgeo(p leaf )

[0128] Step 3: In the near infrared image and thermal infrared images Mark the leaf area coordinates obtained in step 2 and Forming an annotated image and

[0129] Step S40: Identifying dew drop regions in near infrared and thermal infrared images

[0130] The purpose of this step is to use the pre-trained dew drop recognition model based on image segmentation to and thermal infrared images The dew area on the leaf surface is identified within the leaf area, providing the required data for subsequent feature extraction and machine learning model training. The specific steps are as follows:

[0131] Step 1: Retrieve the near infrared image obtained in step S30 from the database and thermal infrared images And the corresponding leaf area coordinates and

[0132] Step 2: Use the pre-trained CNN-based dew drop recognition model to and The leaf area in the image is segmented to obtain the probability distribution of whether each pixel belongs to the dew area. and The structure of the CNN model is as follows:

[0133] Input layer: accepts and data,

[0134] Convolutional layer: extracts the spectral and temperature features of the leaf area in the image, h l =f(Q l *h l-1 +b l )

[0135] Pooling layer: reduces feature dimension and improves model robustness. l+1 =pool(h l )

[0136] Fully connected layer: Comprehensive spectral temperature features, output probability distribution of dew drop area, P dew =softmax(W fc h L +b fc )

[0137] Output layer: Gives the probability of whether each pixel belongs to the dew area

[0138] The CNN model is obtained through supervised training with a large number of labeled dew and non-dew samples, and can accurately identify dew areas in near-infrared and thermal infrared images.

[0139] Step 3: Extract the near-infrared dew area probability distribution obtained in step 2 The average reflectivity μ nir ,variance And texture features Etc.; similarly, extract the probability distribution of thermal infrared dew area The average temperature μ ir , Temperature variance And texture features wait.

[0140] Step 4: The Near Infrared Dew Feature and thermal infrared dew characteristics And the corresponding spatial position coordinates and Stored in the database, it provides the required input features for subsequent machine learning model training.

[0141] Step S50: Dew drop feature extraction

[0142] The purpose of this step is to extract some representative spectral and temperature features from the dew drop region of the near infrared image and thermal infrared image obtained in step S40, respectively, to provide input data for subsequent machine learning model training. The specific steps are as follows:

[0143] Step 1: Retrieve the near infrared image obtained in step S40 from the database and thermal infrared images Dew area coordinates and

[0144] Step 2: According to the dew area coordinates and In near infrared images and thermal infrared images The features of the dew drop area are extracted separately, including:

[0145] Average reflectivity of near-infrared dew drop region

[0146] Reflectance variance of near-infrared dew drop region

[0147] Haralick texture characteristics of near-infrared dew drop region

[0148] Average temperature of thermal infrared dew area

[0149] Temperature variance in thermal infrared dew area

[0150] Haralick texture characteristics of thermal infrared dew area

[0151] Among them, N nir and N ir are the number of pixels in the near infrared and thermal infrared dew regions, respectively.

[0152] Step 3: The Near Infrared Dew Feature and thermal infrared dew characteristics

[0153] Step S60: Poppy recognition model training

[0154] The purpose of this step is to use the near-infrared dew features and thermal infrared dew features extracted in step S50 as input and the poppy sample annotations at known locations as output to train a machine learning model that accurately identifies poppies. The specific steps are as follows:

[0155] Step 1: Retrieve the near-infrared dew drop features extracted in step S50 from the database and thermal infrared dew characteristics As the input data of the machine learning model X = [X1, x2, ..., x n ] T ,in

[0156] Step 2: Retrieve the poppy sample annotation information Y = [y1, y2, ..., y n ] T , where y i ∈{0, 1} indicates whether the i-th sample is poppy (0 means non-poppy, 1 means poppy).

[0157] Step 3: Use the support vector machine (SVM) algorithm, take the near-infrared dew feature and thermal infrared dew feature X as input, and the poppy position label Y as output, and train to obtain the poppy recognition model f(x). The optimization goal of the SVM model is:

[0158]

[0159] sty i (w T φ(x i )+b)≥1-ξ i ,ξ i ≥0

[0160] Where φ(·) is the kernel function mapping and C is the penalty parameter, which can be adjusted to control the generalization ability of the model.

[0161] Step 4: Optimize the hyperparameter C and kernel function parameters of the SVM model through 5-fold cross validation to ensure that the poppy recognition accuracy of the model on the test set reaches more than 95%.

[0162] Step S70: Poppy position prediction

[0163] The purpose of this step is to use the poppy recognition model trained in step S60 to predict the actual farmland multispectral remote sensing image to be tested, and output the poppy position coordinate information detected in the image. The specific steps are as follows:

[0164] Step 1: Collect multispectral remote sensing image data of the farmland to be identified

[0165] Step 2: According to the method of step S30 and step S40, in the near infrared image of the farmland to be identified and thermal infrared images Mark the leaf area and dew area on the graph to get and

[0166] Step 3: Using the feature extraction method in step S50, and Extracting near infrared dew features from farmland images to be identified and thermal infrared dew characteristics Note that xtest .

[0167] Step 4: The dew feature x of the farmland image to be identified test Input into the poppy recognition model f(x) trained in step S60 for prediction, and obtain the model output

[0168] Step 5: Output based on the model The coordinates of the poppy positions detected in the image Output, where k is the number of poppies detected, each Contains latitude and longitude coordinate information. If the model cannot detect poppies in the farmland image to be identified, it will output a prompt message of "no poppies detected".

[0169] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the above-mentioned UAV remote sensing poppy identification method.

[0170] A third aspect of the present invention provides an unmanned aerial vehicle remote sensing poppy identification system, which includes the above-mentioned computer-readable storage medium.

[0171] Specifically, the principle of the present invention is:

[0172] First, in the image acquisition stage, make full use of the spectrum and temperature differences between poppy leaves and surrounding crops in dewy weather. Remote sensing image data of three spectra, visible light, near infrared and thermal infrared, are obtained by low-altitude drone photography. Since poppy identification is mostly about finding very few poppy plants in a large range of farmland, it is difficult to obtain features by simply analyzing the color and shape characteristics of plants due to the occlusion between leaves of different types of plants. If the detailed features of the leaf surface can be collected, such as the texture, villi, stomata and other detailed features of the leaf surface, the recognition accuracy of poppies will be greatly improved. However, since it is difficult to capture such detailed features in remote sensing images, such detailed features of the leaf surface are often not considered in the field of poppy identification. The detailed features of the leaf surface will also affect the size and distribution of dewdrops on its surface. The dewdrops on the leaves of different plants have different sizes due to the surface structure of the leaves. Under dewy conditions, the light reflection and thermal radiation characteristics of the leaves of each plant will be greatly different from those of other types of crops around it due to its own physiological characteristics, which is conducive to the subsequent recognition algorithm for accurate classification. At the same time, the geographical location information of the farmland area is recorded synchronously during collection, and the correspondence between the multispectral remote sensing image and the actual farmland location is established, providing a basis for subsequent spatial positioning.

[0173] Secondly, for the acquired multispectral remote sensing image data, the scheme of the present invention first uses a convolutional neural network model based on color recognition to perform semantic segmentation on the visible light image and accurately identify the leaf area in the image. The purpose of this step is to lay the foundation for the subsequent positioning of the leaf area on the near-infrared and thermal infrared images. Through supervised training with a large number of labeled leaf and non-leaf samples, the model can effectively extract the color and texture features of the leaves in the visible light image and give the probability of whether each pixel belongs to a leaf.

[0174] Next, the coordinates of the leaf areas identified in the visible light image are accurately aligned and annotated on the near-infrared image and thermal infrared image using the method of geometric transformation. The key to this step is to ensure that the leaf areas in the three spectral images are completely corresponding, providing accurate location information for subsequent feature extraction and machine learning model training.

[0175] In the leaf area of ​​the near-infrared image and the thermal infrared image, the scheme of the present invention uses a pre-trained convolutional neural network model based on image segmentation to identify the dew area on the leaf surface. Here, the high sensitivity of the near-infrared band to moisture and the high response of the thermal infrared band to temperature changes are utilized to accurately locate the distribution of dew on the leaves. From these identified dew areas, spectral and temperature features such as average reflectivity, variance, and texture features are extracted to provide input data for subsequent machine learning model training.

[0176] Finally, the poppy recognition model was trained using the support vector machine (SVM) algorithm, with near-infrared and thermal-infrared dew features as input and poppy sample annotations at known locations as supervised output. The model can effectively learn the feature representation of poppy plants in multispectral remote sensing images and make accurate predictions in actual farmland images to be tested. The generalization performance can be further improved by optimizing the model parameters through cross-validation, ensuring a high recognition accuracy even on unknown samples.

[0177] In summary, the UAV remote sensing poppy identification method proposed in the present invention fully utilizes the advantages of multispectral remote sensing technology and machine learning algorithms, and realizes accurate detection of poppy plants in farmland through steps such as leaf recognition, dewdrop recognition, feature extraction and model training.

[0178] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0179] A specific embodiment of the present invention is provided below: This embodiment is used to detect a small number of poppy plants arranged for experiments from farmland, where the farmland is located in the western part of a province, covers an area of ​​about 100 hectares, and is planted with a variety of crops, among which 30 poppy plants are temporarily scattered.

[0180] A DJI drone was selected as the test platform, equipped with a Zenmuse P1 multispectral camera. The camera has a high resolution of 45.7 megapixels and can simultaneously collect image data in three bands: visible light (RGB), near infrared, and thermal infrared. The drone flew automatically along a predetermined route in dewy weather, covering the entire farmland area and collecting a total of 420 multispectral images.

[0181] In order to obtain real reference data, agricultural experts were sent to conduct a field survey of the farmland. After careful inspection, they found about 30 poppy plants in the northeast corner of the farmland and marked their locations with GPS. These marked information will serve as a reference benchmark for the identification results of the method of the present invention.

[0182] Next, using the specific implementation scheme of the present invention, the collected multispectral image data is processed and analyzed, specifically including the following steps:

[0183] Step S10: Collecting multispectral remote sensing images of multiple farmland areas where poppies are sparsely planted under dewy weather conditions

[0184] In this step, the flight path and waypoints of the drone were first determined based on the GPS positioning information, covering the entire 100-hectare farmland area. In order to obtain high-quality image data, the camera exposure time t e Set it to 0.005 seconds. At the same time, according to the eye color temperature of the farmland, calculate the white balance parameter w = (1.2, 1.0, 0.8) to eliminate the influence of eye color cast.

[0185] After completing the camera parameter settings, the drone automatically flew along the predetermined path and collected a total of 420 multispectral images, including 140 visible light images. v , 140 near-infrared images I nir and 140 thermal infrared images I ir The spatial resolution of these image data is about 0.1 meters per pixel, which can clearly capture the details of vegetation in the farmland. At the same time, the drone geographic coordinates P = (x, y) corresponding to each image are also recorded, providing a spatial reference for subsequent processing.

[0186] Step S20: Using a pre-trained leaf recognition model based on color recognition, identify the leaf area in the visible light image.

[0187] In order to accurately extract the leaf area in the image, a convolutional neural network (CNN) based image segmentation model is used. The input of the model is a 224×224 pixel visible light image x, and the output is the probability P of each pixel belonging to the leaf or the background. l (x, y).

[0188] When training the model, a large number of leaf image samples of different types of crops, including rice, wheat, corn, vegetables, etc., totaling 8,000 images, were collected. These sample images were manually annotated to segment the credibility of whether each pixel belongs to a leaf. These image samples and annotation information were used as the training set. Where yi is a binary label map.

[0189] During the training process, VGG1 6 is used as the backbone network structure of the CNN model, and several convolutional layers and upsampling layers are added on this basis to form a complete image segmentation model. The loss function of this model is the cross entropy loss:

[0190]

[0191] Among them, f i (x i ; θ) is the output of the model, which indicates the probability that the i-th pixel belongs to a leaf. The model parameter θ is optimized using the back propagation algorithm until a high segmentation accuracy is achieved on the validation set.

[0192] The trained CNN model was applied to the 420 visible light images collected. v In the above example, the leaf area of ​​each image is quickly extracted to obtain the corresponding probability map P l (x, y).

[0193] Step S30: Marking the leaf area on the near infrared image and the thermal infrared image by aligning them with the visible light image

[0194] In order to accurately map the leaf area information obtained in step S20 to the near infrared image I nir and thermal infrared image I ir In this paper, an image registration algorithm based on SIFT feature points is adopted.

[0195] Specifically, first, in the visible light image I v 1224 SIFT key points were detected And extract their feature descriptors. Similarly, in the near infrared image I nir and thermal infrared image I ir The corresponding SIFT key points are also detected above and

[0196] Then, the nearest neighbor search method is used to establish the matching relationship between these SIFT key points. Based on these matching point pairs, the RANSAC algorithm is used to estimate the affine transformation matrix T from the visible light image to the near infrared / thermal infrared image. v→nir and T v→ir :

[0197]

[0198] Finally, the leaf area probability map P obtained in step S20 is transformed into l (x, y), accurately transferred to the near-infrared image I nir and thermal infrared image I ir On the top, we get the corresponding leaf probability map and

[0199] Step S40: In the leaf area of ​​the near infrared image and the thermal infrared image, a pre-trained dew drop recognition model is used to mark the dew drop area.

[0200] To detect the dew drop locations in the image, a method based on the YOLO object detection model is used. The model divides the image into 13×13 grids, and each grid is responsible for predicting 5 bounding boxes and their confidence that it contains an object.

[0201] A large number of agricultural scene images containing dewdrops were collected in advance as training sets where x i is the input image, y i During the training process, the following multi-task loss function is used to optimize the YOLO model end-to-end.

[0202] After training, the YOLO model is able to nir and thermal infrared image I ir The position and size of dew drops can be detected quickly and accurately in the leaf area, and the corresponding probability map P can be output. d (x, y).

[0203] Step S50: extracting features of the dew drop areas in the near infrared image and the thermal infrared image to obtain near infrared dew drop features and thermal infrared dew drop features respectively.

[0204] In this step, the following categories of visual features are extracted from the detected dew drop regions:

[0205] Color feature c: Convert the RGB image of the dew area to the HSV color space, and quantize the hue (H) to 16 bins, the saturation (S) and the value (V) to 4 bins each, and finally obtain a 256-dimensional color histogram feature vector.

[0206] Texture feature f t :First convert the color image into a grayscale image I g , and then calculate the gray level co-occurrence matrix G of adjacent pixels, and extract four statistical parameters such as contrast, correlation, and entropy as texture feature vectors.

[0207] f t =[Contrast,Correlation,Energy,Homogeneity]

[0208] Geometric characteristics: Parameters such as the area A, perimeter L, equivalent diameter D and aspect ratio AR of the dew drop region were measured.

[0209]

[0210]

[0211] Spectral characteristics f nir , f ir : The peak wavelength λ of the dew area in the near infrared and thermal infrared bands was counted p , half-peak width Δλ and maximum amplitude I m And other parameters.

[0212] Combining the above features, we get two high-dimensional feature vectors:

[0213] x nir =[f c , f t ,A,L,D,AR,f nir ]

[0214] x ir =[f c , f t ,A,L,D,AR,f ir ]

[0215] Step S60: Using the near infrared dew feature and the thermal infrared dew feature as training input and the marked poppy position as training output, a machine learning model is trained to obtain a poppy recognition model.

[0216] In order to build a machine learning model that can identify poppies, 30 poppy plants found by agricultural experts during field surveys were collected as samples and their position coordinates in the image were marked. nir , xir and its labels y∈{0,1} as training set

[0217] Step S60 (continued): using the near-infrared dew drop features and the thermal infrared dew drop features as training inputs and the marked poppy positions as training outputs, training a machine learning model to obtain a poppy recognition model.

[0218] Support vector machine (SVM) is selected as the poppy recognition model. SVM separates positive and negative samples as much as possible by finding an optimal hyperplane. Its decision function is:

[0219]

[0220] Using radial basis kernel function K(x, x′)=exp(-γ||xx′|| 2 ) constructs the SVM model and optimizes the kernel function parameter γ = 0.1 and the penalty coefficient C = 10 through the grid search method to achieve the highest recognition accuracy on the cross-validation set:

[0221]

[0222] The trained SVM model is the final poppy recognition model, which can be used to predict whether poppies exist in new farmland images.

[0223] Step S70: finally obtaining the farmland image of the poppy to be identified collected by the drone in dewy weather, and obtaining the near-infrared dewdrop features and thermal-infrared dewdrop features of the farmland image to be identified, which are recorded as the dewdrop features to be identified, and input into the poppy identification model to obtain the coordinates of the poppy identified in the farmland image to be identified and output

[0224] In actual application, the image of the farmland to be tested was taken by the drone in another dewy weather.

[0225] First, the CNN model trained in step S20 is applied to the visible light image Probability map of leaf area extracted Then, the image registration method in step S30 is used to accurately map these leaf regions to the near infrared image. and thermal infrared images superior.

[0226] Next, the YOLO model trained in step S40 is applied to detect the dew drop positions in the leaf area of ​​the near infrared and thermal infrared images to obtain a probability map From these detection results, near-infrared dew features are extracted and thermal infrared dew characteristics

[0227] Finally, these feature vectors to be tested The result is input into the SVM model trained in step S60 to obtain the prediction output f(x*). If f(x*)>0, it means that poppy exists in the farmland image to be tested, and its position in the image is output in the form of GPS coordinates.

[0228] Technical effect verification

[0229] In order to verify the effectiveness of the method of the present invention, the above processing flow was applied to 420 multispectral images collected by UAVs, and compared and analyzed with the reference results of field surveys.

[0230] First, in step S20, the CNN image segmentation model achieved a leaf recognition accuracy of 95.3% on the validation set. This shows that the model can very accurately extract the leaf area in the image, laying a good foundation for subsequent steps.

[0231] Then, in step S40, the average accuracy (mAP) of the YOLO target detection model on the validation set reaches 92.1%. That is, it can accurately locate the position of the dew drops in the image, providing a reliable input for feature extraction.

[0232] Finally, these features are input into the SVM model trained in step S60 for prediction. In comparison with the field survey results, the model achieved 100% recognition accuracy on 30 known poppy plants, without missing any plant. Although only this case achieved 100% accuracy, it also proved the recognition effect of this solution. At the same time, there was no false positive in the 420 images to be tested.

[0233] Through the above quantitative evaluation, it can be seen that the method of the present invention has achieved high performance indicators in all key links. Since other methods are not suitable for poppy recognition when the leaf surface has moisture or dew. Therefore, in the final case of moisture or dew on the leaf surface, the poppy recognition accuracy is higher than other methods, solving the problem of lack of effective monitoring means in this case.

Claims

1. A method for identifying poppies by remote sensing using an unmanned aerial vehicle, characterized in that: The following steps are involved: S10, collecting multispectral remote sensing images of a plurality of farmland areas where poppies are sparsely planted and located in dewy weather; the multispectral remote sensing images include image data of visible light images, near infrared images, and thermal infrared images; S20, using a pre-trained leaf recognition model based on color recognition to identify a leaf area in the visible light image; S30, marking leaf areas on the near infrared image and the thermal infrared image by aligning them with the visible light image; S40, marking the dew area in the leaf area of ​​the near infrared image and the thermal infrared image using a pre-trained dew recognition model; S50, extracting features of the dew drop areas in the near infrared image and the thermal infrared image to obtain near infrared dew drop features and thermal infrared dew drop features, respectively; S60, using the near-infrared dewdrop features and the thermal infrared dewdrop features as training inputs and the marked poppy positions as training outputs, training a machine learning model to obtain a poppy recognition model; S70, finally obtaining the image of the farmland to be tested of the poppy to be identified collected by the drone in dewy weather, and obtaining the near-infrared dewdrop features and thermal-infrared dewdrop features of the farmland image to be tested, recording them as the dewdrop features to be tested, inputting them into the poppy identification model, obtaining the coordinates of the poppy identified in the farmland image to be tested and outputting them.

2. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 1, characterized in that: The leaf recognition model is a convolutional neural network model, and its structure includes: Input layer: accepts visible light image data; Convolutional layer: extract color and texture features of the image; Pooling layer: reduces feature dimension; Fully connected layer: integrate features and output the segmentation results of the leaf area; Output layer: gives the probability of whether each pixel belongs to the leaf area; Through supervised training with a large number of labeled leaf and non-leaf samples, a model for identifying leaf areas is finally formed.

3. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 2, characterized in that: The dewdrop recognition model is a convolutional neural network model, and its structure includes: Input layer: accepts near infrared image and thermal infrared image data; Convolutional layer: extracts the spectral and temperature features of the leaf area in the image; Pooling layer: reduces feature dimension; Fully connected layer: integrates spectral temperature features and outputs the segmentation result of the dew drop area; Output layer: gives the probability of whether each pixel belongs to the dew area; Through supervised training with a large number of labeled dew and non-dew samples, a model for identifying the position of dew drops on the leaf surface is finally formed.

4. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 1, characterized in that: The near-infrared dew drop characteristics are the spectral and temperature characteristics of the average reflectivity, variance, and texture characteristics of the near-infrared dew drop area; the thermal infrared dew drop characteristics are the spectral and temperature characteristics of the average reflectivity, variance, and texture characteristics of the thermal infrared dew drop area.

5. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 1, characterized in that: The poppy recognition model uses a support vector machine as a basic model.

6. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 1, characterized in that: The dew drop recognition model also considers leaf temperature information.

7. The method for identifying poppy by remote sensing using an unmanned aerial vehicle according to claim 1, characterized in that: The multispectral remote sensing image also includes a mid-infrared image.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed, they are used to execute the method for identifying poppies using remote sensing by an unmanned aerial vehicle according to any one of claims 1 to 7.

9. An unmanned aerial vehicle remote sensing poppy identification system, characterized in that: Contains the computer-readable storage medium of claim 8.

Images