Poisonous grass identification method and equipment based on satellite, unmanned aerial vehicle and near-ground image

Abstract

The invention discloses a poison grass recognition method and device based on a satellite, an unmanned aerial vehicle and a near-ground image, and relates to the technical field of image recognition, and the method comprises the following steps: inputting a current regional scale satellite remote sensing image into a regional scale poison grass recognition model, and obtaining a regional scale poison grass abundance recognition result. Wherein the regional scale poison grass identification model is obtained by training based on a historical regional scale feature data set and a predicted sample plot scale poison grass abundance identification result; the historical region scale feature data set is a data set obtained by performing screening operation on historical region scale satellite remote sensing images on the basis of historical quadrat scale feature variables; the historical quadrat scale feature variable is a feature obtained after processing a historical near-surface hyperspectral image of the target area. By adopting the steps, the recognition precision of the abundance of the toxic grass is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and device for identifying poisonous weeds based on satellite, unmanned aerial vehicle and near-ground images. Background Art

[0002] In recent years, due to climate change and unreasonable human activities (such as overgrazing), alpine grassland ecosystems are facing severe invasion of poisonous weeds, and the production and ecological service functions of grasslands are gradually declining. The invasion and spread of poisonous weeds have become an important factor restricting the sustainable development of grassland animal husbandry, seriously affecting the ecological security of grasslands. Therefore, it is necessary to identify the distribution of poisonous weeds to ensure the smooth progress of poisonous weed control. However, the accuracy of current poisonous weed identification methods is not high enough, which limits the accurate grasp of the distribution of poisonous weeds at the regional scale and restricts the further development of poisonous weed control. Summary of the invention

[0003] The purpose of this application is to provide a method and device for identifying poisonous weeds based on satellite, UAV and near-ground images, which can improve the accuracy of identifying the abundance of poisonous weeds.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a method for identifying poisonous weeds based on satellite, UAV and near-ground images, comprising:

[0006] Get the current regional-scale satellite remote sensing image of the target area.

[0007] The current regional-scale satellite remote sensing image is input into a regional-scale poisonous weed identification model that matches the current regional-scale satellite remote sensing image to obtain a regional-scale poisonous weed abundance identification result.

[0008] The process of determining the regional scale poisonous weed identification model is as follows:

[0009] Obtain a historical plot-scale hyperspectral image of the target area; the historical plot-scale hyperspectral image is collected by unmanned aerial vehicle equipment.

[0010] The historical plot-scale hyperspectral image is input into a plot-scale poisonous weed identification model that matches the historical plot-scale hyperspectral image to obtain a plot-scale poisonous weed abundance identification result.

[0011] A regional-scale poisonous weed identification network is trained based on a historical regional-scale characteristic data set and a sample plot-scale poisonous weed abundance identification result to obtain a regional-scale poisonous weed identification model; the historical regional-scale characteristic data set is a data set obtained by screening historical regional-scale satellite remote sensing images based on historical sample plot-scale characteristic variables; the historical sample plot-scale characteristic variables are features obtained by processing historical near-ground hyperspectral images of the target area.

[0012] Optionally, the training process of the plot-scale poisonous weed identification model matched with the plot-scale hyperspectral image is as follows:

[0013] Construct a historical plot-scale feature dataset;

[0014] Obtain the identification results of poisonous weeds at the historical sample scale;

[0015] The historical plot-scale characteristic data set and the historical plot-scale poisonous weed identification results are used to train a plot-scale poisonous weed identification network to obtain a plot-scale poisonous weed identification model.

[0016] Optionally, the process of determining the historical plot scale characteristic dataset is as follows:

[0017] Based on the characteristic variables of historical plot scale, multiple historical plot scale hyperspectral images are screened to obtain multiple historical plot scale hyperspectral images for identifying poisonous weeds.

[0018] Based on a plurality of historical plot-scale hyperspectral images for identifying poisonous weeds, a historical plot-scale feature dataset is constructed.

[0019] Optionally, the process of determining the historical regional scale feature dataset is as follows:

[0020] Acquire multiple historical regional-scale satellite remote sensing images of the target area.

[0021] Based on the historical sample scale characteristic variables, a number of historical regional-scale satellite remote sensing images were screened to select a number of historical regional-scale satellite remote sensing images for identifying poisonous weeds.

[0022] Based on a plurality of historical regional-scale satellite remote sensing images for identifying poisonous weeds, a historical regional-scale feature dataset is constructed.

[0023] Optionally, the regional-scale poisonous weed identification network is a three-dimensional convolutional neural network.

[0024] Optionally, the three-dimensional convolutional neural network includes X input layers, 3 convolutional layers, 3 maximum pooling layers, 3 fully connected layers and 1 output layer.

[0025] Optionally, the historical near-ground hyperspectral image includes a plurality of sub-images; and the process of determining the historical sample plot scale characteristic variable is as follows:

[0026] For each sub-image, do the following:

[0027] The sub-image is processed to obtain the spectral characteristics of the poisonous weeds and the spectral characteristics of the forage grass corresponding to the sub-image, and the first difference is calculated based on the spectral characteristics of the poisonous weeds and the spectral characteristics of the forage grass corresponding to the sub-image.

[0028] Based on the gray level co-occurrence matrix method, the sub-image is processed to obtain the poisonous grass texture features and the forage grass texture features corresponding to the sub-image, and the second difference is calculated based on the poisonous grass texture features and the forage grass texture features corresponding to the sub-image.

[0029] According to the first difference or the second difference, it is determined whether the sub-image is an image with features for identifying poisonous weeds, and when the sub-image is an image with features for identifying poisonous weeds, the spectral features and texture features of poisonous weeds corresponding to the sub-image are determined as historical sample plot scale feature variables.

[0030] Optionally, the spectral features include: band reflectivity, a first-order derivative spectrum calculated based on a reflectivity image, and a spectral index.

[0031] The first difference includes: the difference between the reflectance of the poisonous grass band and the reflectance of the forage grass band, the difference between the first-order derivative spectrum of the poisonous grass and the first-order derivative spectrum of the forage grass, and the difference between the spectral index of the poisonous grass and the spectral index of the forage grass.

[0032] The texture features include: gray value mean, gray value variance, image contrast, image entropy, second-order moment of image, image correlation, image synergy and image heterogeneity.

[0033] The second difference includes: the difference between the mean grayscale value of poisonous weeds and the mean grayscale value of forage grass, the difference between the grayscale value variance of poisonous weeds and the grayscale value variance of forage grass, the difference between the contrast of poisonous weed images and the contrast of forage grass images, the difference between the entropy of poisonous weed images and the entropy of forage grass images, the difference between the second-order moment of poisonous weed images and the second-order moment of forage grass images, the difference between the image correlation of poisonous weeds and the image correlation of forage grass, the difference between the synergy of poisonous weed images and the synergy of forage grass images, and the difference between the anisotropy of poisonous weed images and the anisotropy of forage grass images.

[0034] Optionally, judging whether the sub-image is an image having features for identifying poisonous weeds according to the first difference or the second difference includes:

[0035] The Mahalanobis distance method is used to determine whether the first difference of each sub-image is greater than a first significance threshold. If so, the sub-image is determined as an image with features for identifying poisonous weeds.

[0036] The Mahalanobis distance method is used to determine whether the second difference of each sub-image is greater than a second significance threshold. If so, the sub-image is determined as an image having a feature for identifying poisonous weeds.

[0037] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for identifying poisonous weeds based on satellite, UAV, and near-ground images.

[0038] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0039] The present application provides a method and device for identifying poisonous weeds based on satellite, UAV and near-ground images, which obtains the current regional-scale satellite remote sensing image of the target area and inputs it into the regional-scale poisonous weed identification model to obtain the regional-scale poisonous weed abundance identification result. The regional-scale poisonous weed identification model is trained based on the historical regional-scale feature data set and the predicted sample plot-scale poisonous weed abundance identification result; the historical regional-scale feature data set is a data set obtained by screening the historical regional-scale satellite remote sensing image based on the historical sample plot scale feature variable; the historical sample plot scale feature variable is a feature obtained by processing the historical near-ground hyperspectral image of the target area.

[0040] This application first uses the measured sample point data in the sample plot and the sample plot scale hyperspectral image to train the sample plot scale poisonous weed identification model, and then obtains the sample plot scale poisonous weed identification results; then, the sample plot scale poisonous weed identification results and historical regional scale satellite remote sensing images are used to train the regional scale poisonous weed identification model, and then obtain the regional scale poisonous weed abundance identification results. By introducing historical sample plot scale hyperspectral images, this application well achieves the scale connection between historical near-ground hyperspectral images and historical regional scale satellite remote sensing images in the modeling process, and significantly improves the recognition accuracy of poisonous weed abundance.

[0041] Before training the model, the present application uses historical sample plot scale feature variables to screen historical regional scale satellite remote sensing images and historical sample plot scale hyperspectral images, and removes a large number of images that are not helpful for identifying poisonous weeds, and only retains a small number of historical regional scale satellite remote sensing images and historical sample plot scale hyperspectral images used to identify poisonous weeds. Therefore, the present application is faster when training a poisonous weed identification model based on historical regional scale satellite remote sensing images used to identify poisonous weeds and historical sample plot scale hyperspectral images used to identify poisonous weeds, and the trained model has higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flowchart of a method for identifying poisonous weeds based on satellite, UAV and near-ground images provided in Example 1 of the present application;

[0044] Figure 2 A schematic diagram of the process of determining a regional-scale noxious weed identification model provided in Example 1 of the present application;

[0045] Figure 3 for Figure 2 Schematic diagram of the process of determining the historical regional scale characteristic dataset;

[0046] Figure 4 A schematic diagram of the training process of the plot-scale poisonous weed identification model provided in Example 1 of the present application;

[0047] Figure 5 A schematic diagram of the structure of a regional-scale poisonous weed identification network provided in Example 1 of the present application;

[0048] Figure 6 A partial enlarged view of the regional-scale poisonous weed identification network provided in Example 1 of the present application;

[0049] Figure 7 A schematic diagram of the process of determining the historical sample scale characteristic variables provided in Example 1 of the present application;

[0050] Figure 8 The overall architecture diagram of the poisonous weed identification method based on satellite, UAV and near-ground images provided in Example 2 of the present application;

[0051] Fig. 9A schematic diagram of the structure of a computer device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] At present, remote sensing observations for poisonous weed identification are mainly based on the scale of ground plots, or small-scale observations are carried out by combining ground plot images and plot drone remote sensing images, while remote sensing identification of poisonous weed abundance at the regional scale is rare and is mainly carried out by combining ground plots and satellite remote sensing images. Due to the large scale difference between ground plots and satellite remote sensing images, the method of combining ground plots and satellite remote sensing images is prone to scale connection difficulties, resulting in poor recognition accuracy. In response to the difficulties in scale connection and poor recognition accuracy of plots and satellite remote sensing images during poisonous weed identification, this patent provides a method for poisonous weed identification based on satellites, drones and near-ground images. By using drones to obtain plot-scale hyperspectral images, the problem of scale connection difficulties between near-ground plot hyperspectral images and regional-scale satellite remote sensing images (satellite remote sensing images) is solved during the modeling process, thereby improving the recognition accuracy of poisonous weed abundance at the regional scale (i.e., satellite remote sensing image scale).

[0054] It should be noted that the regional-scale satellite remote sensing images, plot-scale hyperspectral images and near-ground hyperspectral images mentioned in this article are all remote sensing images.

[0055] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0056] In an exemplary embodiment 1, Figure 1 As shown, a method for identifying poisonous weeds based on satellite, UAV and near-ground images is provided, and the method comprises the following steps:

[0057] Step 101: Acquire a current regional-scale satellite remote sensing image of a target area.

[0058] Step 102: Input the current regional-scale satellite remote sensing image into a regional-scale poisonous weed identification model that matches the current regional-scale satellite remote sensing image to obtain a regional-scale poisonous weed abundance identification result.

[0059] Furthermore, if Figure 2As shown, the determination process of the regional scale poisonous weed identification model is as follows:

[0060] Step 201: Acquire a historical plot-scale hyperspectral image of a target area.

[0061] As an exemplary implementation, in step 201, the historical plot scale hyperspectral image is collected by an unmanned aerial vehicle (UAV), the model of which is DJI M600 PRO, with an imaging spectrometer mounted on the drone. The UAV flies vertically over the plot at a height of about 50 m above the ground, and selects clear, cloudless weather with low wind speed to obtain hyperspectral images in the field between 10 am and 2 pm.

[0062] Before collecting site-scale hyperspectral images, the following preliminary preparations are required:

[0063] Typical alpine degraded grasslands were selected as sample plots, and standard whiteboards and positioning marks were laid out in the sample plots. Typical alpine degraded grassland sample plots should contain common poisonous grasses (such as Stellera chamaejasme and Ligularia japonica) and common fine forage grasses (such as Kobresia sichuanensis and Kobresia dasyphylla) of typical alpine grasslands. The purpose of laying out standard whiteboards is to calculate the spectral reflectance of images, and the purpose of laying out positioning marks is for subsequent multi-strip image stitching and geometric correction.

[0064] Step 202: inputting the historical plot-scale hyperspectral image into a plot-scale noxious weed identification model that matches the historical plot-scale hyperspectral image to obtain a predicted plot-scale noxious weed abundance identification result.

[0065] Step 203: training a regional-scale poisonous weed identification network based on the historical regional-scale characteristic dataset and the predicted plot-scale poisonous weed abundance identification results to obtain a regional-scale poisonous weed identification model.

[0066] The historical regional scale feature data set is a data set obtained by screening the historical regional scale satellite remote sensing images based on the historical sample scale feature variables; the historical sample scale feature variables are the features obtained by processing the near-ground hyperspectral images of the target area.

[0067] Furthermore, if Figure 3 As shown, the process of determining the historical regional scale feature dataset is as follows:

[0068] Step 301: Acquire multiple historical regional-scale satellite remote sensing images of the target area.

[0069] Step 302: Based on the historical sample scale characteristic variables, a plurality of historical regional scale satellite remote sensing images are screened to obtain a plurality of historical regional scale satellite remote sensing images for identifying poisonous weeds.

[0070] Step 303: construct a historical regional scale feature dataset based on the plurality of historical regional scale satellite remote sensing images used to identify poisonous weeds.

[0071] Furthermore, if Figure 4 As shown, the training process of the plot-scale poisonous weed identification model includes:

[0072] Step 401: construct a historical plot scale feature dataset;

[0073] As an exemplary implementation, in step 401, the process of determining the historical plot scale feature dataset is as follows:

[0074] Based on the characteristic variables of historical plot scale, multiple historical plot scale hyperspectral images are screened to obtain multiple historical plot scale hyperspectral images for identifying poisonous weeds.

[0075] Based on a plurality of historical plot-scale hyperspectral images for identifying poisonous weeds, a historical plot-scale feature dataset is constructed.

[0076] Step 402: Obtain historical sample-scale poisonous weed identification results.

[0077] In step 402, the historical plot-scale poisonous weed identification result is derived from historical measured sample point data in the plot.

[0078] As an alternative implementation, the historical quadrat-scale poisonous weed identification result in step 402 can also be obtained by using a quadrat-scale poisonous weed identification model to identify historical near-ground hyperspectral images to obtain the historical quadrat-scale poisonous weed identification result. The quadrat-scale poisonous weed identification model is an existing random forest model.

[0079] Step 403: using the historical plot-scale characteristic data set and the historical plot-scale poisonous weed abundance identification results to train a plot-scale poisonous weed identification network to obtain a plot-scale poisonous weed identification model.

[0080] As an exemplary implementation, in step 403, a plurality of historical plot-scale hyperspectral images for identifying poisonous weeds are used as input data of the model, and historical plot-scale poisonous weed abundance identification results are used as label data of the model to train a plot-scale poisonous weed identification network and obtain a plot-scale poisonous weed identification model.

[0081] Furthermore, the regional-scale poisonous weed identification network is a three-dimensional convolutional neural network.

[0082] The three-dimensional convolutional neural network includes X input layers, 3 convolutional layers, 3 maximum pooling layers, 3 fully connected layers and 1 output layer. X is the number of bands input to the input layer, and the number of bands is determined according to the number of corresponding bands in the multiple historical regional-scale satellite remote sensing images for identifying poisonous weeds screened in step 302.

[0083] The connection relationship between the layers of the three-dimensional convolutional neural network is as follows Figure 5 As shown. Among them, the size of the convolution kernel is 3*3*X, the step size is 1*1*1, and each convolution layer is activated by the Relu function. The number of convolution kernels in the first convolution layer, the second convolution layer, and the third convolution layer are 16, 32, and 64 respectively. In the pooling process, the step size is set to 2 times to compress and extract image features. The output layer uses the Softmax function as the activation function. The output result is a two-dimensional image with the poisonous weed recognition result, and the two-dimensional image with the poisonous weed recognition result is an image with multiple rows and columns, and each pixel in the image will output a predicted plant type.

[0084] like Figure 6 As shown in the figure, for each pixel, the three-dimensional convolutional neural network will calculate the probability of each plant type at the pixel, and select the plant type with the highest probability as the plant type at the pixel, and finally present it to each pixel of the two-dimensional image with the results of poisonous weed identification. For example, the probabilities of Kobresia dasyphylla, Elsholtzia nectarina, Ligustrum lucidum, Wolfberry and Kobresia sichuanensis are 0.03, 0.02, 0.65, 0.12 and 0.18 respectively. The one with the highest probability is Ligustrum lucidum, so the plant type at the pixel is determined to be Ligustrum lucidum.

[0085] Furthermore, the historical near-ground hyperspectral image includes a plurality of sub-images.

[0086] Furthermore, if Figure 7 As shown in FIG. 1 , the process of determining the historical sample scale characteristic variables is as follows:

[0087] For each sub-image, do the following:

[0088] Step 701a: Process the sub-image to obtain the spectral characteristics of poisonous weeds and the spectral characteristics of forage grass corresponding to the sub-image, and calculate the first difference based on the spectral characteristics of poisonous weeds and the spectral characteristics of forage grass corresponding to the sub-image.

[0089] The spectral characteristics include: band reflectivity, first-order derivative spectrum calculated based on the reflectivity image, and spectral index.

[0090] The first difference includes: the difference between the reflectance of the poisonous grass band and the reflectance of the forage grass band, the difference between the first-order derivative spectrum of the poisonous grass and the first-order derivative spectrum of the forage grass, and the difference between the spectral index of the poisonous grass and the spectral index of the forage grass.

[0091] Step 701b: Based on the gray level co-occurrence matrix method, the sub-image is processed to obtain the poisonous grass texture features and the forage grass texture features corresponding to the sub-image, and the second difference is calculated based on the poisonous grass texture features and the forage grass texture features corresponding to the sub-image.

[0092] The texture features include: gray value mean, gray value variance, image contrast, image entropy, second-order moment of image, image correlation, image synergy and image heterogeneity.

[0093] The second difference includes: the difference between the mean grayscale value of poisonous weeds and the mean grayscale value of forage grass, the difference between the grayscale value variance of poisonous weeds and the grayscale value variance of forage grass, the difference between the contrast of poisonous weed images and the contrast of forage grass images, the difference between the entropy of poisonous weed images and the entropy of forage grass images, the difference between the second-order moment of poisonous weed images and the second-order moment of forage grass images, the difference between the image correlation of poisonous weeds and the image correlation of forage grass, the difference between the synergy of poisonous weed images and the synergy of forage grass images, and the difference between the anisotropy of poisonous weed images and the anisotropy of forage grass images.

[0094] Step 702: Determine whether the sub-image is an image with features for identifying poisonous weeds based on the first difference or the second difference, and when the sub-image is an image with features for identifying poisonous weeds, determine the spectral features and texture features of the poisonous weeds corresponding to the sub-image as historical plot scale feature variables.

[0095] The determining, according to the first difference or the second difference, whether the sub-image is an image having a feature for identifying poisonous weeds comprises:

[0096] The Mahalanobis distance method is used to determine whether the first difference of each sub-image is greater than a first significance threshold. If so, the sub-image is determined as an image with features for identifying poisonous weeds.

[0097] The Mahalanobis distance method is used to determine whether the second difference of each sub-image is greater than a second significance threshold. If so, the sub-image is determined as an image having a feature for identifying poisonous weeds.

[0098] In an exemplary embodiment 2, Figure 8 As shown, the overall architecture diagram of the poisonous weed identification method based on satellite, UAV and near-ground images provided by this application is as follows:

[0099] First, obtain near-ground hyperspectral data, drone hyperspectral data of typical plots, and satellite remote sensing images of the study area. The near-ground highlight data set of this embodiment is equivalent to the historical near-ground hyperspectral images in the previous embodiment; the drone hyperspectral data of typical plots in this embodiment is equivalent to the historical plot-scale hyperspectral images in the previous embodiment; and the satellite remote sensing images of the study area in this embodiment are equivalent to the historical regional-scale satellite remote sensing images in the previous embodiment.

[0100] The process of obtaining the UAV hyperspectral image of a typical sample site is as follows: the model of the UAV is DJI M600PRO, with an imaging spectrometer mounted on it. The UAV flies vertically over the sample site at a height of about 50m above the ground, and the weather is clear and cloudless with low wind speed, and the hyperspectral image is obtained in the field between 10 am and 2 pm.

[0101] The preprocessing process of the UAV hyperspectral image of a typical sample site is as follows: the UAV gradually collects data of the entire area according to the predetermined flight route, and then obtains a complete hyperspectral image of the sample site by stitching multiple flight strips. The spectral reflectance image of the sample site is calculated using the white board laid out in the sample site as a reference and the known radiation intensity of the white board as a benchmark.

[0102] The satellite remote sensing images of the study area are Sentinel 2A satellite images, and their imaging time should be consistent with the time of field sampling, at least ensuring that they are in the same year and season.

[0103] Secondly, pure pixels of typical poisonous grasses (such as wolfsbane, Ligusticum chuanxiensis, etc.) and common excellent forage grasses (such as Kobresia sichuanensis and Kobresia xiaoyuensis) were selected from the near-ground hyperspectral data, and the spectral characteristics of poisonous grasses and common excellent forage grasses (including the reflectance of the band, the first-order derivative calculated based on the reflectance image, and the spectral index) were extracted from the pure pixels. At the same time, the texture features of the image were calculated based on the gray-level co-occurrence matrix method. The method for extracting texture features is as follows: the sample quadrat hyperspectral image is transformed by principal component to obtain the first principal component, and the texture features of the first principal component are extracted based on the gray-level co-occurrence matrix method. The size of the moving window is set to 3*3, and one pixel is moved each time for calculation. The extracted texture features include mean, variance, contrast, information entropy, second-order moment, correlation, synergy and dissimilarity.

[0104] After obtaining the spectral characteristics and texture characteristics of poisonous weeds and common high-quality forage grasses, the spectral characteristics and texture characteristics are screened based on the Mahalanobis distance method, and some of the spectral characteristics and texture characteristics are removed to screen out the spectral characteristics and texture characteristics that are truly helpful in distinguishing poisonous weeds from common high-quality forage grasses.

[0105] The feature screening includes: using the Mahalanobis distance method to screen out variables that are truly beneficial to the identification of poisonous weeds, thereby reducing the amount of data. When screening, the number of degrees of freedom is set to be equal to the number of plant types in the sample. When the significance threshold P<0.01, the threshold of the chi-square distribution is obtained. The Mahalanobis distance of the spectral characteristics (or texture characteristics) of poisonous weeds and good forage grasses in each band is calculated. If the Mahalanobis distance is greater than the threshold of the chi-square distribution, the spectral characteristics (or texture characteristics) corresponding to the band with a Mahalanobis distance greater than the threshold are selected as the spectral characteristics (or texture characteristics) that are truly beneficial to distinguishing poisonous weeds from common good forage grasses.

[0106] Next, the drone hyperspectral images of typical sample plots were processed by flight strip stitching and reflectance correction to obtain the spectral reflectance images of the sample plots. Then, based on the spectral reflectance images after the above processing, the spectral features and texture features that are truly helpful in distinguishing poisonous grasses from common high-quality forage grasses were screened for the drone hyperspectral images of typical sample plots to obtain the sample plot scale spectral-texture feature dataset.

[0107] The screening operation may include: calculating the difference in grayscale values ​​between poisonous weeds and common high-quality forage grass images in different bands, and if the difference is greater than a set threshold, the image is included in the plot-scale spectral-texture feature data set. The purpose of screening the drone hyperspectral images of typical plots is to eliminate invalid images that cannot distinguish between poisonous weeds and common high-quality forage grasses, thereby reducing the amount of data and improving the training efficiency of the model and the model's ability to distinguish between poisonous weeds and common high-quality forage grasses.

[0108] It should be noted that the grayscale value difference between poisonous grass and common good forage grass images in different bands of drone hyperspectral images in the screening process is only one of the spectral and texture features. In actual applications, it is necessary to compare more than just the grayscale value parameter, and comprehensively compare the differences in multiple spectral and texture features to select multiple images that are conducive to identifying poisonous grass. Multiple images that are conducive to identifying poisonous grass can be divided into two parts, one for reflecting the spectral characteristics of poisonous grass, and the other for reflecting the texture characteristics of poisonous grass.

[0109] After obtaining the plot-scale spectral-texture feature dataset, the plot-scale spectral-texture feature dataset was used to train a plot-scale convolutional neural network to obtain a plot-scale poisonous weed abundance recognition model.

[0110] Then, the plot-scale poisonous weed abundance identification model is used to identify the plot-scale hyperspectral image to obtain the plot-scale poisonous weed abundance identification results, which specifically includes: taking part of the sample point data of poisonous weeds and other plants measured on the ground as learning samples, and the other part as accuracy verification samples; training and establishing a plot-scale convolutional neural network poisonous weed identification model to identify the poisonous weeds at the plot scale and obtain the spatial distribution map of the poisonous weeds at the plot scale.

[0111] Finally, a part of the data of the poisonous weed abundance identification results at multiple plot scales was used as learning samples, and the other part was used as accuracy verification samples to train the regional-scale poisonous weed abundance convolutional neural network, establish the regional-scale poisonous weed abundance convolutional neural network identification model, complete the high-precision extraction of the regional-scale poisonous weed abundance distribution, and obtain the regional-scale poisonous weed abundance map. According to the regional-scale poisonous weed abundance map, the spatial differentiation characteristics of regional-scale poisonous weeds and the invasion situation, a targeted implementation plan for the control of poisonous weeds is proposed.

[0112] It should be noted that if the time period is short, such as after a week or half a month, and the topography of the target area has not undergone seasonal changes, the current regional-scale satellite remote sensing image can be directly input into the regional-scale poisonous weed identification model to output the regional-scale poisonous weed abundance identification results without the need for repeated training.

[0113] In the process of training the noxious weed identification model at the plot (or regional) scale and the regional scale, the original hyperspectral image is screened. During this screening process, the spectral features and texture features in the near-ground plot hyperspectral image are extracted.

[0114] The present application also provides an application scenario, including a poisonous weed identification link, a poisonous weed display link, and a poisonous weed control link. In the poisonous weed identification link, the poisonous weed identification method based on satellites, drones, and near-ground images provided in the present application is used to identify the current regional-scale satellite remote sensing image to obtain a regional-scale poisonous weed abundance identification result; in the poisonous weed display link, based on the poisonous weed abundance identification result obtained in the poisonous weed identification link, the distribution of poisonous weeds in the satellite remote sensing image is displayed on a handheld terminal device; in the poisonous weed control link, the operator carries out the poisonous weed control work in a targeted manner according to the distribution of poisonous weeds displayed on the handheld terminal device.

[0115] In an exemplary embodiment 3, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying poisonous weeds based on satellite, unmanned aerial vehicle and near-ground images is implemented.

[0116] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In an exemplary embodiment 4, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0118] In an exemplary embodiment 5, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0119] In an exemplary embodiment 6, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0121] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for identifying poisonous weeds based on satellite, UAV and near-ground images, characterized in that: The method for identifying poisonous weeds based on satellite, UAV and near-ground images includes: Obtain current regional-scale satellite remote sensing images of the target area; Inputting the current regional-scale satellite remote sensing image into a regional-scale poisonous weed identification model that matches the current regional-scale satellite remote sensing image to obtain a regional-scale poisonous weed abundance identification result; The process of determining the regional scale poisonous weed identification model is as follows: Acquire a historical plot-scale hyperspectral image of a target area; the historical plot-scale hyperspectral image is collected by an unmanned aerial vehicle device; Inputting the historical plot-scale hyperspectral image into a plot-scale noxious weed identification model that matches the historical plot-scale hyperspectral image to obtain a predicted plot-scale noxious weed abundance identification result; A regional scale poisonous weed identification network is trained based on a historical regional scale characteristic data set and predicted plot scale poisonous weed abundance identification results to obtain a regional scale poisonous weed identification model; the historical regional scale characteristic data set is a data set obtained by screening historical regional scale satellite remote sensing images based on historical plot scale characteristic variables; the historical plot scale characteristic variables are features obtained by processing historical near-ground hyperspectral images of the target area.

2. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 1, characterized in that: The training process of the plot-scale poisonous weed identification model is as follows: Construct a historical plot-scale feature dataset; Obtain the identification results of poisonous weeds at the historical sample scale; The historical plot-scale characteristic data set and the historical plot-scale poisonous weed identification results are used to train a plot-scale poisonous weed identification network to obtain a plot-scale poisonous weed identification model.

3. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 2, characterized in that: The process of determining the historical plot scale characteristic dataset is as follows: Based on the historical plot scale characteristic variables, multiple historical plot scale hyperspectral images are screened to screen out multiple historical plot scale hyperspectral images for identifying poisonous weeds; Based on a plurality of historical plot-scale hyperspectral images for identifying poisonous weeds, a historical plot-scale feature dataset is constructed.

4. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 1, characterized in that: The process of determining the historical regional scale feature dataset is as follows: Acquire multiple historical regional-scale satellite remote sensing images of the target area; Based on the historical sample scale characteristic variables, multiple historical regional scale satellite remote sensing images are screened to select multiple historical regional scale satellite remote sensing images for identifying poisonous weeds; Based on a plurality of historical regional-scale satellite remote sensing images for identifying poisonous weeds, a historical regional-scale feature dataset is constructed.

5. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 1, characterized in that: The regional scale poisonous weed identification network is a three-dimensional convolutional neural network.

6. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 5, characterized in that: The three-dimensional convolutional neural network includes X input layers, 3 convolutional layers, 3 maximum pooling layers, 3 fully connected layers and 1 output layer.

7. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 1, characterized in that: The historical near-ground hyperspectral image includes a plurality of sub-images; the determination process of the historical sample plot scale characteristic variables is as follows: For each sub-image, do the following: Processing the sub-image to obtain spectral features of poisonous weeds and spectral features of forage grass corresponding to the sub-image, and calculating a first difference based on the spectral features of the poisonous weeds and spectral features of forage grass corresponding to the sub-image; Based on the gray level co-occurrence matrix method, the sub-image is processed to obtain the texture features of the poisonous grass and the texture features of the forage grass corresponding to the sub-image, and the second difference is calculated based on the texture features of the poisonous grass and the texture features of the forage grass corresponding to the sub-image; According to the first difference or the second difference, it is determined whether the sub-image is an image with features for identifying poisonous weeds, and when the sub-image is an image with features for identifying poisonous weeds, the spectral features and texture features of poisonous weeds corresponding to the sub-image are determined as historical sample plot scale feature variables.

8. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 7, characterized in that: The spectral features include: band reflectance, first-order derivative spectrum calculated based on the reflectance image, and spectral index; The first difference includes: the difference between the reflectance of the poisonous grass band and the reflectance of the forage grass band, the difference between the first-order derivative spectrum of the poisonous grass and the first-order derivative spectrum of the forage grass, and the difference between the spectral index of the poisonous grass and the spectral index of the forage grass; The texture features include: gray value mean, gray value variance, image contrast, image entropy, image second-order moment, image correlation, image synergy and image heterogeneity; The second difference includes: the difference between the mean grayscale value of poisonous weeds and the mean grayscale value of forage grass, the difference between the grayscale value variance of poisonous weeds and the grayscale value variance of forage grass, the difference between the contrast of poisonous weed images and the contrast of forage grass images, the difference between the entropy of poisonous weed images and the entropy of forage grass images, the difference between the second-order moment of poisonous weed images and the second-order moment of forage grass images, the difference between the image correlation of poisonous weeds and the image correlation of forage grass, the difference between the synergy of poisonous weed images and the synergy of forage grass images, and the difference between the anisotropy of poisonous weed images and the anisotropy of forage grass images.

9. The method for identifying poisonous weeds based on satellite, UAV and near-ground images according to claim 7, characterized in that: The determining, according to the first difference or the second difference, whether the sub-image is an image having a feature for identifying poisonous weeds comprises: Using the Mahalanobis distance method, determining whether the first difference of each sub-image is greater than a first significance threshold, and if so, determining the sub-image as an image with a feature for identifying poisonous weeds; The Mahalanobis distance method is used to determine whether the second difference of each sub-image is greater than a second significance threshold. If so, the sub-image is determined as an image having a feature for identifying poisonous weeds.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for identifying poisonous weeds based on satellite, UAV, and near-ground images as described in any one of claims 1 to 9.

Images