A Crop Recognition Method and Device

Abstract

The present invention provides a crop recognition method and device, including: obtaining a plurality of agricultural machinery operation information and a target remote sensing image within a target plot; performing coordinate projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image; and determining a crop recognition result within the target plot according to the target remote sensing image and the matching relationship. The crop recognition method and device provided by the present invention utilize agricultural machinery equipped with a collection terminal to collect agricultural machinery operation information while operating, and perform coordinate projection and position matching on the agricultural machinery operation information and the remote sensing image within the plot, ultimately realizing the recognition of crops within the plot, greatly reducing the labor cost of remote sensing recognition. Therefore, it is suitable for large-scale crop recognition.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular, to a crop recognition method and device. Background Art

[0002] Crop recognition is the premise and foundation for crop area extraction, growth monitoring, yield estimation, and spatio-temporal distribution research. In particular, the spatio-temporal distribution of crops is related to agricultural machinery operation scheduling. Remote sensing data can quickly provide the areal characteristics of farmland crops. At present, using spaceborne or near-earth aerial remote sensing images for crop recognition and extraction is the main means for crop planting area recognition and area extraction.

[0003] Traditional remote sensing crop recognition mainly considers the spectral characteristics of crops and uses various vegetation indices to classify and recognize crops in specific regions; with the rise of artificial intelligence technology, machine learning methods such as maximum likelihood classification, random forest, and neural network have been applied to crop classification and recognition.

[0004] However, whether it is traditional remote sensing crop recognition or machine learning-based crop classification and recognition, it is necessary to invest a certain amount of human and economic resources to obtain sample data, manually annotate, and verify the results in the research area. It is only suitable for the recognition of specific crops in specific research areas and is not suitable for large-scale promotion and use. Summary of the Invention

[0005] In view of the problems existing in the prior art, embodiments of the present invention provide a crop recognition method and device.

[0006] The present invention provides a crop recognition method, including: obtaining a plurality of agricultural machinery operation information and a target remote sensing image within a target plot;

[0007] Performing coordinate system projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image;

[0008] Determining a crop recognition result of the target plot according to the target remote sensing image and the matching relationship.

[0009] According to the crop recognition method provided by the present invention, the agricultural machinery operation information includes: speed information, agricultural machinery status, position information, time information, model information, and image information; the agricultural machinery status includes an operation status and a non-operation status; each piece of agricultural machinery operation information is obtained by a piece of agricultural machinery collecting information at a collection point within the target plot; the obtaining of the plurality of agricultural machinery operation information and the target remote sensing image within the target plot includes:

[0010] Obtaining agricultural machinery operation information collected by a plurality of agricultural machinery operating within the target plot;

[0011] Obtain the initial remote sensing image of the target plot in the typical phenological period;

[0012] Perform calibration processing on the initial remote sensing image to obtain the target remote sensing image.

[0013] According to a crop recognition method provided by the present invention, the matching relationship is the pixel position relationship of the agricultural machinery operation information in the target remote sensing image; the step of projecting the multiple agricultural machinery operation information and the target remote sensing image into a coordinate system to obtain the matching relationship between the multiple agricultural machinery operation information and the target remote sensing image includes:

[0014] Determine the valid operation data among the multiple agricultural machinery operation information, and use the set composed of all the valid operation data as the valid data set; the agricultural machinery state in the valid operation data is the operation state;

[0015] Determine the collection points of each agricultural machinery operation information in the valid data set;

[0016] Project all the collection points and the target remote sensing image into the same coordinate system to obtain the pixels where the collection points of each agricultural machinery operation information are located in the target remote sensing image.

[0017] According to a crop recognition method provided by the present invention, the step of determining the crop recognition result of the target plot based on the target remote sensing image and the matching relationship includes:

[0018] Determine the vegetation index of each pixel in the target remote sensing image; the vegetation index includes the normalized difference vegetation index and the ratio vegetation index;

[0019] And determine the pixels where each valid operation data in the valid data set is located in the target remote sensing image according to the matching relationship;

[0020] Divide the valid data set into a training set and a validation set;

[0021] Based on the training set, remove the pixels with the valid operation data in the training set from the target remote sensing image to obtain a screened remote sensing image;

[0022] Determine the filtered pixels according to the normalized difference vegetation index of each pixel in the screened remote sensing image;

[0023] Obtain a filtered remote sensing image according to all the filtered pixels;

[0024] Determine the crop recognition result of the target plot according to the training set and the filtered remote sensing image.

[0025] A crop recognition method provided by the present invention, determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image, includes:

[0026] Determine the pixels where all valid operation data in the training set are located as training set pixels;

[0027] According to the normalized difference vegetation index of each training set pixel, determine the first mean and the first variance of the normalized difference vegetation index;

[0028] According to the ratio vegetation index of all pixels in the training set, determine the second mean and the second variance of the ratio vegetation index;

[0029] According to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image, determine the crop recognition result of the target plot.

[0030] A crop recognition method provided by the present invention, determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image, includes:

[0031] In the filtered remote sensing image, determine any pixel as a target pixel;

[0032] According to the first mean, the first variance and the normalized difference vegetation index of the target pixel, determine the first similarity of the target pixel;

[0033] According to the second mean, the second variance and the ratio vegetation index of the target pixel, determine the second similarity of the target pixel;

[0034] Use the first similarity and the second similarity to determine the connectivity of the target pixel;

[0035] According to the first similarity, the second similarity and the connectivity, determine the recognition probability of the crop in the target pixel;

[0036] According to the recognition probability, determine the crop category in the plot corresponding to the target pixel;

[0037] Obtain the crop categories corresponding to all pixels, and determine the crop recognition result of the target plot.

[0038] A crop recognition method provided by the present invention, the agricultural machinery operation information further includes an operation type. After obtaining a plurality of agricultural machinery operation information and a target remote sensing image in the target plot, it further includes determining the operation object corresponding to each agricultural machinery operation information. Determining the operation object corresponding to each agricultural machinery operation information includes:

[0039] Match the code corresponding to the operation type according to the operation type.

[0040] Determine the operation object corresponding to the operation type according to the code; or,

[0041] Determine the operation object corresponding to the model information according to the model information; or,

[0042] Identify the image information to obtain the operation object corresponding to the image information.

[0043] According to a crop identification device provided by the present invention, comprising:

[0044] An acquisition unit for acquiring a plurality of agricultural machinery operation information and a target remote sensing image in a target plot;

[0045] A projection unit for performing coordinate projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image;

[0046] A determination unit for determining a crop identification result of the target plot according to the target remote sensing image and the matching relationship.

[0047] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned crop identification methods are implemented.

[0048] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned crop identification methods are implemented.

[0049] The crop identification method and device provided by the present invention utilize an agricultural machinery equipped with a collection terminal to collect agricultural machinery operation information while operating, and perform coordinate projection and position matching on the agricultural machinery operation information and the remote sensing image in the plot, and finally realize the identification of crops in the plot, greatly reducing the labor cost of remote sensing identification. Therefore, it is suitable for large-scale crop identification. Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0051] Figure 1It is one of the schematic flowcharts of the crop recognition method provided by the present invention;

[0052] Figure 2 It is another schematic flowchart of the crop recognition method provided by the present invention;

[0053] Figure 3 It is the schematic structural diagram of the crop recognition device provided by the present invention;

[0054] Figure 4 It is the schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0055] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, the element defined by the phrase "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] Traditional remote sensing crop recognition mainly considers the spectral characteristics of crops, studies the vegetation types in a specific area, constructs crop time series index characteristics in combination with crop phenological periods, time series remote sensing images, etc., to characterize the characteristics of crops growing, maturing and being harvested during the growth period, and uses various vegetation indices to classify and identify crops in a specific area. However, there is a certain similarity in the spectra between different crops, and the differences in vegetation indices are relatively small, making it difficult to effectively distinguish them.

[0058] The classification and recognition method implemented by machine learning methods depends on the machine learning model algorithm itself. The specific classification and recognition process in the middle is invisible, and it is impossible to well explain the principle and process of model analysis, making it difficult to make targeted improvements to the classification and recognition process.

[0059] With the rapid development of "Internet + agricultural machinery operation", the promotion and application of remote monitoring systems for agricultural machinery operations have gradually started in China. Currently, more than 200,000 agricultural machines have been incorporated into the agricultural machinery supervision platform, covering the entire process of farming, cultivation, management, and harvesting. They are distributed across the country and continuously collect agricultural machinery operation information.

[0060] In an embodiment of the present invention, a method and system for crop recognition with air-ground collaboration are provided. By using the operation trajectories, operation information, and pre-mounted image information of agricultural machinery across different operation links and within the same operation link, specific crop samples are automatically labeled through information such as agricultural machinery models and operation types, and multi-temporal remote sensing image sequence data is automatically matched. Combining with the phenological periods of crops, the recognition and extraction of specific crops are realized, laying a foundation for the popularization of large-scale crop automatic recognition technology through fast, inexpensive, and high-resolution spatial data.

[0061] Among them, the cross-operation link refers to the link where agricultural machinery operates during different phenological periods of crops, and the same-operation link refers to the link where agricultural machinery operates during the same phenological period.

[0062] Next, in combination with Figures 1 to 4 Describe the crop recognition method and device provided by the embodiments of the present invention.

[0063] Figure 1 It is one of the flow diagrams of the crop recognition method provided by the present invention. As Figure 1 shown, it includes but is not limited to the following steps:

[0064] First, in step S1, obtain multiple agricultural machinery operation information and target remote sensing images within the target plot.

[0065] Since each agricultural machine is equipped with a collection terminal, the collection terminal includes: a positioning module, an in-vehicle pre-mounted camera module, and a transmission module.

[0066] When the agricultural machinery operates within the target plot, the collection terminal collects agricultural machinery operation information at a certain frequency. The position where the agricultural machinery is located during the collection of agricultural machinery operation information is the collection point. Therefore, according to the time information, position information, and agricultural machinery code in the agricultural machinery operation information, the operation trajectory of the agricultural machinery can be obtained.

[0067] Retrieve the target remote sensing images of crops in typical phenological periods within the target plot. The typical phenological periods may include tasseling, maturity, and harvest.

[0068] Furthermore, in step S2, project the multiple agricultural machinery operation information and the target remote sensing images into a coordinate system to obtain the matching relationship between the multiple agricultural machinery operation information and the target remote sensing images.

[0069] Among them, the coordinate system projection can be implemented using the Universal Transverse Mercator Projection (UTM).

[0070] Specifically, by performing coordinate system projection on the collection point coordinates corresponding to each agricultural machinery operation information within the target plot and the target remote sensing image, the matching relationship between each agricultural machinery operation information and the position of the target remote sensing image in the same coordinate system can be obtained. The collection points of the agricultural machinery operation information can be distributed on the target remote sensing image.

[0071] Furthermore, in step S3, according to the target remote sensing image and the matching relationship, the crop recognition result of the target plot is determined.

[0072] Specifically, according to the matching relationship between the agricultural machinery operation information and the target remote sensing image, the pixel where the agricultural machinery operation information is located in the target remote sensing image can be determined. It can be considered that there is only one type of crop in each pixel. Thus, the crop type within the pixel area with the collection point of the agricultural machinery operation information can be determined.

[0073] The crop recognition method provided by the present invention utilizes agricultural machinery equipped with a collection terminal to collect agricultural machinery operation information while operating, and performs coordinate projection and position matching on the agricultural machinery operation information and the remote sensing image within the plot, ultimately realizing the recognition of crops within the plot, greatly reducing the labor cost of remote sensing recognition. Therefore, it is suitable for large-scale crop recognition.

[0074] Optionally, the agricultural machinery operation information includes: speed information, agricultural machinery status, position information, time information, model information, and image information; the agricultural machinery status includes an operating status and a non-operating status; each agricultural machinery operation information is obtained by a piece of agricultural machinery collecting information at a collection point within the target plot; the obtaining of multiple agricultural machinery operation information and the target remote sensing image within the target plot includes:

[0075] Obtaining the agricultural machinery operation information collected by multiple agricultural machinery operating within the target plot;

[0076] Obtaining the initial remote sensing image of the target plot during the typical phenological period;

[0077] Performing calibration processing on the initial remote sensing image to obtain the target remote sensing image.

[0078] During the process of agricultural machinery operating within the target plot, the collection terminal collects the agricultural machinery operation information at a certain frequency. When the collection time point is reached, the location where the agricultural machinery is located is the collection point. At this time, the collection terminal of the agricultural machinery can collect the agricultural machinery operation information at this collection point.

[0079] Among them, the agricultural machinery operation information includes: speed information, agricultural machinery status, location information, time information, model information, image information, and may also include the vehicle code of the agricultural machinery; the agricultural machinery status includes: operation status and non-operation status; the location information is the longitude and latitude of the collection point corresponding to the agricultural machinery operation information; the time information is the collection time of the agricultural machinery information; the model information includes agricultural machinery model information and implement model information.

[0080] The location information, time information, agricultural machinery status, and image information can be Global Navigation Satellite System (GNSS) data, and the time information can be GNSS time. Among them, the image information can be collected by the front camera of the agricultural machinery.

[0081] The collection terminal wirelessly transmits the collected agricultural machinery operation information to the data server; in the case where crop recognition needs to be performed on the target plot, the agricultural machinery operation information within the target plot is retrieved from the data server.

[0082] Retrieve the initial remote sensing image of the crop in the target plot at the typical phenological period, and perform radiometric correction, geometric correction, and atmospheric correction on the initial remote sensing image to obtain the corrected target remote sensing image

[0083] Optionally, the agricultural machinery operation information further includes the operation type. After obtaining the multiple agricultural machinery operation information and the target remote sensing image within the target plot, it further includes determining the operation object corresponding to each agricultural machinery operation information. The determining of the operation object corresponding to each agricultural machinery operation information includes:

[0084] According to the operation type, match the code corresponding to the operation type;

[0085] According to the code, determine the operation object corresponding to the operation type; or,

[0086] According to the model information, determine the operation object corresponding to the model information; or,

[0087] Perform recognition on the image information to obtain the operation object corresponding to the image information.

[0088] Specifically, the matching method between the agricultural machinery operation information and the crop type can be determined by any one of the following three methods:

[0089] Method 1: The industry standard "TCAMA 33-2020 Technical Specification for Data Exchange of Agricultural Machinery Operation Remote Monitoring System Platform" encodes the uploaded operation type and crops. According to the operation type in the agricultural machinery operation information, the code can be determined, and then the crop type of the operation object can be determined through the code.

[0090] Method 2: Using the collected model information, including the agricultural machinery model and implement model, according to the industry standards "JB / T8574-2013 Compilation Rules for Agricultural Machinery Product Models" and "NY / T1640-2015 Classification of Agricultural Machinery", the corresponding crop type of the operation can be determined. For example, the implement 2BYCF-3 represents a corn planter, so when this implement is suspended, it means the operation object is corn; if the major category of agricultural machinery is 0402, it represents a corn harvester, indicating that the crop type of the operation object of the agricultural machinery is corn.

[0091] Method 3: Using the on-vehicle terminal of agricultural machinery to collect image information, and determining the crop type of the operation object of the agricultural machinery through machine learning or remote manual visual assistance interpretation.

[0092] During a crop growth cycle, if a crop is deduced and matched through agricultural machinery operation information at any time in a farmland plot, the farmland plot is marked as that crop, and spatially it can be spatially matched with the remote sensing image during the crop growth cycle. For example, if a plot is detected with agricultural machinery corn seeding operation, then the plot is identified as growing corn, and the corresponding position of the remote sensing image collected during the corn growth cycle of this plot should also be marked as corn.

[0093] Take the set of all agricultural machinery operation information in the target plot as the initial data set P, and sort it according to the time information in each agricultural machinery operation information in chronological order:

[0094] P = {P1, P2, P3, …, P i , …};

[0095] where Pi i is the agricultural machinery operation information at the i-th collection point, and Pi i should at least contain the following data:

[0096] Pi i = {d, t, lon, lat, k, v, c};

[0097] where d is the vehicle code of the agricultural machinery; t is the time information; lon is the longitude; lat is the latitude; v is the speed information; k is the agricultural machinery status; c is the crop type.

[0098] Optionally, the matching relationship is the pixel position relationship of the agricultural machinery operation information in the target remote sensing image; the process of performing coordinate system projection on the multiple agricultural machinery operation information and the target remote sensing image to obtain the matching relationship between the multiple agricultural machinery operation information and the target remote sensing image includes:

[0099] Determine the valid operation data among the multiple agricultural machinery operation information, and take the set composed of all the valid operation data as the valid data set; the agricultural machinery status in the valid operation data is the operation status;

[0100] Determine the collection points of each agricultural machinery operation information in the effective data set;

[0101] Project all the collection points and the target remote sensing image into the same coordinate system, and obtain the pixels where the collection points of each agricultural machinery operation information are located in the target remote sensing image.

[0102] Screen out the effective data set Q with the agricultural machinery status being the operation status from the initial data set P, specifically:

[0103] Q = {P | P(k = operation status)};

[0104] where k is the agricultural machinery status.

[0105] For the convenience of calculation, use the UTM projection to convert the agricultural machinery operation information and the target remote sensing image into the same plane coordinate, and match the position in the agricultural machinery operation information to the corresponding pixel in the target remote sensing image. Each pixel of the remote sensing image after projection is L×L meters, and there are a total of m×n pixels.

[0106] For each record Q in the effective data set Q i , its plane coordinate is Q i (ix, iy), and the row and column numbers in the matching target remote sensing image F can be calculated by the following formula:

[0107] a = floor((ix – x0) / L);

[0108] b = floor((iy – y0) / L);

[0109] where b represents the column number of the matched pixel of Q i ; a represents the row number of the matched pixel of Q i ; ix is the abscissa of Q i ; iy is the ordinate of Q i ; x0 is the abscissa of the plane coordinate of the pixel at the lower left corner of the remote sensing image; y0 is the ordinate of the plane coordinate of the pixel at the lower left corner of the remote sensing image; L is the pixel size in the target remote sensing image; floor represents rounding down.

[0110] The matching relationship between the agricultural machinery operation position Q and the pixels in the target remote sensing image F is as follows:

[0111] Q i (ix, iy) = {F | F(a, b)};

[0112] where Q i (ix, iy) is the i-th piece of information in the effective data set Q; F(a, b) is the pixel at the a-th row and b-th column in the target remote sensing image F.

[0113] Optionally, determining the crop recognition result of the target plot according to the target remote sensing image and the matching relationship includes:

[0114] Determining the vegetation index of each pixel in the target remote sensing image; the vegetation index includes the normalized difference vegetation index and the ratio vegetation index;

[0115] And determining the pixel in the target remote sensing image where each valid operation data in the valid data set is located according to the matching relationship;

[0116] Dividing the valid data set into a training set and a validation set;

[0117] Based on the training set, removing the pixels with the valid operation data in the training set from the target remote sensing image to obtain a screened remote sensing image;

[0118] Determining filtered pixels according to the normalized difference vegetation index of each pixel in the screened remote sensing image;

[0119] Obtaining a filtered remote sensing image according to all the filtered pixels;

[0120] Determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image.

[0121] Analysis of common vegetation indices. The normalized difference vegetation index (NDVI) is an important indicator for detecting vegetation growth status and vegetation coverage, while the variance and standard deviation of the ratio vegetation index (RVI) are the largest, and it has a greater ability to distinguish crops. Therefore, these two indices are selected for crop recognition. It is defaulted that there is only one type of crop in the plot corresponding to each pixel. The calculation formulas for the normalized difference vegetation index and the ratio vegetation index of the target remote sensing image F are as follows:

[0122]

[0123]

[0124] Among them, C NDVI (a, b) represents the value of the normalized difference vegetation index of the pixel with row number a and column number b in the target remote sensing image F; C RVI (a, b) represents the value of the ratio vegetation index of the pixel with row number a and column number b in the target remote sensing image F; C NIR (a, b) represents the reflectance of the near-infrared band of the pixel with row number a and column number b in the target remote sensing image F; C RED (a, b) represents the reflectance of the red light band of the pixel with row number a and column number b in the target remote sensing image F.

[0125] The obtained matching data set Q is divided and labeled as a training set and a validation set according to a ratio of 8:2.

[0126] For the target remote sensing image F, according to the matching relationship, after removing the pixels where the collection points of the agricultural machinery operation information in the training set are located, the remaining pixels form the filtered remote sensing image.

[0127] Use NDVI to extract vegetation, which provides a preliminary filter for crop extraction. The NDVI value of the pixel is between -1 and 1. Among them, if the NDVI value is negative, it means that the ground cover is cloud, water, snow, etc.; if the NDVI value is 0, it means rock or bare soil, etc.; if the NDVI value is positive, it means there is vegetation cover, and it increases with the increase of the coverage.

[0128] Therefore, the pixels with NDVI values not greater than 0 are used as non-vegetation pixels, and the non-vegetation pixels are filtered out to obtain all filtered pixels with positive NDVI values.

[0129] Optionally, determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image includes:

[0130] Determine the pixels where all the valid operation data in the training set are located as the training set pixels;

[0131] According to the normalized vegetation index of each training set pixel, determine the first mean and the first variance of the normalized vegetation index;

[0132] According to the ratio vegetation index of all the pixels in the training set, determine the second mean and the second variance of the ratio vegetation index;

[0133] According to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image, determine the crop recognition result of the target plot.

[0134] For the pixels where the collection points of the agricultural machinery operation information in the training set are located, calculate the first mean and the first variance NDVI SD , and calculate the second mean and the second variance RVI SD of the ratio vegetation index RVI respectively.

[0135] Optionally, determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image includes:

[0136] In the filtered remote sensing image, determine any pixel as the target pixel;

[0137] Determine the first similarity of the target pixel according to the first mean, the first variance, and the normalized difference vegetation index of the target pixel;

[0138] Determine the second similarity of the target pixel according to the second mean, the second variance, and the ratio vegetation index of the target pixel;

[0139] Determine the connectivity of the target pixel by using the first similarity and the second similarity;

[0140] Determine the recognition probability of the crop in the target pixel according to the first similarity, the second similarity, and the connectivity;

[0141] Determine the crop category in the plot corresponding to the target pixel according to the recognition probability;

[0142] Obtain the crop categories corresponding to all pixels, and determine the crop recognition result of the target plot.

[0143] Specifically, for each pixel in the filtered remote sensing image, calculate the first similarity of the pixel, which represents the degree of similarity to the average NDVI of the training set. The calculation formula is as follows:

[0144]

[0145] Calculate the second similarity of the pixel, which represents the degree of similarity to the average RVA of the training set. The calculation formula is as follows:

[0146]

[0147] Among them, since the crop is a planar ground object, when identifying the crop, to remove the "salt and pepper effect" of remote sensing classification, the situation of adjacent pixels needs to be considered. If the adjacent pixel is the matching point of the filtered agricultural machinery operation, then the adjacent pixel P RVI is assigned as If the adjacent pixel is the non-vegetation image pixel that has been filtered out, then the adjacent pixel P RVI is assigned zero. By calculating the mean value of the adjacent pixels (up, down, left, and right) P RVI around the pixel, the connectivity P Neighbor (a, b) is obtained:

[0148]

[0149] Among them, P NDVI (a, b), P RVI (a, b), P Neighbor(a, b) are influencing factors that jointly affect crop determination. By assigning different weight coefficients to the three influencing factors, the crop determination probability P(a, b) of the pixel is obtained. The calculation formula is as follows:

[0150] P(a, b) = P NDVI (a, b) × w1 + P RVI × w2 + P Neighbor × w3;

[0151] Among them, w1 is the first weight, 0 < w1 < 1, w2 is the second weight, 0 < w2 < 1, w3 is the third weight, 0 < w3 < 1; and w1 + w2 + w3 = 1.

[0152] Given a threshold z, if the probability of a specified pixel point being a specified crop is greater than or equal to z, it is recognized and extracted as a crop. The calculation formula is as follows:

[0153]

[0154] Finally, the agricultural machinery operation information in the validation set can be used to obtain the predicted crop category by calculating the recognition probability. According to the predicted crop category and the actual crop category, the confusion matrix is calculated to evaluate the accuracy of the recognition result of the specified crop.

[0155] Figure 2 is the second flow schematic diagram of the crop recognition method provided by the present invention. As Figure 2 shown, during the operation of the agricultural machinery in the target plot, the collection terminal collects the agricultural machinery operation information at the collection point. The agricultural machinery operation information can include GNSS data, model information, vehicle coding, and speed information, and uploads the agricultural machinery operation information to the on-vehicle terminal of the agricultural machinery. Then, the on-vehicle terminal of the agricultural machinery transmits the agricultural machinery operation information to the agricultural machinery operation data center through the 2nd Generation Mobile Communication System (2G), 3rd Generation (3G), 4th Generation mobile communication technology (4G), or 5th Generation mobile networks (5G) wireless network. In the case of needing to identify the crops in the target plot, determine the crop objects corresponding to the agricultural machinery operation information in the target plot, and obtain the crop types at the collection points of the agricultural machinery operation information. Among them, the GNSS data includes: location information, time information, agricultural machinery status, and image information.

[0156] On the other hand, during the typical phenological periods of crops, such as the periods of tasseling, maturity or harvest, target remote sensing images of the target plot during the typical phenological periods are acquired, and the vegetation indices of each pixel in the target remote sensing images are calculated. The vegetation indices include NDVI and RVI.

[0157] All the collection points are projected and matched with the target remote sensing image in the air and on the ground, projected into the same coordinate system, and the pixels where the collection points of each agricultural machinery operation information are located in the target remote sensing image are obtained.

[0158] Automatically label the crop types for each pixel containing a collection point, and divide all the agricultural machinery operation information into a training set and a validation set.

[0159] Filter the pixels in the target remote sensing image that contain the collection points of the training set to obtain a filtered remote sensing image.

[0160] Filter the pixels in the filtered remote sensing image whose normalized vegetation index is not positive, and determine the filtered pixels whose normalized vegetation index is positive.

[0161] Calculate the recognition probability of the crops for all the filtered pixels, obtain the recognition result, and evaluate the accuracy of the recognition result using the validation set.

[0162] The crop recognition method provided by the present invention automatically labels the target remote sensing images at the key growth periods through agricultural machinery operation information, comprehensively utilizes influencing factors such as the NDVI similarity, RVI similarity and pixel connectivity of pixels, realizes the recognition of specified crops, and reduces the labor cost of remote sensing recognition.

[0163] Figure 3 It is a schematic structural diagram of the crop recognition device provided by the present invention, as Figure 3 shown, including:

[0164] An acquisition unit 301, configured to acquire a plurality of agricultural machinery operation information and a target remote sensing image in a target plot;

[0165] A projection unit 302, configured to project the coordinate systems of the plurality of agricultural machinery operation information and the target remote sensing image, and obtain the matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image;

[0166] A determination unit 303, configured to determine the crop recognition result of the target plot according to the target remote sensing image and the matching relationship.

[0167] First, the acquisition unit 301 acquires a plurality of agricultural machinery operation information and a target remote sensing image in a target plot.

[0168] Since each agricultural machine is equipped with a collection terminal, the collection terminal includes: a positioning module, a vehicle-mounted front camera module and a transmission module.

[0169] When the agricultural machine operates in the target plot, the acquisition terminal will collect agricultural machine operation information at a certain frequency. The position where the agricultural machine is located during the collection of agricultural machine operation information is the collection point. Therefore, according to the time information, position information, and agricultural machine code in the agricultural machine operation information, the operation trajectory of the agricultural machine can be obtained.

[0170] Retrieve the target remote sensing images of the crops in the target plot at typical phenological stages. The typical phenological stages can include tasseling, maturity, and harvest.

[0171] Furthermore, the projection unit 302 projects the multiple agricultural machine operation information and the target remote sensing images in a coordinate system to obtain the matching relationship between the multiple agricultural machine operation information and the target remote sensing images.

[0172] Among them, the coordinate system projection can adopt the Universal Transverse Mercator Projection (UTM).

[0173] Specifically, by projecting the collection point coordinates corresponding to each agricultural machine operation information in the target plot and the target remote sensing images in a coordinate system, the matching relationship between each agricultural machine operation information and the position of the target remote sensing image in the same coordinate system can be obtained. The collection points of the agricultural machine operation information can be distributed on the target remote sensing images.

[0174] Furthermore, the determination unit 303 determines the crop recognition result of the target plot according to the target remote sensing images and the matching relationship.

[0175] Specifically, according to the matching relationship between the agricultural machine operation information and the target remote sensing images, the pixel where the agricultural machine operation information is located in the target remote sensing image can be determined. It can be considered that there is only one type of crop in each pixel. Therefore, the crop type in the pixel area with the collection point of the agricultural machine operation information can be determined.

[0176] The crop recognition device provided by the present invention utilizes the agricultural machine equipped with the acquisition terminal to collect agricultural machine operation information while operating, and projects the coordinates and matches the positions of the agricultural machine operation information with the remote sensing images in the plot, ultimately realizing the recognition of the crops in the plot, greatly reducing the labor cost of remote sensing recognition. Therefore, it is suitable for large-scale crop recognition.

[0177] It should be noted that the crop recognition device provided in the embodiments of the present invention can be implemented based on the crop recognition method described in any of the above embodiments when specifically executed, and this embodiment will not be elaborated herein.

[0178] Figure 4 is the structural schematic diagram of the electronic device provided by the present invention, as Figure 4As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 complete communication with each other through the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute a crop recognition method, which includes: obtaining a plurality of agricultural machinery operation information and a target remote sensing image within a target plot; performing coordinate system projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image; and determining a crop recognition result within the target plot according to the target remote sensing image and the matching relationship.

[0179] In addition, when the logic instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0180] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the crop recognition method provided by the above-mentioned various methods. The method includes: obtaining a plurality of agricultural machinery operation information and a target remote sensing image within a target plot; performing coordinate system projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image; and determining a crop recognition result within the target plot according to the target remote sensing image and the matching relationship.

[0181] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the crop recognition method provided in the above embodiments. The method includes: obtaining a plurality of agricultural machinery operation information and a target remote sensing image within a target plot; performing coordinate system projection on the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image; and determining a crop recognition result within the target plot according to the target remote sensing image and the matching relationship.

[0182] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0183] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A crop recognition method, characterized in that, Including: Obtaining a plurality of agricultural machinery operation information and target remote sensing images within a target plot; Projecting the coordinates of the plurality of agricultural machinery operation information and the target remote sensing image to obtain a matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image; Determining a crop recognition result of the target plot according to the target remote sensing image and the matching relationship; Wherein, the matching relationship is the pixel position relationship of the agricultural machinery operation information in the target remote sensing image; the projecting the coordinates of the plurality of agricultural machinery operation information and the target remote sensing image to obtain the matching relationship between the plurality of agricultural machinery operation information and the target remote sensing image includes: determining valid operation data among the plurality of agricultural machinery operation information, and using the set composed of all the valid operation data as a valid data set; the agricultural machinery state in the valid operation data is the operation state; determining the collection points of each agricultural machinery operation information in the valid data set; projecting all the collection points and the target remote sensing image into the same coordinate system to obtain the pixels where the collection points of each agricultural machinery operation information are located in the target remote sensing image; The determining the crop recognition result of the target plot according to the target remote sensing image and the matching relationship includes: determining the vegetation index of each pixel in the target remote sensing image; the vegetation index includes the normalized difference vegetation index and the ratio vegetation index; and according to the matching relationship, determining the pixels where each valid operation data in the valid data set is located in the target remote sensing image; dividing the valid data set into a training set and a validation set; based on the training set, removing the pixels with the valid operation data in the training set from the target remote sensing image to obtain a filtered remote sensing image; determining filtered pixels according to the normalized difference vegetation index of each pixel in the filtered remote sensing image; obtaining a filtered remote sensing image according to all the filtered pixels; determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image; The determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image includes: determining the pixels where all the valid operation data in the training set are located as training set pixels; determining the first mean and the first variance of the normalized difference vegetation index according to the normalized difference vegetation index of each training set pixel; determining the second mean and the second variance of the ratio vegetation index according to the ratio vegetation index of all the pixels in the training set; determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image; Determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance, and the filtered remote sensing image includes: determining any pixel as a target pixel in the filtered remote sensing image; determining the first similarity of the target pixel according to the first mean, the first variance, and the normalized difference vegetation index of the target pixel; determining the second similarity of the target pixel according to the second mean, the second variance, and the ratio vegetation index of the target pixel; determining the connectivity of the target pixel by using the first similarity and the second similarity; determining the recognition probability of the crop within the target pixel according to the first similarity, the second similarity, and the connectivity; determining the crop category within the plot corresponding to the target pixel according to the recognition probability; obtaining the crop categories corresponding to all pixels, and determining the crop recognition result of the target plot; Wherein, the agricultural machinery operation information includes: speed information, agricultural machinery status, position information, time information, model information, and image information; the agricultural machinery status includes an operation status and a non-operation status; The first similarity is calculated as follows: ； The second similarity is calculated as follows: ； The connectivity is calculated as follows: ； Among them, represents the value of the normalized difference vegetation index of the pixel with row number a and column number b in the target remote sensing image F; represents the value of the ratio vegetation index of the pixel with row number a and column number b in the target remote sensing image F; and are respectively the first mean value and the first variance of the normalized difference vegetation index NDVI; and are respectively the second mean value and the second variance of the ratio vegetation index RVI; , , are influencing factors that jointly affect crop determination; there are a total of m×n pixels.

2. The crop recognition method according to claim 1, wherein Each piece of agricultural machinery operation information is obtained by a piece of agricultural machinery collecting information at a collection point within the target plot; The obtaining of multiple pieces of agricultural machinery operation information and the target remote sensing image within the target plot includes: Obtaining the agricultural machinery operation information collected by multiple pieces of agricultural machinery operating within the target plot; Obtaining the initial remote sensing image of the target plot during the typical phenological period; Performing calibration processing on the initial remote sensing image to obtain the target remote sensing image.

3. The crop recognition method according to claim 2, characterized in that The agricultural machinery operation information further includes an operation type. After obtaining multiple pieces of agricultural machinery operation information and the target remote sensing image within the target plot, it further includes determining the operation object corresponding to each piece of agricultural machinery operation information. The determining of the operation object corresponding to each piece of agricultural machinery operation information includes: Matching the code corresponding to the operation type according to the operation type; Determining the operation object corresponding to the operation type according to the code; or, Determining the operation object corresponding to the model information according to the model information; or, Identifying the operation object corresponding to the image information by identifying the image information.

4. A crop recognition device, characterized in that, It includes: An obtaining unit, configured to obtain multiple pieces of agricultural machinery operation information and the target remote sensing image within the target plot; A projection unit, configured to project the multiple pieces of agricultural machinery operation information and the target remote sensing image in a coordinate system to obtain the matching relationship between the multiple pieces of agricultural machinery operation information and the target remote sensing image; A determining unit, configured to determine the crop recognition result of the target plot according to the target remote sensing image and the matching relationship; Among them, the matching relationship is the pixel position relationship of the agricultural machinery operation information in the target remote sensing image; the projection unit is specifically used for: determining valid operation data among the multiple agricultural machinery operation information, and taking the set composed of all the valid operation data as the valid data set; the agricultural machinery state in the valid operation data is the operation state; determining the collection points of each agricultural machinery operation information in the valid data set; projecting all the collection points and the target remote sensing image into the same coordinate system, and obtaining the pixels where the collection points of each agricultural machinery operation information are located in the target remote sensing image; The determination unit is specifically used for: determining the vegetation index of each pixel in the target remote sensing image; the vegetation index includes the normalized difference vegetation index and the ratio vegetation index; and according to the matching relationship, determining the pixels where each valid operation data in the valid data set is located in the target remote sensing image; dividing the valid data set into a training set and a validation set; based on the training set, removing the pixels with the valid operation data in the training set in the target remote sensing image, and obtaining a screened remote sensing image; determining filtered pixels according to the normalized difference vegetation index of each pixel in the screened remote sensing image; obtaining a filtered remote sensing image according to all the filtered pixels; determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image; Determining the crop recognition result of the target plot according to the training set and the filtered remote sensing image includes: determining the pixels where all the valid operation data in the training set are located as the training set pixels; determining the first mean and the first variance of the normalized difference vegetation index according to the normalized difference vegetation index of each training set pixel; determining the second mean and the second variance of the ratio vegetation index according to the ratio vegetation index of all the pixels in the training set; determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image; Determining the crop recognition result of the target plot according to the first mean, the first variance, the second mean, the second variance and the filtered remote sensing image includes: determining any pixel as the target pixel in the filtered remote sensing image; determining the first similarity of the target pixel according to the first mean, the first variance and the normalized difference vegetation index of the target pixel; determining the second similarity of the target pixel according to the second mean, the second variance and the ratio vegetation index of the target pixel; determining the connectivity of the target pixel by using the first similarity and the second similarity; determining the recognition probability of the crop in the target pixel according to the first similarity, the second similarity and the connectivity; determining the crop category in the plot corresponding to the target pixel according to the recognition probability; obtaining the crop categories corresponding to all the pixels, and determining the crop recognition result of the target plot; Among them, the agricultural machinery operation information includes: speed information, agricultural machinery state, position information, time information, model information and image information; the agricultural machinery state includes the operation state and the non-operation state; The first similarity is calculated as follows: ； The second similarity is calculated as follows: ； The connectivity is calculated as follows: ； Among them, represents the value of the normalized difference vegetation index of the pixel with row number a and column number b in the target remote sensing image F; represents the value of the ratio vegetation index of the pixel with row number a and column number b in the target remote sensing image F; and are respectively the first mean value and the first variance of the normalized difference vegetation index NDVI; and are respectively the second mean value and the second variance of the ratio vegetation index RVI; , , are influencing factors that jointly affect crop determination; there are a total of m×n pixels.

5. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the crop recognition method according to any one of claims 1 to 3 are implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the crop recognition method according to any one of claims 1 to 3 are implemented.

Images