Large-area tobacco field yield prediction method, device, equipment and storage medium

Abstract

The present invention relates to the technical field of crop yield estimation, and discloses a method, device, equipment and storage medium for estimating the yield of a large-area tobacco field. The method includes: respectively inputting a tobacco field sample data set and a tobacco plant sample data set into a camouflaged target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model; acquiring the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the unmanned aerial vehicle; using the tobacco field recognition model and the tobacco plant recognition model to identify the orthophoto image of tobacco leaves, constructing a tobacco leaf yield estimation model, and using the tobacco leaf yield estimation model to estimate the yield of the tobacco field in the area to be estimated. By respectively performing tobacco field recognition and tobacco plant recognition on the area to be estimated, and considering the tobacco field area and tobacco plant data determined by the tobacco field edge pixel points and tobacco plant candidate boxes, the present invention estimates the yield of the tobacco field in the area to be estimated, and has higher accuracy and scene adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of crop yield prediction, and particularly to a method, device, equipment and storage medium for predicting the yield of large-area tobacco fields. Background Art

[0002] Crop yield prediction is an important agricultural intelligence indispensable for the country to formulate agricultural policies. Timely and accurate prediction can provide effective support for agricultural experience management and is also an urgent need for the development of precision agriculture. The tobacco industry plays a very important role in the process of national economic and social development. The yield of tobacco affects the price of tobacco, and then affects the income of tobacco farmers. Accurate yield estimation of tobacco is of great significance for the pricing and overall distribution of the tobacco industry.

[0003] Traditional tobacco yield estimation is mostly carried out by means of field investigation and sampling. By selecting representative tobacco planting sample areas, the tobacco leaves in the sample areas are harvested and baked to obtain the tobacco leaf yield in the tobacco area, and then the tobacco leaf yield in a larger area is extrapolated. The traditional tobacco yield estimation method is not only time-consuming, laborious and difficult to carry out on a large scale, but also the yield data obtained by the point-covering method has low accuracy.

[0004] Remote sensing technology, with its advantages of being macroscopic, dynamic, fast, accurate, etc., is widely used in crop yield estimation. However, traditional satellite remote sensing yield estimation is applicable to large-scale crops. For small-scale crop yield estimation, due to its long repetition period, low temporal and spatial resolution, and the influence of the atmospheric environment, it is impossible to obtain accurate ground information in a timely and effective manner. UAV remote sensing has the characteristics of low cost, high spatial resolution, real-time performance and little influence by the atmosphere, providing a new technical means for remote sensing of crop yield estimation at a small scale.

[0005] In existing UAV remote sensing tobacco yield estimation, hyperspectral data is mainly used. However, because hyperspectral images contain extremely large amounts of information and data, in actual production applications, for large-area tobacco yield estimation, the cost of collecting hyperspectral data is relatively high and the efficiency is relatively low. Therefore, how to accurately and efficiently estimate the yield of large-area tobacco fields is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method, device, equipment and storage medium for predicting the yield of large-area tobacco fields, aiming to solve the technical problem of low accuracy and efficiency in predicting the yield of large-area tobacco fields at present.

[0007] To achieve the above object, the present invention provides a method for predicting the yield of large-area tobacco fields, including the following steps:

[0008] Construct a sample dataset; wherein, the sample dataset includes a tobacco field sample dataset and a tobacco plant sample dataset, the tobacco field sample dataset includes a number of orthophotos of tobacco planting areas with tobacco field annotations, and the tobacco plant sample dataset includes a number of orthophotos of tobacco planting areas with tobacco plant annotations;

[0009] Input the tobacco field sample dataset and the tobacco plant sample dataset into a camouflaged object instance segmentation model and a small object detection model respectively for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0010] Obtain the orthophoto of the tobacco leaves in the area to be estimated for production collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the orthophoto of the tobacco leaves;

[0011] Use the tobacco field recognition model and the tobacco plant recognition model to identify the orthophoto of the tobacco leaves to obtain a tobacco field recognition image and a tobacco plant recognition image of the area to be estimated for production; wherein, the tobacco field recognition image includes a number of tobacco field edge pixel points, and the tobacco plant recognition image includes a number of tobacco plant candidate boxes;

[0012] Use the tobacco field edge pixel points and the tobacco plant candidate boxes to determine the tobacco field area and tobacco plant data; wherein, the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the diameter of the tobacco plant;

[0013] Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center point of the tobacco plant, and the diameter of the tobacco plant, construct a tobacco leaf yield prediction model, and use the tobacco leaf yield prediction model to predict the tobacco field yield of the area to be estimated for production.

[0014] Optionally, the step of constructing the sample dataset specifically includes:

[0015] Obtain the orthophoto of the tobacco planting area collected by the drone; wherein, the orthophoto of the tobacco planting area includes each key period of the tobacco growth process;

[0016] Annotate the tobacco fields in the orthophoto of the tobacco planting area, and use the annotated orthophoto of the tobacco planting area to construct a tobacco field sample dataset; wherein, the annotation tool is a semi-automatic polygon annotation tool;

[0017] Annotate the tobacco plants in the orthophoto of the tobacco planting area, and use the annotated orthophoto of the tobacco planting area to construct a tobacco plant sample dataset; wherein, the annotation tool is a rectangular box annotation tool;

[0018] Randomly extract the sample dataset in proportion and divide it into a training dataset and a validation dataset.

[0019] Optionally, inputting the tobacco field sample dataset and the tobacco plant sample dataset into a camouflaged object instance segmentation model and a small object detection model respectively for training to obtain a tobacco field recognition model and a tobacco plant recognition model, the steps specifically include:

[0020] Training the camouflaged object instance segmentation model with the training dataset in the tobacco field sample dataset, and validating the trained model with the validation dataset in the tobacco field sample dataset; wherein, the camouflaged object instance segmentation model adopts the OSFormer model;

[0021] Training the small object detection model with the training dataset in the tobacco plant sample dataset, and validating the trained model with the validation dataset in the tobacco plant sample dataset; wherein, the small object detection model adopts the CEASC model.

[0022] Optionally, obtaining the orthophoto image of tobacco leaves in the area to be estimated by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves, the steps specifically include:

[0023] Taking the tobacco leaf maturity period as the period for yield estimation, arranging image control points in the area to be estimated, and obtaining the coordinate data of the tobacco plants at the sampling points with a positioning instrument;

[0024] Obtaining the orthophoto image of tobacco leaves of the entire area to be estimated collected by the drone, and extracting the longitude and latitude coordinates of the waypoints during the drone collection as the longitude and latitude coordinates of the center point of the orthophoto image;

[0025] Obtaining the tobacco leaf yield data obtained by picking the tobacco leaves of the tobacco plants at the sampling points and handing them over to the tobacco curing station for baking.

[0026] Optionally, using the tobacco field edge pixel points and the tobacco plant candidate boxes to determine the tobacco field area and tobacco plant data, the steps specifically include:

[0027] Calculating the actual ground distance corresponding to a single pixel on the orthophoto image of tobacco leaves according to the drone camera parameters and flight altitude;

[0028] Converting the offset of the tobacco field edge pixel points relative to the center point of the orthophoto image of tobacco leaves from pixel distance to actual ground distance;

[0029] Extracting the pixel coordinates of the center point and the four vertices of the tobacco plant candidate box, determining the longitude and latitude coordinates of the center point and the four vertices of the tobacco plant candidate box based on the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves, and calculating the longitude and latitude coordinates of the tobacco field edge pixel points;

[0030] Assemble the longitude and latitude coordinates of the pixel points at the edge of the tobacco field into a valid polygon object according to the order of the pixels, calculate the area of the polygon object to obtain the tobacco field area; assemble the longitude and latitude coordinates of the four vertices of the tobacco plant candidate box into a valid rectangle, calculate the length of the short side of the rectangle to obtain the tobacco plant diameter, and associate it with the tobacco plant candidate box.

[0031] Optionally, the steps of constructing a tobacco leaf yield prediction model based on the tobacco leaf yield data of the sampled tobacco plants, the coordinate data of the sampled tobacco plants, the longitude and latitude coordinates of the center point of the tobacco plant, and the tobacco plant diameter specifically include:

[0032] Match the longitude and latitude coordinates of the center point of the tobacco plant and the coordinate data of the sampled tobacco plants, and extract the diameter of the sampled tobacco plants;

[0033] Use the diameter of the sampled tobacco plants and the tobacco leaf yield data, and adopt the least squares linear regression machine learning algorithm to construct a tobacco leaf yield prediction model.

[0034] Optionally, the steps of predicting the tobacco field yield of the area to be estimated using the tobacco leaf yield prediction model specifically include:

[0035] Use the polygon object and the center point of the tobacco plant candidate box to query the positional relationship, and eliminate the tobacco plants located outside the polygon object;

[0036] Input the tobacco plant diameters located within each polygon object into the yield prediction model to obtain the tobacco leaf yields of each tobacco plant in the area to be estimated;

[0037] Statistically calculate the average value of the tobacco leaf yields of the tobacco plants contained in each polygon object, and multiply it by the area of the corresponding tobacco field to obtain the tobacco field yield of the area to be estimated.

[0038] In addition, to achieve the above purpose, a large-area tobacco field yield prediction device includes:

[0039] A construction module for constructing a sample data set; wherein, the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes several orthophotos of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes several orthophotos of tobacco planting areas with tobacco plant annotations;

[0040] A training module for respectively inputting the tobacco field sample data set and the tobacco plant sample data set into a camouflaged target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0041] An extraction module for obtaining the orthophoto of the tobacco leaves in the area to be estimated collected by the drone, the tobacco leaf yield data and coordinate data of the sampled tobacco plants, and extracting the longitude and latitude coordinates of the center point of the orthophoto of the tobacco leaves;

[0042] An identification module, configured to identify the orthophoto image of the tobacco leaves by using the tobacco field identification model and the tobacco plant identification model, so as to obtain a tobacco field identification image and a tobacco plant identification image of the area to be estimated; wherein, the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate boxes;

[0043] A determination module, configured to determine the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate boxes; wherein, the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter;

[0044] An estimation module, configured to construct a tobacco leaf yield estimation model based on the tobacco leaf yield data of the sampled tobacco plants, the coordinate data of the sampled tobacco plants, the longitude and latitude coordinates of the tobacco plant center point, and the tobacco plant diameter, and use the tobacco leaf yield estimation model to estimate the tobacco field yield of the area to be estimated.

[0045] In addition, to achieve the above object, the present invention further provides a large-area tobacco field yield estimation device, where the large-area tobacco field yield estimation device includes: a memory, a processor, and a large-area tobacco field yield estimation program stored on the memory and executable on the processor, and when the large-area tobacco field yield estimation program is executed by the processor, the steps of the above-mentioned large-area tobacco field yield estimation method are implemented.

[0046] In addition, to achieve the above object, the present invention further provides a storage medium, on which a large-area tobacco field yield estimation program is stored, and when the large-area tobacco field yield estimation program is executed by a processor, the steps of the above-mentioned large-area tobacco field yield estimation method are implemented.

[0047] The beneficial effects of the present invention are as follows: A method, device, equipment, and storage medium for predicting the yield of a large-area tobacco field are proposed. The method includes: constructing a sample data set; inputting the tobacco field sample data set and the tobacco plant sample data set into a camouflaged target instance segmentation model and a small target detection model respectively for training to obtain a tobacco field recognition model and a tobacco plant recognition model; acquiring the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the unmanned aerial vehicle, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves; using the tobacco field recognition model and the tobacco plant recognition model to recognize the orthophoto image of tobacco leaves, obtaining the tobacco field recognition image and the tobacco plant recognition image of the area to be estimated, determining the tobacco field area and the tobacco plant data, constructing a tobacco leaf yield prediction model, and using the tobacco leaf yield prediction model to predict the tobacco field yield of the area to be estimated. By separately recognizing the tobacco field and the tobacco plants in the area to be estimated, and according to the tobacco field recognition image and the tobacco plant recognition image, considering the tobacco field area and the tobacco plant data determined by the tobacco field edge pixel points and the tobacco plant candidate boxes, the present invention constructs a tobacco leaf yield prediction model to predict the tobacco field yield of the area to be estimated, which has higher accuracy and scene adaptability compared with the existing prediction methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment related to the solution of the embodiment of the present invention.

[0050] Figure 2 It is a schematic flowchart of the embodiment of the method for predicting the yield of a large-area tobacco field of the present invention.

[0051] Figure 3 It is an example diagram of tobacco field sample annotation in the embodiment of the present invention.

[0052] Figure 4 It is an example diagram of tobacco plant sample annotation in the embodiment of the present invention.

[0053] Figure 5 It is a tobacco field recognition result diagram in the embodiment of the present invention.

[0054] Figure 6 It is a tobacco plant recognition result diagram in the embodiment of the present invention.

[0055] Figure 7 It is a tobacco field yield prediction diagram in the embodiment of the present invention.

[0056] Figure 8 This is the structural block diagram of a large-area tobacco field yield prediction device in an embodiment of the present invention.

[0057] The realization of the purpose, functional characteristics and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0059] As Figure 1 shown, Figure 1 This is the schematic structural diagram of the device of the hardware operating environment involved in the embodiment solution of the present invention.

[0060] As Figure 1 shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art can understand that Figure 1 the structure of the device shown in

[0062] As Figure 1 shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a large-area tobacco field yield prediction program.

[0063] In Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user side) and communicate data with the client; and the processor 1001 can be used to call the large-area tobacco field yield estimation program stored in the memory 1005 and perform the following operations:

[0064] Construct a sample data set; wherein, the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes several orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes several orthophoto images of tobacco planting areas with tobacco plant annotations;

[0065] Input the tobacco field sample data set and the tobacco plant sample data set into a camouflaged object instance segmentation model and a small object detection model respectively for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0066] Obtain the orthophoto image of the tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extract the longitude and latitude coordinates of the center point of the orthophoto image of the tobacco leaves;

[0067] Use the tobacco field recognition model and the tobacco plant recognition model to identify the orthophoto image of the tobacco leaves to obtain a tobacco field recognition image and a tobacco plant recognition image of the area to be estimated; wherein, the tobacco field recognition image includes several tobacco field edge pixel points, and the tobacco plant recognition image includes several tobacco plant candidate boxes;

[0068] Use the tobacco field edge pixel points and the tobacco plant candidate boxes to determine the tobacco field area and tobacco plant data; wherein, the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the diameter of the tobacco plant;

[0069] Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center point of the tobacco plant, and the diameter of the tobacco plant, construct a tobacco leaf yield estimation model, and use the tobacco leaf yield estimation model to estimate the tobacco field yield of the area to be estimated.

[0070] The specific embodiments of the present invention applied to the device are basically the same as those of the following embodiments of the method for estimating the yield of large-area tobacco fields, and will not be elaborated here.

[0071] The embodiments of the present invention provide a method for estimating the yield of large-area tobacco fields, referring to Figure 2 , Figure 2 is a schematic flowchart of an embodiment of the method for estimating the yield of large-area tobacco fields of the present invention.

[0072] In this embodiment, the method for estimating the yield of large-area tobacco fields includes the following steps:

[0073] Step S100, construct a sample data set; wherein, the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes several orthophotos of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes several orthophotos of tobacco planting areas with tobacco plant annotations.

[0074] Specifically, in this embodiment, orthophotos of tobacco planting areas are obtained by a drone; wherein, the orthophotos of tobacco planting areas include various key periods in the tobacco growth process; the tobacco fields in the orthophotos of tobacco planting areas are annotated, and the annotated orthophotos of tobacco planting areas are used to construct a tobacco field sample data set; wherein, the annotation tool is a semi-automatic polygon annotation tool; the tobacco plants in the orthophotos of tobacco planting areas are annotated, and the annotated orthophotos of tobacco planting areas are used to construct a tobacco plant sample data set; wherein, the annotation tool is a rectangular box annotation tool; the sample data set is randomly sampled according to a ratio and divided into a training data set and a validation data set.

[0075] In practical applications, in this embodiment, a DJI Phantom 4 RTK drone is used to fly to areas known to have tobacco plantings for aerial photography to obtain JPEG tobacco orthophotos with a resolution of 4864×3648 (4:3) at different heights. The collected orthophotos include 4 key growth periods: the seedling return period, the root extension period, the vigorous growth period, and the maturity period, which helps to improve the accuracy and generalization ability of the model, and thus better applies to actual tobacco growth monitoring and analysis tasks.

[0076] Through experimental tests, the optimal collection height of the drone is 50 - 80m. At this time, the orthophotos collected can clearly distinguish tobacco plants.

[0077] Furthermore, this embodiment also includes separately annotating the tobacco fields and tobacco plants in the orthophotos, and using the annotated orthophotos to construct a sample data set.

[0078] In practical applications, 9857 orthophotos with a collection height between 50 - 80m are selected for annotation. As Figure 3 shown, when annotating the tobacco fields, the semi-automatic polygon annotation tool AnyLabeling-GPU is used to annotate each plot along the plot edge, ensuring that the entire plot boundary is completely depicted and fully contained within the annotation area to prevent the plot edge from being missed or incompletely annotated; as Figure 4 shown, when annotating the tobacco plants, the rectangular box annotation tool Labelimg is used to frame the tobacco plants, ensuring that each rectangular box completely contains the entire tobacco plant to avoid any part being missed or truncated; then 80% of the annotated orthophoto samples are randomly sampled as the training data set, and 20% as the validation data set.

[0079] Step S200: Input the tobacco field sample dataset and the tobacco plant sample dataset into the camouflaged target instance segmentation model and the small target detection model respectively for training to obtain a tobacco field recognition model and a tobacco plant recognition model.

[0080] Specifically, in this embodiment, the camouflaged target instance segmentation model is trained using the tobacco field sample training dataset, and the trained model is verified using the tobacco field sample validation dataset. Among them, the camouflaged target instance segmentation model uses the OSFormer model. The small target detection model is trained using the tobacco plant sample training dataset, and the trained model is verified using the tobacco plant sample validation dataset. Among them, the small target detection model uses the CEASC model.

[0081] In practical applications, the OSFormer model is used to train the tobacco field sample dataset. The specific steps include: dividing the ortho-images of the randomly selected tobacco field sample training dataset and the tobacco field sample validation dataset into folders, inputting the tobacco field sample training dataset into the OSFormer network model for training. First, update the weight file of the original OSFormer model to the weight of the tobacco field plot recognition model. Secondly, through repeated training, a model with an accuracy rate of 30.2% is obtained. Then, optimize the model structure, modify the learning rate in the optimizer parameters to 1e-05, the weight decay to 0.0001, the learning rate to 0.0001, modify the hidden layer dimension in the model parameters to 256, the number of heads in the multi-head attention mechanism to 8, the number of key points to 1, the sigma value of non-maximum suppression to 2.0 and other model parameters, optimize the cross-entropy loss coefficient in the loss function to 2.0, the mask focal loss coefficient to 2.0, the edge focal loss coefficient to 2.0 and other loss function-related parameters. Finally, output the tobacco field recognition model for tobacco field plot recognition, and its recognition accuracy rate can reach 85.87%.

[0082] In practical applications, the CEASC model is used to train the tobacco plant sample dataset. The specific steps include: dividing the ortho-images of the randomly selected tobacco plant sample training dataset and the tobacco plant sample validation dataset into folders, inputting the tobacco plant sample training dataset into the CEASC network model for training, selecting the weight file of the swin_tiny model, using 4 threads for data loading to improve the model training speed, setting the pooling layer scales in the model to [1, 2, 3, 6], setting the classification loss coefficient to 10, the point loss coefficient to 2, the Tversky loss coefficient to 2, setting the learning rate lr to 1e-5 to avoid over-large learning rate resulting in non-fitting training results and over-small learning rate resulting in increased training time cost and getting stuck in local optimal solutions and being difficult to jump out to find the global optimal solution, and adopting a method of dynamically adjusting the learning rate with the learning rate decay period number being 50 to further improve the training effect. After repeated training, a model with an accuracy of 86.3% is obtained, and finally the tobacco plant model is output for tobacco plant recognition.

[0083] Step S300: Obtain the ortho-image of the tobacco leaves in the area to be estimated for yield collected by the UAV, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the ortho-image of the tobacco leaves.

[0084] Specifically, in this embodiment, the tobacco leaf maturity period is used as the period for yield estimation. Image control points are arranged in the area to be estimated for yield, and coordinate data of the sampling points are obtained using a positioning instrument; the UAV obtains the ortho-image of the entire area to be estimated for yield; the tobacco leaves of the sample point tobacco plants are picked and handed over to the tobacco roasting station for baking, and the tobacco leaf yield data of the tobacco plants is the tobacco leaf yield obtained from the tobacco roasting station; the longitude and latitude coordinates of the waypoints during the UAV collection are extracted as the longitude and latitude coordinates of the center point of the ortho-image.

[0085] In practical applications, from late July to early to mid-August, some tobacco planting areas in Daguan County, Zhaotong City are selected to carry out large-area tobacco field yield estimation.

[0086] During the tobacco leaf maturity period, a DJI Phantom 4 RTK UAV is used to collect the ortho-image of the entire area to be estimated for yield; before collection, image control points are evenly arranged in the area to be estimated for yield, and the image control point positions are clear and easy to distinguish; then, a RTK positioning instrument is used to obtain the coordinate data of the selected sampling points, specifically, the center point coordinate data of the sampling point tobacco plants are obtained, and each sampling tobacco plant is numbered; after the UAV collects the ortho-image, the tobacco leaves of the sampling point tobacco plants are picked, and the mature tobacco leaves are collected according to the tobacco leaf picking and roasting specifications and methods and handed over to the tobacco roasting station for roasting, and the tobacco roasting station provides the tobacco leaf yield data, and the yield data corresponds one by one to the sampling tobacco plant numbers; the longitude and latitude coordinates of the waypoints during the UAV collection are extracted as the longitude and latitude coordinates of the center point of the ortho-image.

[0087] Step S400: Identify the orthophoto image of the tobacco leaves using the tobacco field identification model and the tobacco plant identification model to obtain the tobacco field identification image and the tobacco plant identification image of the area to be estimated for production; wherein, the tobacco field identification image includes a number of tobacco field edge pixel points, and the tobacco plant identification image includes a number of tobacco plant candidate boxes.

[0088] Specifically, in this embodiment, as Figure 5 shown, the identified tobacco field identification image includes a series of tobacco field edge pixel points; as Figure 6 shown, the tobacco plant identification image includes tobacco plant candidate boxes.

[0089] Step S500: Determine the tobacco field area and tobacco plant data using the tobacco field edge pixel points and the tobacco plant candidate boxes; wherein, the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter.

[0090] Specifically, in this embodiment, according to the UAV camera parameters and the flight altitude, calculate the actual ground distance corresponding to a single pixel on the orthophoto image; convert the offset of the tobacco field edge pixel points relative to the center point of the orthophoto image from pixel distance to actual ground distance; extract the pixel coordinates of the center point and the four vertices of the tobacco plant candidate box, and based on the longitude and latitude coordinates of the center point of the orthophoto image, calculate the longitude and latitude coordinates of the center point and the four vertices of the tobacco plant candidate box, and calculate the longitude and latitude coordinates of the target tobacco field edge pixel points; as Figure 5 shown, assemble the longitude and latitude coordinates of the target tobacco field edge pixel points into a valid polygon object according to the order of the pixels, calculate the area of the polygon object to obtain the tobacco field area; as Figure 6 shown, assemble the longitude and latitude coordinates of the four vertices of the tobacco plant candidate box into a valid rectangle, calculate the short side length of the rectangle to obtain the tobacco plant diameter, and associate it with the tobacco plant candidate box.

[0091] In practical applications, according to the sensor size, focal length, flight altitude, pixel size, and the ratio of the photographed photo of the DJI Phantom 4 RTK drone, calculate the actual ground distance corresponding to a single pixel on the orthophoto image. The specific calculation steps are as follows: First, convert the relationship between the field of view angle FOV, sensor size, and focal length as: tan(FOV / 2) = (sensor size / 2) / focal length. Then, according to the FOV and flight altitude, calculate the hypotenuse length A of the actual photograph of a single orthophoto image as A = 2 * tan(FOV / 2) * flight altitude. Then, according to A and the photo shooting ratio of 4:3, calculate the actual length and width of the ground corresponding to the shooting range of a single orthophoto image through the Pythagorean theorem. Finally, according to the pixel size of the camera shooting, calculate the actual ground distance corresponding to a single pixel on the orthophoto image; then convert the offset of the tobacco field edge pixel points relative to the center point of the orthophoto image from pixel distance to actual ground distance; then extract the pixel coordinates of the center point and four vertices of the tobacco plant candidate box. Based on the longitude and latitude coordinates of the center point of the orthophoto image, use the conversion relationship between longitude and latitude and distance angle to calculate the longitude and latitude coordinates of the center point and four vertices of the tobacco plant candidate box, and calculate the longitude and latitude coordinates of the target tobacco field edge pixel points.

[0092] Assemble the longitude and latitude coordinates of the target tobacco field edge pixel points into a valid polygon object according to the order of the pixels, calculate the area of the polygon object to obtain the tobacco field area; assemble the longitude and latitude coordinates of the four vertices of the tobacco plant candidate box into a valid rectangle, calculate the short side length of the rectangle to obtain the tobacco plant diameter, and associate it with the tobacco plant candidate box.

[0093] Step S600, based on the tobacco leaf yield data of the sampled tobacco plants, the coordinate data of the sampled tobacco plants, the longitude and latitude coordinates of the center point of the tobacco plant, and the tobacco plant diameter, construct a tobacco leaf yield prediction model, and use the tobacco leaf yield prediction model to predict the tobacco field yield of the area to be estimated.

[0094] Specifically, in this embodiment, match the longitude and latitude coordinates of the center point of the tobacco plant and the coordinate data of the sampled tobacco plants, and extract the diameter of the sampled tobacco plants; use the diameter of the sampled tobacco plants and the tobacco leaf yield data, and adopt the least squares linear regression machine learning algorithm to construct a tobacco leaf yield prediction model. The model evaluation index selects the coefficient of determination R 2 , the closer the value is to 1, the higher the accuracy. The model formula is as follows: Yield = 52.445 + 0.761 * Diameter, R 2 is 0.949; where, Yield represents the yield of a single tobacco plant, is the average yield per mu, and the unit is kilograms; Diameter represents the diameter of each tobacco plant, and the unit is cm.

[0095] After that, using the tobacco field polygon object and the center points of the tobacco plant candidate boxes, a position relationship query is performed to eliminate the tobacco plants located outside the tobacco field polygon object; the diameters of the tobacco plants located within each tobacco field polygon are input into the yield prediction model to obtain the tobacco leaf yields of each tobacco plant in the area to be estimated. The average value of the tobacco leaf yields of the tobacco plants contained in each tobacco field polygon is statistically calculated and multiplied by the area of the corresponding tobacco field to obtain the tobacco field yield in the area to be estimated, as Figure 7 shown.

[0096] Refer to Figure 8 , Figure 8 which is the structural block diagram of the embodiment of the large-area tobacco field yield prediction device of the present invention.

[0097] As Figure 8 shown, the large-area tobacco field yield prediction device proposed in the embodiment of the present invention includes:

[0098] A construction module 10 for constructing a sample data set; wherein, the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes several orthophotos of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes several orthophotos of tobacco planting areas with tobacco plant annotations;

[0099] A training module 20 for respectively inputting the tobacco field sample data set and the tobacco plant sample data set into a camouflaged target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0100] An extraction module 30 for obtaining the orthophoto of the tobacco leaves in the area to be estimated collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extracting the longitude and latitude coordinates of the center point of the orthophoto of the tobacco leaves;

[0101] A recognition module 40 for using the tobacco field recognition model and the tobacco plant recognition model to recognize the orthophoto of the tobacco leaves to obtain a tobacco field recognition image and a tobacco plant recognition image of the area to be estimated; wherein, the tobacco field recognition image includes several tobacco field edge pixel points, and the tobacco plant recognition image includes several tobacco plant candidate boxes;

[0102] A determination module 50 for using the tobacco field edge pixel points and the tobacco plant candidate boxes to determine the tobacco field area and tobacco plant data; wherein, the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the diameter of the tobacco plant;

[0103] An estimation module 60 for constructing a tobacco leaf yield prediction model based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center point of the tobacco plant, and the diameter of the tobacco plant, and using the tobacco leaf yield prediction model to estimate the tobacco field yield in the area to be estimated.

[0104] Other embodiments or specific implementation manners of the large-area tobacco field yield prediction device of the present invention may refer to the above method embodiments, and will not be elaborated here.

[0105] In addition, the present invention also provides a large-area tobacco field yield prediction device, which includes: a memory, a processor, and a large-area tobacco field yield prediction program stored on the memory and executable on the processor. When the large-area tobacco field yield prediction program is executed by the processor, the steps of the large-area tobacco field yield prediction method described above are implemented.

[0106] The specific implementation manner of the large-area tobacco field yield prediction device of the present application is basically the same as that of each embodiment of the above large-area tobacco field yield prediction method, and will not be elaborated here.

[0107] In addition, the present invention also provides a readable storage medium, which includes a computer-readable storage medium with a large-area tobacco field yield prediction program stored thereon. The readable storage medium may be Figure 1 the memory 1005 in the terminal, or at least one of ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, and optical disk. The readable storage medium includes several instructions for causing a large-area tobacco field yield prediction device with a processor to execute the large-area tobacco field yield prediction method described in each embodiment of the present invention.

[0108] The specific implementation manner of the readable storage medium of the present application is basically the same as that of each embodiment of the above large-area tobacco field yield prediction method, and will not be elaborated here.

[0109] It can be understood that in the description of this specification, the descriptions referring to terms such as "one embodiment", "another embodiment", "other embodiments", or "the first embodiment to the Nth embodiment" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent in such a process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or system including that element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate rather than limit the technical solution of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be modified or equivalently replaced, and any modification or partial replacement without departing from the spirit and scope of the present invention shall be covered by the scope of the claims of the present invention.

Claims

1. A method for estimating the yield of a large area of ​​tobacco field, characterized in that: The following steps are involved: Constructing a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations; The tobacco field sample data set and the tobacco plant sample data set are respectively input into a disguised target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model; Obtaining the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves; The tobacco leaf orthophoto image is identified by using the tobacco field identification model and the tobacco plant identification model to obtain a tobacco field identification image and a tobacco plant identification image of the yield estimation area; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames; The tobacco field area and tobacco plant data are determined by using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the tobacco plant diameter; specifically including: calculating the actual ground distance corresponding to the unit pixel on a single tobacco leaf orthophoto image according to the drone camera parameters and the flight altitude; converting the offset of the tobacco field edge pixel point relative to the center point of the tobacco leaf orthophoto image from the pixel distance to the actual ground distance; extracting the pixel coordinates of the center point and four vertices of the tobacco plant candidate frame, determining the longitude and latitude coordinates of the center point and four vertices of the tobacco plant candidate frame based on the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image, and calculating the longitude and latitude coordinates of the tobacco field edge pixel point; assembling the longitude and latitude coordinates of the tobacco field edge pixel point into a valid polygon object according to the order of pixels, calculating the area of ​​the polygon object, and obtaining the tobacco field area; assembling the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame into a valid rectangle, calculating the short side length of the rectangle, obtaining the tobacco plant diameter, and associating it with the tobacco plant candidate frame; Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameters of the tobacco plants, a tobacco leaf yield estimation model is constructed, and the tobacco field yield in the to-be-estimated production area is estimated using the tobacco leaf yield estimation model; specifically, the method comprises: matching the longitude and latitude coordinates of the center points of the tobacco plants with the coordinate data of the tobacco plants at the sampling points to extract the diameters of the tobacco plants at the sampling points; using the diameters of the tobacco plants at the sampling points and the tobacco leaf yield data, and adopting a least squares linear regression machine learning algorithm to construct a tobacco leaf yield estimation model; using the polygonal object and the center point of the tobacco plant candidate frame to perform a position relationship query, and eliminating the tobacco plants outside the polygonal object; inputting the diameters of the tobacco plants within each polygonal object into the yield estimation model to obtain the tobacco leaf yield of each tobacco plant in the to-be-estimated production area; and calculating the average tobacco leaf yield of the tobacco plants contained in each polygonal object, and multiplying the result by the area of ​​the corresponding tobacco field to obtain the tobacco field yield in the to-be-estimated production area.

2. The method for estimating the yield of a large area of ​​tobacco field according to claim 1, characterized in that: The steps to construct a sample data set include: Obtaining an orthophoto image of a tobacco planting area collected by a drone; wherein the orthophoto image of the tobacco planting area includes each key period of the tobacco growth process; Annotating tobacco fields in the orthophoto image of the tobacco-growing area, and constructing a tobacco field sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a polygon semi-automatic annotation tool; Annotating tobacco plants in the orthophoto image of the tobacco-growing area, and constructing a tobacco plant sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a rectangular frame annotation tool; The sample data set is randomly selected in proportion and divided into a training data set and a validation data set.

3. The method for estimating the yield of a large area of ​​tobacco field as claimed in claim 2, characterized in that: The steps of inputting the tobacco field sample data set and the tobacco plant sample data set into the camouflaged target instance segmentation model and the small target detection model for training respectively to obtain the tobacco field recognition model and the tobacco plant recognition model specifically include: The disguised target instance segmentation model is trained using the training data set in the tobacco field sample data set, and the trained model is verified using the verification data set in the tobacco field sample data set; wherein the disguised target instance segmentation model adopts the OSFormer model; The small target detection model is trained using the training data set in the tobacco plant sample data set, and the trained model is verified using the verification data set in the tobacco plant sample data set; wherein the small target detection model adopts the CEASC model.

4. The method for estimating the yield of a large area of ​​tobacco field according to claim 1, characterized in that: The steps of obtaining the tobacco leaf orthophoto image of the area to be estimated collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extracting the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image specifically include: Taking the tobacco leaf maturity period as the period for yield estimation, image control points are arranged in the area to be estimated, and the coordinate data of the tobacco plants at the sampling points are obtained using positioning instruments; Obtain an orthophoto of tobacco leaves in the entire area to be estimated collected by a drone, and extract the longitude and latitude coordinates of the waypoints collected by the drone as the longitude and latitude coordinates of the center point of the orthophoto; The tobacco leaf yield data is obtained by obtaining tobacco leaves from tobacco plants picked at sampling points and handing them over to tobacco flue-curing stations for baking.

5. A large-area tobacco field yield estimation device, characterized in that: include: A construction module, used to construct a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations; A training module, used for inputting the tobacco field sample data set and the tobacco plant sample data set into a disguised target instance segmentation model and a small target detection model for training, so as to obtain a tobacco field recognition model and a tobacco plant recognition model; An extraction module is used to obtain the orthophoto image of tobacco leaves in the area to be estimated, collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the orthophoto image of the tobacco leaves; A recognition module, used to recognize the tobacco leaf orthophoto image using the tobacco field recognition model and the tobacco plant recognition model, and obtain a tobacco field recognition image and a tobacco plant recognition image of the yield estimation area; wherein the tobacco field recognition image includes a plurality of tobacco field edge pixel points, and the tobacco plant recognition image includes a plurality of tobacco plant candidate frames; A determination module is used to determine the tobacco field area and tobacco plant data using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the diameter of the tobacco plant; specifically includes: calculating the actual ground distance corresponding to the unit pixel on a single tobacco leaf orthophoto image according to the drone camera parameters and the flight altitude; converting the offset of the tobacco field edge pixel point relative to the center point of the tobacco leaf orthophoto image from the pixel distance to the actual ground distance; extracting the pixel coordinates of the center point and four vertices of the tobacco plant candidate frame, determining the longitude and latitude coordinates of the center point and four vertices of the tobacco plant candidate frame based on the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image, and calculating the longitude and latitude coordinates of the tobacco field edge pixel point; assembling the longitude and latitude coordinates of the tobacco field edge pixel point into a valid polygon object according to the order of pixels, calculating the area of ​​the polygon object, and obtaining the tobacco field area; assembling the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame into a valid rectangle, calculating the short side length of the rectangle, obtaining the tobacco plant diameter, and associating it with the tobacco plant candidate frame; The estimation module is used to construct a tobacco leaf yield estimation model based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, and use the tobacco leaf yield estimation model to estimate the tobacco field yield in the area to be estimated; specifically including: matching the longitude and latitude coordinates of the center points of the tobacco plants and the coordinate data of the tobacco plants at the sampling points to extract the diameters of the tobacco plants at the sampling points; using the diameters of the tobacco plants at the sampling points and the tobacco leaf yield data, and using the least squares linear regression machine learning algorithm to construct a tobacco leaf yield estimation model; using the polygonal object and the center point of the tobacco plant candidate frame to query the position relationship, and eliminating the tobacco plants outside the polygonal object; inputting the diameter of the tobacco plants in each polygonal object into the yield estimation model to obtain the tobacco leaf yield of each tobacco plant in the area to be estimated; counting the average tobacco leaf yield of the tobacco plants contained in each polygonal object, multiplying it by the area of ​​the corresponding tobacco field, to obtain the tobacco field yield in the area to be estimated.

6. A large-area tobacco field yield estimation device, characterized in that: The large-area tobacco field yield estimation device includes: a memory, a processor, and a large-area tobacco field yield estimation program stored in the memory and executable on the processor. When the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method as described in any one of claims 1 to 4 are implemented.

7. A storage medium, characterized in that: The storage medium stores a large-area tobacco field yield estimation program, and when the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method according to any one of claims 1 to 4 are implemented.

Images