Method, device and equipment for estimating yield of large-area tobacco field and storage medium

Abstract

The invention relates to the technical field of crop yield estimation, and discloses a large-area tobacco field yield estimation method, device and equipment and a storage medium, and the method comprises the steps: respectively inputting a tobacco field sample data set and a tobacco plant sample data set into a camouflage target instance segmentation model and a small target detection model for training, obtaining a tobacco field identification model and a tobacco plant identification model; obtaining a tobacco leaf orthoimage of a to-be-estimated-yield area collected by the unmanned aerial vehicle, and tobacco leaf yield data and coordinate data of tobacco plants at sampling points; and identifying the tobacco leaf orthoimage by using the tobacco field identification model and the tobacco plant identification model, constructing a tobacco leaf yield pre-estimation model, and pre-estimating the tobacco field yield of the region to be subjected to yield estimation by using the tobacco leaf yield pre-estimation model. According to the method, tobacco field identification and tobacco plant identification are respectively carried out on the to-be-estimated area, the tobacco field yield of the to-be-estimated area is estimated by considering the tobacco field edge pixel points and the tobacco field area and tobacco plant data determined by the tobacco plant candidate frame, and the method has higher accuracy and scene adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of crop yield estimation, and in particular to a method, device, equipment and storage medium for estimating the yield of a large-area tobacco field. Background Art

[0002] Crop yield forecasting is an important agricultural intelligence that is indispensable for the country to formulate agricultural policies. Timely and accurate forecasting can provide effective support for agricultural experience management and is also an urgent need for the development of precision agriculture. The tobacco industry plays a very important role in the national economic and social development process. The output of tobacco affects the price of tobacco, which in turn affects the income of tobacco farmers. Accurate estimation of tobacco production is of great significance to the pricing and overall allocation of the tobacco industry.

[0003] Traditional tobacco yield estimation is mostly carried out through field survey sampling. By selecting representative tobacco planting sample areas, the tobacco leaves in the sample areas are harvested and cured to obtain the tobacco leaf yield of the tobacco area, and then the tobacco leaf yield of a larger area is extrapolated. The traditional tobacco yield estimation method is not only time-consuming, labor-intensive, and difficult to carry out on a large scale, but also the yield data obtained by the point-to-surface method is of low accuracy.

[0004] Remote sensing technology is widely used in crop yield estimation due to its advantages of being macroscopic, dynamic, fast and accurate. However, traditional satellite remote sensing yield estimation is suitable for large-scale crops. For small-scale crop yield estimation, due to its long repetition cycle, low temporal and spatial resolution and the influence of the atmospheric environment, it is impossible to obtain accurate ground information in a timely and effective manner. UAV remote sensing has the characteristics of low cost, high spatial resolution, real-time and little influence by the atmosphere, providing a new technical means for remote sensing crop yield estimation at a small scale.

[0005] Existing UAV remote sensing tobacco yield estimation mainly uses hyperspectral data, but because hyperspectral images contain a large amount of information and data, in actual production applications, for large-scale tobacco yield estimation, the cost of collecting hyperspectral data is high and the efficiency is low. Therefore, how to accurately and efficiently estimate the yield of large-scale tobacco fields is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The main purpose of the present invention is to provide a large-area tobacco field yield estimation method, device, equipment and storage medium, aiming to solve the current technical problems of low accuracy and efficiency in large-area tobacco field yield estimation.

[0007] To achieve the above object, the present invention provides a method for estimating the yield of a large area of ​​tobacco field, comprising the following steps:

[0008] Constructing a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations;

[0009] The tobacco field sample data set and the tobacco plant sample data set are respectively input into a disguised target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0010] Obtaining the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves;

[0011] The tobacco leaf orthophoto image is identified by using the tobacco field identification model and the tobacco plant identification model to obtain a tobacco field identification image and a tobacco plant identification image of the yield estimation area; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames;

[0012] Determine the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter;

[0013] Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, a tobacco leaf yield estimation model is constructed, and the tobacco field yield in the area to be estimated is estimated using the tobacco leaf yield estimation model.

[0014] Optionally, the steps for constructing a sample data set include:

[0015] Obtaining an orthophoto image of a tobacco planting area collected by a drone; wherein the orthophoto image of the tobacco planting area includes each key period of the tobacco growth process;

[0016] Annotating tobacco fields in the orthophoto image of the tobacco-growing area, and constructing a tobacco field sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a polygon semi-automatic annotation tool;

[0017] Annotating tobacco plants in the orthophoto image of the tobacco-growing area, and constructing a tobacco plant sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a rectangular frame annotation tool;

[0018] The sample data set is randomly selected in proportion and divided into a training data set and a validation data set.

[0019] Optionally, the steps of inputting the tobacco field sample dataset and the tobacco plant sample dataset into a disguised target instance segmentation model and a small target detection model for training respectively to obtain a tobacco field recognition model and a tobacco plant recognition model specifically include:

[0020] The camouflaged target instance segmentation model is trained using the training data set in the tobacco field sample data set, and the trained model is verified using the verification data set in the tobacco field sample data set; wherein the camouflaged target instance segmentation model adopts the OSFormer model;

[0021] The small target detection model is trained using the training data set in the tobacco plant sample data set, and the trained model is verified using the verification data set in the tobacco plant sample data set; wherein the small target detection model adopts the CEASC model.

[0022] Optionally, the steps of obtaining an orthophoto image of tobacco leaves in the area to be estimated, tobacco leaf yield data and coordinate data of tobacco plants at sampling points collected by a drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves specifically include:

[0023] Taking the tobacco leaf maturity period as the period for yield estimation, image control points are arranged in the area to be estimated, and the coordinate data of the tobacco plants at the sampling points are obtained using positioning instruments;

[0024] Obtain an orthophoto of tobacco leaves in the entire area to be estimated collected by a drone, and extract the longitude and latitude coordinates of the waypoints collected by the drone as the longitude and latitude coordinates of the center point of the orthophoto;

[0025] The tobacco leaf yield data is obtained by obtaining tobacco leaves from tobacco plants picked at sampling points and handing them over to tobacco flue-curing stations for baking.

[0026] Optionally, the step of determining the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame specifically includes:

[0027] Calculate the actual ground distance corresponding to a unit pixel on a single orthophoto of the tobacco leaf according to the camera parameters and the flight altitude of the drone;

[0028] Converting the offset of the edge pixel point of the tobacco field relative to the center point of the tobacco leaf orthophoto image from pixel distance to actual ground distance;

[0029] Extract the pixel coordinates of the center point and four vertices of the tobacco plant candidate frame, determine the latitude and longitude coordinates of the center point and four vertices of the tobacco plant candidate frame based on the latitude and longitude coordinates of the center point of the tobacco leaf orthophoto image, and calculate the latitude and longitude coordinates of the edge pixel points of the tobacco field;

[0030] The longitude and latitude coordinates of the edge pixel points of the tobacco field are assembled into a valid polygonal object according to the order of pixels, and the area of ​​the polygonal object is calculated to obtain the area of ​​the tobacco field; the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame are assembled into a valid rectangle, and the length of the short side of the rectangle is calculated to obtain the diameter of the tobacco plant and associate it with the tobacco plant candidate frame.

[0031] Optionally, based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants, and the diameter of the tobacco plants, the steps of constructing a tobacco leaf yield estimation model specifically include:

[0032] Matching the longitude and latitude coordinates of the center point of the tobacco plant with the coordinate data of the tobacco plant at the sampling point, and extracting the diameter of the tobacco plant at the sampling point;

[0033] The tobacco leaf yield estimation model was constructed using the tobacco plant diameter and tobacco leaf yield data at the sampling points and the least squares linear regression machine learning algorithm.

[0034] Optionally, the step of estimating the tobacco field yield in the area to be estimated by using the tobacco leaf yield estimation model specifically includes:

[0035] Using the center points of the polygonal object and the tobacco plant candidate frame to query the position relationship, and eliminating tobacco plants outside the polygonal object;

[0036] The diameter of the tobacco plant within each polygonal object is input into the yield estimation model to obtain the tobacco leaf yield of each tobacco plant in the yield estimation area;

[0037] The average tobacco leaf yield of the tobacco plants contained in each polygon object is calculated and multiplied by the area of ​​the corresponding tobacco field to obtain the tobacco field yield in the area to be estimated.

[0038] In addition, in order to achieve the above purpose, a large-area tobacco field yield estimation device includes:

[0039] A construction module, used to construct a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations;

[0040] A training module, used for inputting the tobacco field sample data set and the tobacco plant sample data set into a disguised target instance segmentation model and a small target detection model for training, so as to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0041] An extraction module is used to obtain the orthophoto image of tobacco leaves in the area to be estimated, collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the orthophoto image of the tobacco leaves;

[0042] A recognition module, used to recognize the tobacco leaf orthophoto image using the tobacco field recognition model and the tobacco plant recognition model, and obtain a tobacco field recognition image and a tobacco plant recognition image of the yield estimation area; wherein the tobacco field recognition image includes a plurality of tobacco field edge pixel points, and the tobacco plant recognition image includes a plurality of tobacco plant candidate frames;

[0043] A determination module, used to determine the tobacco field area and tobacco plant data using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter;

[0044] The estimation module is used to construct a tobacco leaf yield estimation model based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, and use the tobacco leaf yield estimation model to estimate the tobacco field yield in the area to be estimated.

[0045] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a large-area tobacco field yield estimation device, which includes: a memory, a processor, and a large-area tobacco field yield estimation program stored in the memory and executable on the processor, and when the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method as described above are implemented.

[0046] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a large-area tobacco field yield estimation program is stored. When the large-area tobacco field yield estimation program is executed by a processor, the steps of the above-mentioned large-area tobacco field yield estimation method are implemented.

[0047] The beneficial effects of the present invention are as follows: a method, device, equipment and storage medium for estimating the yield of a large-area tobacco field are proposed, and the method comprises: constructing a sample data set; inputting the tobacco field sample data set and the tobacco plant sample data set into a camouflage target instance segmentation model and a small target detection model for training respectively, and obtaining a tobacco field recognition model and a tobacco plant recognition model; obtaining an orthophoto image of tobacco leaves in the area to be estimated, tobacco leaf yield data and coordinate data of tobacco plants at sampling points collected by a drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of the tobacco leaves; using the tobacco field recognition model and the tobacco plant recognition model to recognize the orthophoto image of the tobacco leaves, and obtaining a tobacco field recognition image and a tobacco plant recognition image of the area to be estimated, determining the tobacco field area and tobacco plant data, constructing a tobacco leaf yield estimation model, and using the tobacco leaf yield estimation model to estimate the tobacco field yield in the area to be estimated. The present invention respectively performs tobacco field identification and tobacco plant identification on the area to be estimated, and constructs a tobacco leaf yield estimation model based on tobacco field identification images and tobacco plant identification images, taking into account tobacco field edge pixels and tobacco plant candidate frames to determine tobacco field area and tobacco plant data, so as to estimate the tobacco field yield in the area to be estimated, which has higher accuracy and scenario adaptability than existing estimation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0050] Figure 2 The figure is a schematic diagram of the process of an embodiment of a method for estimating the yield of a large area of ​​tobacco field according to the present invention.

[0051] Figure 3 This is an example diagram of tobacco field sample annotation in an embodiment of the present invention.

[0052] Figure 4 This is an example diagram of the labeling of tobacco plant samples in an embodiment of the present invention.

[0053] Figure 5 This is a diagram of tobacco field identification results in an embodiment of the present invention.

[0054] Figure 6 This is a diagram of the tobacco plant identification results in an embodiment of the present invention.

[0055] Figure 7 This is a tobacco field yield estimation diagram in an embodiment of the present invention.

[0056] Figure 8 The present invention is a block diagram of a large-area tobacco field yield estimation device according to an embodiment of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0060] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art will understand that Figure 1 The structure of the device shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0062] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a large-area tobacco field yield estimation program.

[0063] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and perform data communication with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and perform data communication with the client; and the processor 1001 can be used to call the large-area tobacco field yield estimation program stored in the memory 1005 and perform the following operations:

[0064] Constructing a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations;

[0065] The tobacco field sample data set and the tobacco plant sample data set are respectively input into a disguised target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0066] Obtaining the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves;

[0067] The tobacco leaf orthophoto image is identified by using the tobacco field identification model and the tobacco plant identification model to obtain a tobacco field identification image and a tobacco plant identification image of the yield estimation area; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames;

[0068] Determine the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter;

[0069] Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, a tobacco leaf yield estimation model is constructed, and the tobacco field yield in the area to be estimated is estimated using the tobacco leaf yield estimation model.

[0070] The specific embodiments of the present invention applied to the device are basically the same as the embodiments of the method for estimating the yield of a large area of ​​tobacco field described below, and will not be described in detail here.

[0071] The embodiment of the present invention provides a method for estimating the yield of a large area of ​​tobacco field, referring to Figure 2 , Figure 2 The figure is a schematic diagram of the process of an embodiment of a method for estimating the yield of a large area of ​​tobacco field according to the present invention.

[0072] In this embodiment, the method for estimating the yield of a large area of ​​tobacco field comprises the following steps:

[0073] Step S100, constructing a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a number of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a number of orthophoto images of tobacco planting areas with tobacco plant annotations.

[0074] Specifically, in this embodiment, an orthophoto image of a tobacco-growing area collected by a drone is obtained; wherein the orthophoto image of the tobacco-growing area includes each key period of the tobacco growth process; the tobacco fields in the orthophoto image of the tobacco-growing area are annotated, and a tobacco field sample data set is constructed using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a polygonal semi-automatic annotation tool; the tobacco plants in the orthophoto image of the tobacco-growing area are annotated, and a tobacco plant sample data set is constructed using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a rectangular frame annotation tool; the sample data set is randomly extracted in proportion and divided into a training data set and a validation data set.

[0075] In practical applications, this embodiment uses the DJI Phantom 4 RTK UAV to go to areas known to be planted with tobacco for aerial photography, and obtains JPEG tobacco orthophotos with a resolution of 4864×3648 (4:3) at different altitudes. The collected orthophotos include four key growth periods of tobacco: seedling stage, root extension stage, vigorous growth stage, and maturity stage, which helps to improve the accuracy and generalization ability of the model, so that it can be better applied to actual tobacco growth monitoring and analysis tasks.

[0076] According to experimental tests, the optimal height for drone collection is 50-80m, at which the orthophoto images collected can clearly distinguish tobacco plants.

[0077] Furthermore, the present embodiment further includes labeling the tobacco fields and tobacco plants in the orthophoto image, and constructing a sample data set using the labeled orthophoto image.

[0078] In practical applications, 9857 orthophoto images with acquisition heights between 50 and 80 m were selected for annotation, such as Figure 3 As shown in the figure, when marking tobacco fields, the polygon semi-automatic labeling tool AnyLabeling-GPU is used to label each plot along the edge of the plot to ensure that the boundary of the entire plot is completely depicted and completely included in the labeled area to prevent the edge of the plot from being missed or incompletely labeled; Figure 4 As shown in the figure, when annotating tobacco plants, use the rectangular box annotation tool Labelimg to select the tobacco plants, ensuring that each rectangular box completely contains the entire tobacco plant to avoid any part being missed or truncated; then randomly select 80% of the annotated orthophoto samples as a training data set and 20% as a verification data set.

[0079] Step S200 , inputting the tobacco field sample data set and the tobacco plant sample data set into a disguised target instance segmentation model and a small target detection model for training respectively, to obtain a tobacco field recognition model and a tobacco plant recognition model.

[0080] Specifically, in this embodiment, the camouflaged target instance segmentation model is trained using the tobacco field sample training data set, and the trained model is verified using the tobacco field sample verification data set; wherein the camouflaged target instance segmentation model adopts the OSFormer model; the small target detection model is trained using the tobacco plant sample training data set, and the trained model is verified using the tobacco plant sample verification data set; wherein the small target detection model adopts the CEASC model.

[0081] In practical applications, the OSFormer model is used to train the tobacco field sample data set. The specific steps include: dividing the randomly selected tobacco field sample training data set and the tobacco field sample verification data set orthophoto images into folders, inputting the tobacco field sample training data set into the OSFormer network model for training, first updating the weight file of the OSFormer original model to the weight of the tobacco field plot recognition model, and then after repeated training, a model with an accuracy of 30.2% is obtained, and then the model structure is optimized, the learning rate in the optimizer parameters is modified to be 1e-05, the weight decay is 0.0001, and the learning rate is 0.0001. The model parameters are modified such as the hidden layer dimension of 256, the number of heads of the multi-head attention mechanism is 8, the number of key points is 1, and the sigma value of non-maximum suppression is 2.0. The cross entropy loss coefficient in the optimized loss function is 2.0, the mask focus loss coefficient is 2.0, the edge focus loss coefficient is 2.0, and other loss function related parameters. Finally, the tobacco field recognition model is output for tobacco field plot recognition, and its recognition accuracy can reach 85.87%.

[0082] In practical applications, the CEASC model is used to train the tobacco plant sample data set. The specific steps include: dividing the randomly selected tobacco plant sample training data set and the tobacco plant sample verification data set orthophoto images into folders, inputting the tobacco plant sample training data set into the CEASC network model for training, selecting the weight file of the swin_tiny model, using 4 threads for data loading to improve the model training speed, the pooling layer scale in the model is [1,2,3,6], the classification loss coefficient is 10, the point loss coefficient is 2, the Tversky loss coefficient is 2, and the learning rate lr is set to 1e-5 to avoid excessive learning rate leading to unfitting training results, too small learning rate leading to increased training time cost and falling into the local optimal solution, it is difficult to jump out of the global optimal solution and adopt a method of dynamically adjusting the learning rate, the number of learning rate reduction cycles is 50, and the training effect is further improved. After repeated training, a model with an accuracy of 86.3% is obtained, and finally the tobacco plant model is used for tobacco plant recognition.

[0083] Step S300, obtaining the tobacco leaf orthophoto image of the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extracting the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image.

[0084] Specifically, in this embodiment, the tobacco leaf maturity period is used as the period for yield estimation, image control points are arranged in the area to be estimated, and the coordinate data of the sampling points are obtained by a positioning instrument; the drone obtains an orthophoto image of the entire area to be estimated; the tobacco leaves of the tobacco plants picked at the sample points are handed over to the tobacco curing station for baking, and the tobacco leaf yield data of the tobacco plants is the tobacco leaf yield obtained from the tobacco curing station; the longitude and latitude coordinates of the waypoints when the drone is collecting are extracted as the longitude and latitude coordinates of the center point of the orthophoto image.

[0085] In actual application, from late July to mid-to-early August, some tobacco-growing areas in Daguan County, Zhaotong City were selected to carry out large-scale tobacco field yield estimation.

[0086] During the tobacco leaf maturity period, the DJI Phantom 4 RTK UAV is used to collect orthophoto images of the entire area to be estimated. Before the collection, the image control points are evenly arranged in the area to be estimated, and the positions of the image control points are clear and easy to identify. Then the RTK positioning instrument is used to obtain the coordinate data of the selected sampling points, specifically to obtain the coordinate data of the center point of the tobacco plant at the sampling point, and each sampled tobacco plant is numbered. After the drone collects the orthophoto image, the tobacco leaves of the tobacco plants at the sampling points are picked, and the mature tobacco leaves are collected according to the tobacco leaf picking and curing specifications and methods. They are handed over to the tobacco curing station for curing, and the tobacco curing station provides tobacco leaf yield data, which corresponds one by one to the sampled tobacco plant number. The longitude and latitude coordinates of the waypoints when the drone collects the orthophoto image are extracted as the longitude and latitude coordinates of the center point of the orthophoto image.

[0087] Step S400, using the tobacco field identification model and the tobacco plant identification model to identify the tobacco leaf orthophoto image, to obtain a tobacco field identification image and a tobacco plant identification image of the area to be estimated; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames.

[0088] Specifically, in this embodiment, if Figure 5 As shown, the tobacco field recognition image after the recognition is completed includes a series of tobacco field edge pixel points; Figure 6 As shown, the cigarette plant identification image includes a cigarette plant candidate frame.

[0089] Step S500, using the tobacco field edge pixel points and the tobacco plant candidate frame, determine the tobacco field area and tobacco plant data; wherein the tobacco plant data includes the longitude and latitude coordinates of the center point of the tobacco plant and the diameter of the tobacco plant.

[0090] Specifically, in this embodiment, according to the camera parameters of the drone and the flight altitude, the actual ground distance corresponding to the unit pixel on the single orthophoto image is calculated; the offset of the edge pixel point of the tobacco field relative to the center point of the orthophoto image is converted from the pixel distance to the actual ground distance; the pixel coordinates of the center point and the four vertices of the tobacco plant candidate frame are extracted, and the longitude and latitude coordinates of the center point and the four vertices of the tobacco plant candidate frame are calculated based on the longitude and latitude coordinates of the center point of the orthophoto image, and the longitude and latitude coordinates of the edge pixel point of the target tobacco field are calculated; Figure 5 As shown, the longitude and latitude coordinates of the edge pixels of the target tobacco field are assembled into a valid polygon object according to the order of pixels, and the area of ​​the polygon object is calculated to obtain the tobacco field area; Figure 6 As shown, the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame are assembled into a valid rectangle, the short side length of the rectangle is calculated, the tobacco plant diameter is obtained, and is associated with the tobacco plant candidate frame.

[0091] In practical applications, the actual ground distance corresponding to a unit pixel on a single orthophoto is calculated based on the sensor size, focal length, flight altitude, pixel size and photo ratio of the DJI Phantom 4 RTK drone. The specific calculation steps are as follows: First, the relationship between the field of view angle FOV and the sensor size and focal length is converted to: tan (FOV / 2) = (sensor size / 2) focal length. Then, based on the FOV and flight altitude, the actual hypotenuse length of a single orthophoto can be calculated as A = 2*tan (FOV / 2) * flight altitude. Then, based on A and the photo shooting ratio of 4:3, the Pythagorean theorem is used to calculate The actual length and width of the ground corresponding to the shooting range of a single orthophoto image, and finally, according to the pixel size of the camera, the actual ground distance corresponding to the unit pixel on the single orthophoto image is calculated; then the offset of the edge pixel point of the tobacco field relative to the center point of the orthophoto image is converted from pixel distance to actual ground distance; then the pixel coordinates of the center point and four vertices of the tobacco plant candidate frame are extracted, and based on the longitude and latitude coordinates of the center point of the orthophoto image, the conversion relationship between longitude and latitude and distance angle is used to calculate the longitude and latitude coordinates of the center point and four vertices of the tobacco plant candidate frame, and the longitude and latitude coordinates of the edge pixel point of the target tobacco field are calculated.

[0092] The longitude and latitude coordinates of the edge pixels of the target tobacco field are assembled into a valid polygonal object according to the order of pixels, and the area of ​​the polygonal object is calculated to obtain the area of ​​the tobacco field; the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame are assembled into a valid rectangle, and the length of the short side of the rectangle is calculated to obtain the diameter of the tobacco plant and associate it with the tobacco plant candidate frame.

[0093] Step S600, based on the tobacco yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, a tobacco yield estimation model is constructed, and the tobacco field yield in the area to be estimated is estimated using the tobacco yield estimation model.

[0094] Specifically, in this embodiment, the longitude and latitude coordinates of the center point of the tobacco plant and the coordinate data of the tobacco plant at the sampling point are matched to extract the diameter of the tobacco plant at the sampling point; the tobacco plant diameter at the sampling point and the tobacco leaf yield data are used to construct a tobacco leaf yield estimation model using the least squares linear regression machine learning algorithm, and the model evaluation index uses the determination coefficient R 2 The closer the value is to 1, the higher the accuracy. The model formula is as follows: yield = 52.445 + 0.761 * diameter, R 2 It is 0.949; among them, yield refers to the yield of tobacco leaves per single plant, which is the average yield per mu, in kilograms; diameter refers to the diameter of each tobacco plant, in cm.

[0095] After that, the tobacco field polygon object and the center point of the tobacco plant candidate frame are used to query the position relationship, and the tobacco plants outside the tobacco field polygon object are eliminated; the diameter of the tobacco plants within each tobacco field polygon is input into the yield estimation model to obtain the tobacco leaf yield of each tobacco plant in the estimated yield area. The average tobacco leaf yield of the tobacco plants contained in each tobacco field polygon is calculated and multiplied by the area of ​​the corresponding tobacco field to obtain the tobacco field yield in the estimated yield area, such as Figure 7 shown.

[0096] Reference Figure 8 , Figure 8 The figure is a structural block diagram of an embodiment of a large-area tobacco field yield estimation device of the present invention.

[0097] like Figure 8 As shown, the large-area tobacco field yield estimation device proposed in the embodiment of the present invention includes:

[0098] A construction module 10 is used to construct a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, wherein the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations;

[0099] A training module 20, for inputting the tobacco field sample data set and the tobacco plant sample data set into a disguised target instance segmentation model and a small target detection model for training, to obtain a tobacco field recognition model and a tobacco plant recognition model;

[0100] The extraction module 30 is used to obtain the tobacco leaf orthophoto image of the area to be estimated collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image;

[0101] The identification module 40 is used to identify the tobacco leaf orthophoto image using the tobacco field identification model and the tobacco plant identification model to obtain a tobacco field identification image and a tobacco plant identification image of the yield estimation area; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames;

[0102] The determination module 50 is used to determine the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter;

[0103] The estimation module 60 is used to construct a tobacco leaf yield estimation model based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, and use the tobacco leaf yield estimation model to estimate the tobacco field yield in the area to be estimated.

[0104] Other embodiments or specific implementations of the large-area tobacco field yield estimation device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0105] In addition, the present invention also proposes a large-area tobacco field yield estimation device, which includes: a memory, a processor, and a large-area tobacco field yield estimation program stored in the memory and executable on the processor, and when the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method as described above are implemented.

[0106] The specific implementation methods of the large-area tobacco field yield estimation device of the present application are basically the same as the various embodiments of the large-area tobacco field yield estimation method described above, and will not be repeated here.

[0107] In addition, the present invention also provides a readable storage medium, which includes a computer-readable storage medium on which a large-area tobacco field yield estimation program is stored. The readable storage medium may be Figure 1 The memory 1005 in the terminal may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The readable storage medium includes a number of instructions for enabling a large-area tobacco field yield estimation device with a processor to execute the large-area tobacco field yield estimation method described in each embodiment of the present invention.

[0108] The specific implementation methods in the readable storage medium of the present application are basically the same as the embodiments of the above-mentioned large-area tobacco field yield estimation method, and will not be repeated here.

[0109] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for estimating the yield of a large area of ​​tobacco field, characterized in that: The following steps are involved: Constructing a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations; The tobacco field sample data set and the tobacco plant sample data set are respectively input into a disguised target instance segmentation model and a small target detection model for training to obtain a tobacco field recognition model and a tobacco plant recognition model; Obtaining the orthophoto image of tobacco leaves in the area to be estimated, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points collected by the drone, and extracting the longitude and latitude coordinates of the center point of the orthophoto image of tobacco leaves; The tobacco leaf orthophoto image is identified by using the tobacco field identification model and the tobacco plant identification model to obtain a tobacco field identification image and a tobacco plant identification image of the yield estimation area; wherein the tobacco field identification image includes a plurality of tobacco field edge pixel points, and the tobacco plant identification image includes a plurality of tobacco plant candidate frames; Determine the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter; Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, a tobacco leaf yield estimation model is constructed, and the tobacco field yield in the area to be estimated is estimated using the tobacco leaf yield estimation model.

2. The method for estimating the yield of a large area of ​​tobacco field according to claim 1, characterized in that: The steps to construct a sample data set include: Obtaining an orthophoto image of a tobacco planting area collected by a drone; wherein the orthophoto image of the tobacco planting area includes each key period of the tobacco growth process; Annotating tobacco fields in the orthophoto image of the tobacco-growing area, and constructing a tobacco field sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a polygon semi-automatic annotation tool; Annotating tobacco plants in the orthophoto image of the tobacco-growing area, and constructing a tobacco plant sample data set using the annotated orthophoto image of the tobacco-growing area; wherein the annotation tool is a rectangular frame annotation tool; The sample data set is randomly selected in proportion and divided into a training data set and a validation data set.

3. The method for estimating the yield of a large area of ​​tobacco field as claimed in claim 2, characterized in that: The steps of inputting the tobacco field sample data set and the tobacco plant sample data set into the camouflaged target instance segmentation model and the small target detection model for training respectively to obtain the tobacco field recognition model and the tobacco plant recognition model specifically include: The disguised target instance segmentation model is trained using the training data set in the tobacco field sample data set, and the trained model is verified using the verification data set in the tobacco field sample data set; wherein the disguised target instance segmentation model adopts the OSFormer model; The small target detection model is trained using the training data set in the tobacco plant sample data set, and the trained model is verified using the verification data set in the tobacco plant sample data set; wherein the small target detection model adopts the CEASC model.

4. The method for estimating the yield of a large area of ​​tobacco field according to claim 1, characterized in that: The steps of obtaining the tobacco leaf orthophoto image of the area to be estimated collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extracting the longitude and latitude coordinates of the center point of the tobacco leaf orthophoto image specifically include: Taking the tobacco leaf maturity period as the period for yield estimation, image control points are arranged in the area to be estimated, and the coordinate data of the tobacco plants at the sampling points are obtained using positioning instruments; Obtain an orthophoto of tobacco leaves in the entire area to be estimated collected by a drone, and extract the longitude and latitude coordinates of the waypoints collected by the drone as the longitude and latitude coordinates of the center point of the orthophoto; The tobacco leaf yield data is obtained by obtaining tobacco leaves from tobacco plants picked at sampling points and handing them over to tobacco flue-curing stations for baking.

5. The method for estimating the yield of a large area of ​​tobacco field as claimed in claim 1, characterized in that: The step of determining the tobacco field area and tobacco plant data by using the tobacco field edge pixel points and the tobacco plant candidate frame specifically includes: Calculate the actual ground distance corresponding to a unit pixel on a single orthophoto of the tobacco leaf according to the camera parameters and the flight altitude of the drone; Converting the offset of the edge pixel point of the tobacco field relative to the center point of the tobacco leaf orthophoto image from pixel distance to actual ground distance; Extract the pixel coordinates of the center point and four vertices of the tobacco plant candidate frame, determine the latitude and longitude coordinates of the center point and four vertices of the tobacco plant candidate frame based on the latitude and longitude coordinates of the center point of the tobacco leaf orthophoto image, and calculate the latitude and longitude coordinates of the edge pixel points of the tobacco field; The longitude and latitude coordinates of the edge pixel points of the tobacco field are assembled into a valid polygonal object according to the order of pixels, and the area of ​​the polygonal object is calculated to obtain the area of ​​the tobacco field; the longitude and latitude coordinates of the four vertices of the tobacco plant candidate frame are assembled into a valid rectangle, and the length of the short side of the rectangle is calculated to obtain the diameter of the tobacco plant and associate it with the tobacco plant candidate frame.

6. The method for estimating the yield of a large area of ​​tobacco field as claimed in claim 5, characterized in that: Based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, the steps of constructing a tobacco leaf yield estimation model specifically include: Matching the longitude and latitude coordinates of the center point of the tobacco plant with the coordinate data of the tobacco plant at the sampling point, and extracting the diameter of the tobacco plant at the sampling point; The tobacco leaf yield estimation model was constructed using the tobacco plant diameter and tobacco leaf yield data at the sampling points and the least squares linear regression machine learning algorithm.

7. The method for estimating the yield of a large area of ​​tobacco field as claimed in claim 6, characterized in that: The steps of estimating the tobacco field yield in the area to be estimated by using the tobacco leaf yield estimation model specifically include: Using the center points of the polygonal object and the tobacco plant candidate frame to query the position relationship, and eliminating tobacco plants outside the polygonal object; The diameter of the tobacco plant within each polygonal object is input into the yield estimation model to obtain the tobacco leaf yield of each tobacco plant in the yield estimation area; The average tobacco leaf yield of the tobacco plants contained in each polygon object is calculated and multiplied by the area of ​​the corresponding tobacco field to obtain the tobacco field yield in the area to be estimated.

8. A large-area tobacco field yield estimation device, characterized in that: include: A construction module, used to construct a sample data set; wherein the sample data set includes a tobacco field sample data set and a tobacco plant sample data set, the tobacco field sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco field annotations, and the tobacco plant sample data set includes a plurality of orthophoto images of tobacco planting areas with tobacco plant annotations; A training module, used for inputting the tobacco field sample data set and the tobacco plant sample data set into a disguised target instance segmentation model and a small target detection model for training, so as to obtain a tobacco field recognition model and a tobacco plant recognition model; An extraction module is used to obtain the orthophoto image of tobacco leaves in the area to be estimated, collected by the drone, the tobacco leaf yield data and coordinate data of the tobacco plants at the sampling points, and extract the longitude and latitude coordinates of the center point of the orthophoto image of the tobacco leaves; A recognition module, used to recognize the tobacco leaf orthophoto image using the tobacco field recognition model and the tobacco plant recognition model, and obtain a tobacco field recognition image and a tobacco plant recognition image of the yield estimation area; wherein the tobacco field recognition image includes a plurality of tobacco field edge pixel points, and the tobacco plant recognition image includes a plurality of tobacco plant candidate frames; A determination module, used to determine the tobacco field area and tobacco plant data using the tobacco field edge pixel points and the tobacco plant candidate frame; wherein the tobacco plant data includes the longitude and latitude coordinates of the tobacco plant center point and the tobacco plant diameter; The estimation module is used to construct a tobacco leaf yield estimation model based on the tobacco leaf yield data of the tobacco plants at the sampling points, the coordinate data of the tobacco plants at the sampling points, the longitude and latitude coordinates of the center points of the tobacco plants and the diameter of the tobacco plants, and use the tobacco leaf yield estimation model to estimate the tobacco field yield in the area to be estimated.

9. A large-area tobacco field yield estimation device, characterized in that: The large-area tobacco field yield estimation device includes: a memory, a processor, and a large-area tobacco field yield estimation program stored in the memory and executable on the processor. When the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method as described in any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores a large-area tobacco field yield estimation program, and when the large-area tobacco field yield estimation program is executed by the processor, the steps of the large-area tobacco field yield estimation method according to any one of claims 1 to 7 are implemented.

Images