Field rice ear identification and yield prediction method based on unmanned aerial vehicle and point counting network

The orthograph image of rice is obtained through drones and the rice ear detection is performed using an improved point counting network. Combined with field sampling data, the problem of insufficient efficiency and accuracy of traditional rice yield prediction methods is solved, and fast and accurate yield prediction and high efficiency are achieved. production measurement process.

CN119007047AActive Publication Date: 2024-11-22HUAZHONG AGRI UNIV

Patent Information

Application Number
CN202411210108.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-22
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

Traditional rice yield prediction methods have problems such as untimely data collection, limited accuracy, and difficulty in covering large areas of farmland, resulting in low yield measurement efficiency and insufficient accuracy.

Method used

The field rice ear recognition method based on drone and point counting network is adopted to obtain orthophoto images through high-definition visible light cameras, combine digital image processing and machine learning technology to enhance image, and use the improved point count-based object detection network P2PNet Rice ear detection and field sampling data are combined to construct a mathematical relationship of yield prediction.

Benefits of technology

It significantly improves the efficiency and accuracy of rice yield prediction, can quickly and accurately cover large areas of farmland, reduce labor consumption, and is suitable for rice in different regions and varieties. The improved target detection network is performed in high-density and low-precision images. good.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure FT_1
    Figure FT_1
  • Figure FT_2
    Figure FT_2
  • Figure FT_3
    Figure FT_3
Patent Text Reader

Abstract

The invention discloses a field rice panicle identification and yield prediction method based on an unmanned aerial vehicle and a point counting network, and the method mainly comprises the steps: obtaining an orthoimage from the upper part of a rice field through employing unmanned aerial vehicle equipment carrying a high-definition visible light camera after the rice enters a full heading stage; processing the collected image, and performing color enhancement on rice ears in the image; improving and using a point counting-based target detection network P2PNet, and detecting and counting rice ears in the obtained rice field image; and constructing a regional yield prediction mathematical relationship according to a field rice panicle detection counting result in combination with rice evidence parameters obtained by field sampling. According to the method, rapid high-throughput detection can be carried out on the rice field based on the unmanned aerial vehicle platform, operation in a larger area range can be completed in unit time, and the rice field yield prediction efficiency is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of smart agriculture and agricultural product yield prediction and estimation, and in particular to a method for identifying rice ears and predicting yield in a field based on an unmanned aerial vehicle and a point counting network. Background Art

[0002] As an important food crop in my country, the production of rice is not only related to my country's food security, but also has great significance for global food stability. The prediction of rice yield is crucial to agricultural decision-making, market supply and demand balance and food security. The traditional method of rice yield estimation is mainly based on field surveys and statistical data, which has certain subjectivity and time lag. Although the traditional method provides information on yield estimation to a certain extent, it has the disadvantages of untimely data collection, limited accuracy and difficulty in covering large areas of farmland. In order to improve the accuracy and efficiency of rice yield prediction, the present invention proposes a field rice ear recognition and yield prediction method based on unmanned aerial vehicle and point counting network. Summary of the invention

[0003] 1. Technical issues to be resolved The technical problem to be solved by the present invention is to address the above-mentioned problems and requirements. Based on the UAV platform, rapid high-throughput detection of rice fields can be carried out, and operations in a larger area can be completed within a unit of time, which significantly improves the data collection efficiency. With the help of a deep learning network based on point counting, rapid labeling and detection of rice ears in the collected images can be achieved. The present invention aims to achieve rapid and accurate prediction of field rice yield.

[0004] (II) Technical solution In order to realize the contents of the present invention, the following scheme is adopted.

[0005] A method for identifying rice ears and predicting yield in a field based on drones and point counting networks specifically includes the following steps: S1: Use a drone equipped with a high-definition visible light camera to obtain orthophoto images from above the rice field after the rice enters the heading stage; Specifically, in step S1, the drone cruises along a route in a designated area, maintains a flight altitude, a flight speed, and a camera angle, and takes pictures above the rice field at equal intervals to obtain discrete images sufficient to cover the entire area, and stitches the images taken to restore a complete orthophoto image of the rice field; S2: The acquired orthophotos of rice fields are enhanced using digital image processing, machine learning and other related technologies to make the rice ears in the images more obvious; Specifically, in step S2, the image is preprocessed using the OpenCV tool, the RGB channel value of each pixel of the image is read, and the pixel values ​​of the image in the HSV and L*a*b color spaces are calculated, and then the rice ear features are extracted using a machine learning method, and the color combination that can most significantly reflect the rice ear features is determined from the above 9 color features; S3: Improve and use a point counting-based object detection network P2PNet to detect rice ears in the acquired rice field images and count the detection results; Specifically, in step S3, the conventional target detection algorithm cannot achieve the optimal detection effect for small targets, high density targets, and targets with large overlapping coverage, and the conventional target detection algorithm still performs poorly in blurry images with low resolution. Therefore, the present invention improves and uses a point counting-based target detection network P2PNet based on the Triplet attention mechanism module to detect and count rice ears in rice field images; S4: Before rice matures and is harvested, a sampling test is conducted on rice in the field to collect and calculate the intrinsic parameters of rice grains of various varieties in different planting areas and the ear density in the field; Specifically, in step S4, the intrinsic parameters of rice grains include parameters related to rice yield, namely, the average number of panicles per plant, the number of grains per panicle, the thousand-grain weight and the fruit setting rate of rice; and also include the actual panicle density in the rice field, namely, the actual number of rice panicles per unit area; S5: Based on the rice ear detection and counting results in the above steps, the intrinsic parameters of rice obtained by field sampling and the regional ear number density are combined to construct a mathematical relationship for regional yield prediction; Specifically, in step S5, the theoretical occlusion rate of rice ears in the regional area is calculated based on the rice ear detection and counting results described in step S3 and the actual ear number density in the rice field described in step S4, and the actual ear number distribution in the field is inferred from it, and then the rice intrinsic parameters obtained by sampling are substituted to calculate the theoretical yield in the region.

[0006] (III) Beneficial effects (1) The present invention provides a method for directly predicting the theoretical yield of rice by using an unmanned aerial vehicle to take orthophoto images of rice fields. This method can replace the traditional manual yield measurement method, significantly improve the yield measurement efficiency and reduce labor consumption; (2) The method of the present invention can be promoted and applied to various rice field yield measurements, and can be used for a long time. For rice from different regions and varieties, it only needs to retrain the deep learning model and manually perform a small amount of sampling work. The method is highly practical and portable. (3) The point counting-based target detection network P2PNet modified and used in the present invention makes up for the shortcomings of the current commonly used target detection algorithms for agricultural products and has better calculation effects in high-density and low-precision rice images. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 This is a diagram showing the color enhancement effect of rice ears in the rice field image of the present invention; Figure 3 The structure diagram of the counting network P2PNet improved and used by the present invention; Figure 4 This is a diagram showing the effect of using the method of the present invention to detect rice ears in the field; Figure 5 This is a diagram showing the effect of using the method of the present invention to predict rice yield in a region. DETAILED DESCRIPTION

[0008] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0009] The purpose of the present invention is to propose a method for identifying rice ears and predicting yield in the field based on unmanned aerial vehicles and point counting networks.

[0010] In order to achieve the above-mentioned purpose of the present invention and make the features and advantages more clearly understood, the present invention is further described in detail below with reference to pictures and examples.

[0011] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0012] As attached Figure 1 As shown, the present invention relates to a method for identifying rice ears and predicting yield in a field based on a drone and a point counting network, and the specific steps are as follows: S1: Use a drone equipped with a high-definition visible light camera to obtain orthophoto images from above the rice field after the rice enters the heading stage. The method of the present invention takes images of the rice field after the rice enters the heading stage. At this time, the rice has fully headed and the number will not change significantly, which is conducive to improving the yield prediction accuracy; at the same time, the rice ears at this period have obvious characteristics, and there is less overlap and occlusion between the rice ears, which is conducive to improving the accuracy of target detection; When the drone equipment of the present invention collects images of rice fields, it is necessary to first carry out regional planning. After determining the area to be operated, the flight route of the drone is then planned within the area, and the drone equipment can autonomously perform cruise flight according to the flight route; during the flight of the drone, it is necessary to ensure that the flight altitude, flight speed and camera angle of the drone do not change; when the drone is shooting, an equidistant shooting method is adopted, that is, an image is shot at the same distance above the rice field. After the flight of the route is completed, the position of the image points shot can cover the entire operation area. At the same time, since the images shot by the drone are discrete, the discretized images need to be spliced ​​with the help of remote sensing software to restore the complete orthophoto image of the rice field; S2: The acquired orthophoto of the rice field is enhanced by using digital image processing and machine learning and other related technologies to make the features of the rice ears in the image more obvious. The method of the present invention uses the OpenCV tool to read the RGB channel value of each pixel in the image, and then calculates the channel value of each pixel in the image in the HSV color space and the L*a*b color space according to the pixel space conversion formula. Each pixel in the image can be expressed by 9 color features; Furthermore, a certain amount of rice ear color features in the image is extracted, and the value range of the nine color features of the rice ear is used as the standard. The color feature value of each pixel in the image is compared with the rice ear feature standard using the machine learning classification method. If the color feature value of the pixel is close to the color feature value of the rice ear, the part of the image content is highlighted, and the rice ear features are enhanced by the image content classification method. Figure 2 As shown, after the rice ear features in the image are enhanced using the method of the present invention, the color difference between the rice ear and other irrelevant parts is significantly increased, making the rice ear more obvious; S3: Improve and use a point counting-based target detection network P2PNet to detect rice ears in the acquired rice field images and count the detection results. Figure 3 As shown, the method of the present invention uses a point counting-based target detection network P2PNet, innovatively introduces P2PNet into the field of agricultural engineering, and realizes the detection and counting of small target objects such as rice ears in a dense and low-definition environment; at the same time, when preparing the data set required for P2Pnet training, compared with the traditional bounding box annotation method, points are used to locate and annotate objects. This point annotation method significantly reduces the time required for annotation work, thereby greatly reducing the time cost; As attached Figure 3As shown, the improvement of the target detection network P2PNet by the method of the present invention is implemented based on the Triplet attention mechanism module. The Triplet attention mechanism belongs to the channel-space attention mechanism. The Triplet attention mechanism focuses on the cross-dimensional interaction of the image in three dimensions: height, width and channel. The Triplet attention mechanism of the present invention consists of three parallel branches, two of which are used to establish cross-channel interaction between channel C and space H or space W respectively. Taking the first branch as an example, given the shape of the input tensor is C×H×W, first rotate the input tensor 90° counterclockwise along the H axis, and the shape of the rotated tensor is W×H×C; secondly, the dimension of the tensor is reduced to two dimensions through the Z-Pool layer, and the shape of the tensor is 2×H×C; then, through the convolution layer with a convolution kernel size of 7×7 and the normalization layer, a tensor with a shape of 1×H×C is obtained, and then the attention weight is generated through the Sigmoid activation layer; finally, the output is rotated 90° clockwise along the H axis to ensure that the output is consistent with the input shape, and the last branch is used to calculate the attention weights of space H and space W; finally, the outputs of the three branches are merged by weighted average to obtain the output feature map.

[0013] As attached Figure 3 As shown, the improvement of the P2PNet network structure described in the present invention is to integrate the Triplet attention mechanism module before the result regression prediction module in the original P2PNet network structure, which can improve the generalization ability of the model while increasing less computing resources, and improve the detection effect of the small target of rice ears in rice field images of different sizes; S4: Before the rice matures and is harvested, a sampling test is conducted on the rice in the field, and the intrinsic parameters of the rice grains of each variety in different planting areas and the ear density in the field are collected and calculated. The method described in the present invention requires sampling and measuring the rice, and the sampling is carried out before the rice matures and is harvested. The embodiment described in the present invention divides a total of 198 rice planting plots, covering a variety of different rice varieties and different management measures. Sampling is carried out in different rice planting areas according to the rule of randomly sampling 10 rice plants, and the sampled rice plants are manually counted, and the average number of ears of rice per plant in each area is counted; after the statistics are completed, the single rice plant is threshed, and the sampled rice grains are tested using a high-throughput rice seed testing machine to obtain the number of rice grains per ear, thousand-grain weight and fruit set rate; Furthermore, in each planting area of ​​the rice field, a grid is divided into units of one square meter, and a five-point sampling method is used in the area, that is, a grid area is taken from the four sides and the middle of the planting area, and the total number of rice ears is manually counted, thereby characterizing the overall rice ear density of the planting area; S5: Based on the rice ear detection and counting results in the above steps, the rice grain intrinsic parameters and regional rice ear density obtained by field sampling are combined to construct a mathematical relationship for regional yield prediction. The method of the present invention uses the mean absolute error MAE, the root mean square error RMSE and the mean absolute percentage error MAPE as evaluation indicators for the accuracy of rice ear detection in the field, and the calculation formulas of each indicator are as follows:

[0014]

[0015]

[0016] In the formula, is the actual number of rice ears in the i-th image, is the counting result of rice ears in the i-th image, and N is the number of images in the test set; MAE and MAPE indicate the accuracy of model detection, and RMSE reflects the robustness of the model; the lower the values ​​of MAE, MAPE, and RMSE, the higher the accuracy of model detection and the better the performance; as shown in the attached figure Figure 4 As shown, it is the effect diagram of detecting rice ears in the image; Furthermore, the rice ear density per unit area in the rice field planting area described in step S4 is combined with the rice ear detection and counting result of the corresponding area image to calculate the theoretical rice ear occlusion rate F in the rice field, and the calculation formula is as follows:

[0017] In the formula, FP refers to the total number of rice ears detected in a rice planting area, TP refers to the actual number of rice ears counted manually per unit area in the area, and S refers to the total area of ​​the planting area. The closer the value of F is to 1, the less occlusion of rice ears in the area. Furthermore, the rice grain intrinsic parameters obtained from the test were integrated to establish a rice yield prediction formula as shown below:

[0018] In the formula, Y refers to the predicted yield of the rice field as a whole, P refers to the total number of rice panicles detected in the image, C refers to the average number of grains per panicle per rice plant in the region, R refers to the average fruit set rate of the rice variety, and W refers to the average thousand-grain weight of the rice variety. The yield of the 198 rice-growing areas in the embodiment of the present invention is predicted according to the method of the present invention. The fitting effect of the predicted results and the actual yield value is shown in the attached figure. Figure 5 shown.

[0019] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in the field. Although the illustrative specific embodiments of the present invention are described above to facilitate the understanding of the present invention by the technical personnel in the field, it should be clear that the present invention is not limited to the scope of the specific embodiments. For ordinary technicians in the field of this technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for identifying and predicting rice ears in the field based on drones and point counting networks. The method of the present invention is based on a drone platform to perform rapid high-throughput detection of rice fields, and can complete operations in a larger area within a unit time. At the same time, with the help of a deep learning network based on point counting, the rice ears in the collected images can be quickly marked and detected. The method is completed by the following steps: S1 uses a drone equipped with a high-definition visible light camera to obtain orthophoto images from above the rice field after the rice enters the heading stage; S2 uses digital image processing, machine learning and other related technologies to enhance the acquired orthophotos of rice fields, making the rice ears in the images more obvious; S3 improves and uses a point counting-based target detection network P2PNet to detect rice ears in the acquired rice field images and count the detection results; S4 Before rice matures and is harvested, a sampling test is conducted on rice in the field to collect and calculate the intrinsic parameters of rice grains of various varieties in different planting areas and the ear density in the field; S5: constructing a mathematical relationship for regional yield prediction based on the rice ear detection and counting results in the field described in the above steps, combined with the rice grain intrinsic parameters obtained by field sampling and the regional rice ear density.

2. The method for identifying rice ears and predicting yield in the field based on drones and point counting networks according to claim 1, characterized in that: As described in S1, after the rice enters the heading stage, the orthophoto image is obtained from above the rice field: S1.1: Shoot rice field images after the rice enters the heading stage. At this time, the rice has fully headed and the number will not change significantly, which is conducive to improving the yield prediction accuracy. At the same time, the rice ears have obvious characteristics during this period, and there is less overlap and occlusion between the rice ears, which is conducive to improving the accuracy of target detection. S1.2 When the UAV equipment collects images of rice fields, it is necessary to first carry out regional planning. After determining the area to be operated, the flight route of the UAV is planned within the area. The UAV equipment can autonomously cruise according to the flight route; S1.3 During the flight of the drone, ensure that the flight altitude, flight speed and camera angle of the drone do not change; When taking pictures, the S1.4 UAV adopts an equidistant shooting method, that is, taking an image at the same distance above the rice field. After the flight route is completed, the positions of the image points taken can cover the entire working area.

3. The method for identifying rice ears and predicting yield in the field based on drones and point counting networks according to claim 1, characterized in that: Image enhancement using digital image processing and machine learning techniques as described in S2: S2.1 uses OpenCV tools to read the RGB channel value of each pixel in the image, and then calculates the channel value of each pixel in the image in the HSV color space and L*a*b color space according to the pixel space conversion formula. Each pixel in the image can be expressed by 9 color features; S2.2 extracts a certain amount of rice ear color features in the image, takes the value range of the 9 color features of rice ears as the standard, and uses the machine learning classification method to compare the color feature value of each pixel in the image with the rice ear feature standard. If the color feature value of the pixel is close to the color feature of the rice ear, then this part of the image content is highlighted.

4. The method for identifying rice ears and predicting yield in the field based on drones and point counting networks according to claim 1, characterized in that: The improvements described in S3 use a point counting based object detection network P2Pnet: S3.1 uses the point counting-based object detection network P2PNet, innovatively introducing P2PNet into the field of agricultural engineering, and realizes the detection and counting of small target objects such as rice ears in dense and low-definition environments; S3.2's improvement on the target detection network P2PNet is based on the Triplet attention mechanism module. In the original P2PNet network structure, the Triplet attention mechanism module is integrated before its result regression prediction module. This can improve the generalization ability of the model while increasing fewer computing resources, and improve the detection effect of small targets such as rice ears in images of different sizes.

5. The method for identifying rice ears and predicting yield in the field based on drones and point counting networks according to claim 1, characterized in that: The collection and calculation of the intrinsic parameters of rice grains of various varieties in different planting areas and the ear density in the field as described in S4: S4.1 The intrinsic parameters of rice grains include parameters related to rice yield. The method of the present invention is to sample 10 rice plants at random in different rice planting areas, and manually count the sampled rice plants to obtain the average number of rice ears, number of grains per ear, thousand-grain weight and fruit setting rate; S4.2 In each rice planting area, a grid is divided into units of one square meter. Within the area, a five-point sampling method is used, that is, a grid area is taken around the planting area and in the middle, and the density of ears in the entire area is manually counted.

6. The method for identifying rice ears and predicting yield in the field based on drones and point counting networks according to claim 1, characterized in that: The mathematical relationship for constructing regional yield prediction as described in S5 is: S5.1 First, the density of rice ears per unit area of ​​the rice field and the number of rice ears detected in the corresponding area image are calculated to obtain the theoretical rice ear occlusion rate F in the rice field 7.S5.2 Further, the rice grain intrinsic parameters obtained from the test are integrated to establish the rice yield prediction formula Y 。

Citation Information

Patent Citations

  • Theoretical yield measurement method based on field rice canopy spike image characteristics

    CN110619417A

  • Field wheat ear counting and wheat yield prediction based on deep learning network

    CN113222991A

  • Wheat ear detection method based on improved YOLOv8s

    CN117333863A

  • Rice ear detection counting and rice yield estimation method based on unmanned aerial vehicle

    CN117853961A

  • Method and device for identifying and counting rice ear grains and electronic equipment

    CN118470547A

Cited By

  • Method and device for identifying nickel shots according to particle sizes

    CN121353252A