A field rice panicle recognition and yield prediction method based on a unmanned aerial vehicle and a point counting network
By acquiring images of rice paddies using drones and detecting rice panicles using an improved P2PNet network, and combining this with field sampling data to construct a yield prediction model, the problem of low efficiency and insufficient accuracy in traditional rice yield prediction was solved, achieving efficient and accurate rice yield prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG AGRI UNIV
- Filing Date
- 2024-08-30
- Publication Date
- 2026-07-24
AI Technical Summary
Traditional rice yield prediction methods suffer from problems such as untimely data collection, limited accuracy, and difficulty in covering large areas of farmland, resulting in low yield measurement efficiency and high subjectivity.
A method based on UAVs and point counting networks was adopted. Orthophotos of rice fields were acquired by UAVs, and image enhancement technology and an improved P2PNet network were used to detect and count rice panicles. A yield prediction model was constructed by combining field sampling data.
It significantly improves the efficiency and accuracy of rice yield prediction, reduces labor consumption, and is applicable to field yield measurement of rice in different regions and varieties, with good practicality and adaptability.
Smart Images
Figure CN119007047B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of smart agriculture, agricultural product yield prediction and estimation, and specifically to a method for field rice panicle identification and yield prediction based on unmanned aerial vehicles and point counting networks. Background Technology
[0002] Rice, as a crucial food crop in my country, is vital not only for the country's food security but also for global food stability. Rice yield forecasting is critical for agricultural decision-making, market supply and demand balance, and food security. Traditional rice yield estimation methods primarily rely on field surveys and statistical data, which are inherently subjective and subject to time lags. While these methods provide some yield estimation information, they suffer from drawbacks such as untimely data collection, limited accuracy, and difficulty in covering large areas of farmland. To improve the accuracy and efficiency of rice yield forecasting, this invention proposes a field-based rice panicle identification and yield forecasting method based on unmanned aerial vehicles (UAVs) and a point counting network. Summary of the Invention
[0003] (a) Technical problems to be solved The technical problem to be solved by this invention is to address the aforementioned problems and requirements by enabling rapid, high-throughput detection of rice paddies based on a drone platform, which can complete operations over a larger area within a unit of time, significantly improving data acquisition efficiency; by using a point-counting-based deep learning network, it is possible to quickly label and detect rice ears in the acquired images; this invention aims to achieve rapid and accurate yield prediction for rice in the field.
[0004] (II) Technical Solution To achieve the content described in this invention, the following solution is adopted.
[0005] A method for identifying rice panicles and predicting yield in the field based on unmanned aerial vehicles (UAVs) and point counting networks includes the following steps: S1: Using a drone equipped with a high-definition visible light camera, orthophotos are obtained from above the rice field after the rice has entered the heading stage; Specifically, in step S1, the UAV cruises along a route within a designated area, maintaining flight altitude, flight speed, and camera angle, and takes photos above the rice paddy at equal intervals to obtain discrete images that cover the entire area. The captured images are then stitched together to reconstruct a complete orthophoto image of the rice paddy. S2: The obtained orthophoto image of the rice field is enhanced using digital image processing and machine learning techniques to make the rice ears in the image more prominent. Specifically, in step S2, the image is preprocessed using OpenCV tools, the RGB channel values of each pixel in the image are read, and the pixel values of the image in the HSV and L*a*b color spaces are calculated. Then, machine learning methods are used to extract the characteristics of rice ears and determine the color combination that can most significantly reflect the characteristics of rice ears from the above 9 color features. S3: Improve and use a point-count-based object detection network P2PNet to detect rice ears in the acquired paddy field images and count the detection results; Specifically, in step S3, conventional target detection algorithms fail to achieve optimal detection results for small, high-density targets with significant overlap. Furthermore, conventional target detection algorithms still perform poorly in low-resolution, blurry images. Therefore, this invention improves upon the Triplet attention mechanism module and uses a point-counting-based target detection network, P2PNet, to detect and count rice ears in paddy field images. S4: Before rice matures and is harvested, a sampling experiment is conducted on rice in the field to collect and calculate the intrinsic parameters of rice grains of various varieties in different planting areas and the panicle density in the field. Specifically, in step S4, the intrinsic parameters of rice grains include parameters related to rice yield, namely, the average number of panicles per plant, the number of grains per panicle, the thousand-grain weight, and the seed setting rate; it also includes the actual panicle density in the rice field, namely, the actual number of rice panicles per unit area. S5: Based on the field rice panicle detection and counting results described in the above steps, combine them with the intrinsic parameters of rice obtained from field sampling and the regional panicle density to construct a mathematical formula for regional yield prediction; Specifically, in step S5, the theoretical shading rate of rice panicles in the area is calculated from the rice panicle detection and counting results in step S3 and the actual panicle density in the paddy field in step S4. The actual panicle distribution in the field is then deduced from this, and the intrinsic parameters of rice obtained from sampling are substituted to calculate the theoretical yield in the area.
[0006] (III) Beneficial Effects (1) This invention provides a method for calculating the theoretical yield of rice by taking orthophotos of rice fields with drones and directly predicting the yield. This method can replace the traditional manual yield measurement method, which significantly improves the yield measurement efficiency and reduces labor consumption. (2) The method described in this invention can be promoted and is applicable to the yield measurement of various rice fields. It can be used for a long time. For different regions and different varieties of rice, it is only necessary to retrain the deep learning model and manually perform a small amount of sampling. This method is highly practical and has good portability. (3) The point-count-based target detection network P2PNet modified in this invention makes up for the shortcomings of the current commonly used target detection algorithms for agricultural products and has a better computational effect in high-density, low-precision rice images. Attached Figure Description
[0007] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is an image showing the enhanced color of rice ears in a paddy field image as described in this invention. Figure 3 This is a structural diagram of the improved and used counting network P2PNet in this invention; Figure 4 The image shows the effect of using the method described in this invention to detect rice panicles in the field. Figure 5 This is a diagram illustrating the effect of using the method described in this invention to predict rice yield in a region. Detailed Implementation
[0008] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0009] The purpose of this invention is to propose a method for identifying rice panicles and predicting yield in the field based on unmanned aerial vehicles and point counting networks.
[0010] To achieve the above-mentioned objectives of the present invention, the features and advantages will become clearer and easier to understand. The present invention will be further described in detail below with reference to pictures and examples.
[0011] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0012] As attached Figure 1 As shown, this invention relates to a method for identifying rice panicles and predicting yield in the field based on unmanned aerial vehicles (UAVs) and point counting networks. The specific steps are as follows: S1: Using a drone equipped with a high-definition visible light camera, orthophotos are acquired from above the rice field after the rice has entered the heading stage. The method described in this invention captures images of the rice field after the rice has entered the heading stage, at which point the rice has fully headed and the quantity will not change significantly, which helps improve the accuracy of yield prediction. Simultaneously, at this stage, the characteristics of the rice panicles are distinct, and there is less overlap and occlusion between the panicles, which helps improve the accuracy of target detection. When the UAV equipment described in this invention collects images of rice paddies, it needs to first plan the area. After determining the area to be operated, the UAV's flight path is then planned within the area. The UAV equipment can autonomously cruise along the flight path. During the flight, the UAV's flight altitude, flight speed, and camera angle must remain unchanged. When taking pictures, the UAV uses an equal-interval shooting method, that is, it takes one picture at equal intervals above the rice paddy. After the flight path is completed, the positions of the captured image points can cover the entire operating area. At the same time, since the images captured by the UAV are discrete, remote sensing software is used to stitch the discrete images together to restore a complete orthophoto image of the rice paddy. S2: The obtained orthophoto image of the rice paddy is enhanced using digital image processing and machine learning techniques to make the rice panicle features in the image more prominent. The method described in this invention uses OpenCV to read the RGB channel values of each pixel in the image, and then calculates the channel values of each pixel in the HSV color space and L*a*b color space according to the pixel space conversion formula. Thus, each pixel in the image can be represented by nine color features. Furthermore, a certain amount of rice ear color features are extracted from the image. Using the value range of nine color features of rice ears as a standard, a machine learning classification method is used to compare the color feature value of each pixel in the image with the standard rice ear features. If the color feature value of a pixel is similar to that of a rice ear, that part of the image content is highlighted. This image content classification method enhances the rice ear features. (See attached image.) Figure 2 As shown, after enhancing the rice panicle features in the image using the method described in this invention, the color difference between the rice panicle and other unrelated parts is significantly increased, making the rice panicle more prominent. S3: An improved version of the P2PNet object detection network, based on point counting, is used to detect rice ears in acquired paddy field images and count the detection results. (See attached image) Figure 3 As shown, the method of this invention uses the point-counting-based object detection network P2PNet, which innovatively introduces P2PNet into the field of agricultural engineering, enabling the detection and counting of small objects such as rice ears in dense, low-resolution environments. At the same time, when creating the dataset required for P2PNet training, compared with the traditional bounding box annotation method, points are used to locate and annotate objects. This point annotation method significantly reduces the time required for annotation work, thereby greatly reducing time costs. As attached Figure 3As shown, the improvement to the object detection network P2PNet by the method described in this invention is implemented based on the Triplet attention mechanism module. The Triplet attention mechanism belongs to the channel-space attention mechanism, which focuses on the cross-dimensional interaction of the image in the three dimensions of height, width, and channels. The Triplet attention mechanism described in this invention consists of three parallel branches, two of which are used to establish cross-channel interaction between channel C and space H or space W, respectively. Taking the first branch as an example, given an input tensor with shape C×H×W, the input tensor is first rotated 90° counterclockwise along the H-axis, resulting in a tensor with shape W×H×C. Next, a Z-Pool layer reduces the tensor's dimensions to two, resulting in a tensor with shape 2×H×C. Then, a 7×7 convolutional layer and a normalization layer are used to obtain a tensor with shape 1×H×C. A Sigmoid activation layer is then used to generate attention weights. Finally, the output is rotated 90° clockwise along the H-axis to ensure the output shape matches the input shape. The last branch is used to calculate the attention weights for space H and space W. Finally, the outputs of the three branches are combined by a weighted average to obtain the output feature map.
[0013] As attached Figure 3 As shown, the improvement to the P2PNet network structure described in this invention is to integrate a Triplet attention mechanism module before the result regression prediction module in the original P2PNet network structure. This can improve the generalization ability of the model while increasing computational resources, and enhance the detection effect of small targets such as rice ears in rice field images of different sizes. S4: Before rice maturity and harvest, a field sampling experiment is conducted to collect and calculate the intrinsic parameters of rice grains of various varieties in different planting areas and the panicle density in the field. The method described in this invention requires sampling and testing of rice. Sampling is carried out before rice maturity and harvest. In the embodiment described in this invention, a total of 198 rice planting plots are divided, covering a variety of different rice varieties and different management measures. Sampling is carried out in different rice planting areas according to the rule of randomly sampling 10 rice plants. The sampled rice plants are manually counted, and the average number of panicles per plant in each area is calculated. After the count is completed, the individual rice plants are threshed, and the sampled rice grains are tested using a high-throughput rice seed testing machine to obtain the number of grains per panicle, thousand-grain weight, and seed setting rate of rice. Furthermore, in each planting area of the paddy field, a grid is divided with one square meter as the unit. Within the area, a five-point sampling method is used, that is, a grid area is taken from the perimeter and the middle of the planting area, and the total number of rice ears in the area is manually counted to characterize the overall rice ear density of the planting area. S5: Based on the field rice panicle detection and counting results described in the above steps, and combined with the intrinsic parameters of rice grains obtained from field sampling and the regional rice panicle density, a mathematical relationship for regional yield prediction is constructed. The method of this invention uses Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) as evaluation indicators for the accuracy of field rice panicle detection. The calculation formulas for each indicator are as follows:
[0014]
[0015]
[0016] In the formula, It represents the actual number of rice ears in the i-th image. This represents the count of rice ears in the i-th image, where N is the number of images in the test set. MAE and MAPE indicate the accuracy of the model's detection, while RMSE reflects the model's robustness. Lower values for MAE, MAPE, and RMSE indicate higher model accuracy and better performance. (See attached image.) Figure 4 The image shown is a result of detecting rice ears in an image. Furthermore, by combining the rice panicle density per unit area within the rice paddy planting area described in step S4 above with the rice panicle detection and counting results of the corresponding area image, the theoretical rice panicle occlusion rate F in the rice paddy is calculated, and its calculation formula is as follows:
[0017] In the formula, FP refers to the total number of rice ears detected in a rice planting area, TP refers to the actual number of rice ears counted manually per unit area in the area, S refers to the total area of the planting area, and the closer the value of F is to 1, the less rice ears are obscured in the area. Furthermore, by integrating the intrinsic parameters of rice grains obtained from variety testing, a rice yield prediction formula is established, as shown below:
[0018] In the formula, Y refers to the overall predicted yield of the rice paddy area, P refers to the total number of rice panicles detected in the image, C refers to the average number of grains per panicle per rice plant in the area, R refers to the average seed setting rate of the rice variety, and W refers to the average thousand-grain weight of the rice variety. Yield prediction was performed on 198 rice planting areas in the embodiments of this invention according to the method described in this invention. The fitting effect between the predicted results and the actual yield values is shown in the attached figure. Figure 5 As shown.
[0019] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.
Claims
1. A method for identifying rice panicles and predicting yield in the field based on unmanned aerial vehicles (UAVs) and point counting networks, the method is completed through the following steps: S1 uses a drone equipped with a high-definition visible light camera to acquire orthophotos from above the rice field after the rice has entered the heading stage. S2 uses digital image processing and machine learning techniques to enhance the obtained orthophotos of rice paddies, making the rice ears in the images more prominent. S3 improves upon and uses a point-counting-based object detection network, P2PNet, to detect rice ears in acquired paddy field images and count the detection results. The improvement and use of the point-counting-based object detection network P2PNet includes: In the original P2PNet network structure, a Triplet attention mechanism module is integrated before its result regression prediction module. This improves the model's generalization ability while increasing computational resources, and enhances the detection effect of small targets such as rice ears in images of different sizes. S4 Before rice matures and is harvested, a field sampling experiment is conducted on rice to collect and calculate the intrinsic parameters of rice grains of various varieties in different planting areas and the panicle density in the field. S5. Based on the field rice panicle detection and counting results in step S3 above, and combined with the intrinsic parameters of rice grains obtained from field sampling and the regional panicle density, a mathematical formula for regional yield prediction is constructed. The construction of the regional yield prediction mathematical formula includes: S5.1 First, the rice panicle density collected and calculated in step S4 and the number of rice panicles detected in step S3 are used to calculate the theoretical rice panicle shading rate F in the rice field. The formula for calculating the rice panicle shading rate F is as follows: In the formula, FP refers to the total number of rice ears detected in a rice planting area, TP refers to the actual number of rice ears counted manually per unit area in the area, S refers to the total area of the planting area, and the closer the value of F is to 1, the less rice ears are obscured in the area. S5.2 Integrate the intrinsic parameters of rice grains obtained in step S4 to establish the rice yield prediction formula Y. In the formula, Y refers to the overall predicted yield of the rice field area, P refers to the total number of rice panicles detected in the image, C refers to the average number of grains per panicle of a single rice plant in the area, R refers to the average seed setting rate of the rice variety, and W refers to the average thousand-grain weight of the rice variety.
2. The method for field rice panicle identification and yield prediction based on UAVs and point counting networks according to claim 1, characterized in that, The orthophoto image obtained from above the rice field after the rice has entered the heading stage, as described in S1, includes: S1.1 The rice field images are taken after the rice has entered the heading stage. At this time, the rice has fully headed and the number will not change significantly, which is conducive to improving the accuracy of yield prediction. At the same time, the characteristics of rice panicles are obvious at this stage, and there is less overlap and occlusion between rice panicles, which is conducive to improving the accuracy of target detection. When collecting images of rice paddies, the S1.2 UAV equipment needs to first plan the area. After determining the area to be operated, the UAV's flight path is then planned within the area. The UAV equipment can then autonomously cruise along the flight path. S1.3 During the flight of the drone, it is necessary to ensure that the drone's flight altitude, flight speed, and camera angle do not change; When taking pictures, the S1.4 UAV adopts the equal-interval shooting method, that is, it takes a picture at the same distance above the rice field. After the flight path is completed, the position of the captured image points can cover the entire operation area.
3. The method for field rice panicle identification and yield prediction based on UAVs and point counting networks according to claim 1, characterized in that, Image enhancement using digital image processing and machine learning techniques as described in S2 includes: S2.1 uses OpenCV to read the RGB channel values of each pixel in the image, and then calculates the channel values of each pixel in the image in the HSV color space and L*a*b color space according to the pixel space conversion formula. Each pixel in the image can then be expressed by 9 color features. S2.2 Extract a certain amount of rice ear color features from the image. Using the value range of the nine color features of rice ears as the standard, a machine learning classification method is used to compare the color feature value of each pixel in the image with the rice ear feature standard. If the color feature value of the pixel is similar to the color feature of the rice ear, then the image content of that part is highlighted.
4. The method for field rice panicle identification and yield prediction based on UAVs and point counting networks according to claim 1, characterized in that, The intrinsic parameters of rice grains and panicle density in the field for different rice varieties in different planting areas, as described in S4, include: S4.1 Intrinsic parameters of rice grains include parameters related to rice yield. Sampling is carried out in different rice planting areas according to the rule of randomly sampling 10 rice plants. The sampled rice plants are counted manually to obtain the average number of panicles per plant, number of grains per panicle, thousand-grain weight and seed setting rate. S4.2 In each planting area of the paddy field, grids are divided with one square meter as the unit. Within each area, a five-point sampling method is used, that is, a grid area is taken from the perimeter and the middle of the planting area, and the panicle density of the entire area is manually counted.