Unmanned aerial vehicle image rice ear orientation identification method based on deep learning
By adopting a deep learning-based rice ear orientation recognition method in drone image recognition, combining angle information and small-scale detection layer, and using a directional detection frame instead of the traditional detection frame, the problems of low rice ear recognition efficiency and difficulty in dealing with small sizes and shading are solved, and more efficient and accurate rice ear recognition is achieved.
Patent Information
- Application Number
- CN202411562457.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-03-25
AI Technical Summary
The rice ear recognition method in existing drone images is inefficient and costly, and it is difficult to cope with the problems of small size of rice ears, susceptible to occlusion and background interference.
The directional recognition method of rice ears based on deep learning is adopted. By combining the angle information of rice ears and a small-scale detection layer, the directional detection frame is used to replace the traditional horizontal detection frame to achieve accurate identification of rice ears.
It improves the accuracy and efficiency of rice ear recognition, has a wider coverage and lower cost, and can more accurately reflect the spatial location and direction of rice ears, especially in densely distributed or complex field environments.
Smart Images
Figure CN119693821A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of smart agricultural technology, and more specifically, to a method for directional recognition of rice ears using drone images based on deep learning. Background Art
[0002] At present, the commonly used methods for rice ear recognition are mainly divided into three categories: First, the traditional manual survey method relies on manual sampling, but this method is inefficient and costly, especially when human resources are limited and the sampling area is large, it is easy to produce sampling errors, which in turn affects the final statistical results. Secondly, the recognition method based on machine learning uses algorithms such as Bayesian classifiers, support vector machines and random forests to identify rice ears through color, texture and morphological features. However, this type of method is highly dependent on the prior knowledge of experts for the selection of features, has limited generalization ability, and is difficult to cope with changes in object size, displacement and rotation, especially when the rice ears are densely distributed or severely occluded in the field. The performance is poor. Finally, as an emerging technology, the deep learning method can automatically learn more realistic features from the data without relying on manually designed features, overcoming the limitations of traditional methods, which is also the technology used in the present invention.
[0003] However, since rice ears in drone images are often small in size and are easily affected by occlusion and background interference, there is still a lack of rice ear recognition methods for drone images.
[0004] Therefore, the present invention provides a method for rice ear orientation recognition using drone images based on deep learning, which improves the above-mentioned technical problems. Summary of the invention
[0005] The disclosed embodiments aim to address the deficiencies in the prior art and provide a method for directional identification of rice ears using drone images based on deep learning. The present invention combines drone images with the angle information of rice ears and a small-scale detection layer, and adopts a directional detection frame to replace the traditional horizontal detection frame, thereby achieving accurate identification of rice ears.
[0006] The above technical objectives of the present invention are achieved through the following technical solutions: A method for identifying rice ears oriented by drone images based on deep learning, comprising the following steps: S1, collecting rice ear image data by drone;
[0007] S2, preprocessing the collected rice ear image data;
[0008] S3. First, a rice ear orientation recognition model is constructed. The preprocessed rice ear image data is extracted and classified through the rice ear orientation recognition model to complete the processing and expression of the rice ear orientation features. Then, based on the positioning loss of the weighted intersection-over-union ratio, the network loss of the rice ear boundary box and the prediction box in complex scenes is improved.
[0009] S4. Identify rice ears in the UAV image through the rice ear orientation recognition model; delete duplicate detection results according to the intersection-over-union ratio of the marked orientation frame and the identified detection frame, and output and save the position, angle, length, width and other information of the rice ear detection frame.
[0010] As a preferred technical solution of the present invention, the preprocessing process of the collected rice ear image data is as follows:
[0011] The acquired rice ear image data is cropped and the cropped image data is enhanced; the data enhancement includes: image mosaicking and brightness balancing;
[0012] The enhanced rice ear image data is manually annotated with rice ears in the image and a labeling file is generated; one image corresponds to one labeling text, and the images are classified into a training set and a test set in a ratio of 8:2.
[0013] As a preferred technical solution of the present invention, the steps of constructing the rice ear orientation recognition model are specifically as follows:
[0014] S3-1, initialize the YOLOv8 network, combine the circular smooth label to classify the angle information of the rice ear, each category is 1°, and there are 360 categories in total, so as to provide angle information for the rice ear prediction frame;
[0015] S3-2, based on the rice ear prediction frame generated by the circular smooth label and the multi-scale feature map, the position, length, width and angle information of the prediction frame are compared with the position, length, width and angle information of the annotated rice ear frame to construct the network loss function L.
[0016] As a preferred technical solution of the present invention, the distance and periodicity between the angles of rice ears are processed based on a circular smooth label, and the expression of the circular smooth label is as follows:
[0017]
[0018] Among them, θ is the angle of the rice ear orientation frame, g(x) is the Gaussian function, r is the window radius of the Gaussian function, and x is the manually marked angle.
[0019] As a preferred technical solution of the present invention, the positioning loss l is used p , classification loss l c and confidence loss l f Describe the error l of the network:
[0020] l=l p +l c +l f
[0021] Among them, the classification loss l cThe angle information quantified by the circular smooth label is added to the model so that the model can measure the angular distance between the detection box and the bounding box:
[0022]
[0023] Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes, and θ is the angle category; I ij Indicates whether the jth anchor in the i-th grid can detect an object: the detected value is 1, and the undetected value is 0; P i (c) represents the probability that the object is a rice ear, P i (θ) represents the probability that the object’s angle is θ; represents the probability that the object is not a rice ear; Represents the probability that the object angle is not θ; c∈class represents the total number of object categories.
[0024] As a preferred technical solution of the present invention, the positioning loss of the rice ear orientation recognition model is p By combining the Wasserstein distance and the traditional IoU, the evaluation model not only focuses on the overlapping area when evaluating the spatial overlap between bounding boxes, but also considers the differences between the shapes and positions of the bounding boxes. Its goal is to solve the shortcomings of the traditional IoU when the bounding boxes do not overlap, so that the detection model can give a reasonable evaluation value even in the case of incomplete overlap. The calculation formula is as follows:
[0025]
[0026] IoU(M1,M2)=∣M1∪M2∣M1∩M2∣
[0027] W(M1,M2)=||c(M1)-c(M2)||2+||s(M1)-s(M2)||2
[0028] Where W(M1,M2) is used to calculate the Wasserstein distance between bounding boxes M1 and M2, c(M1) and c(M2) are the centroids of bounding boxes M1 and M2 respectively, s(M1) and s(M2) are the sizes of the bounding boxes, ||*||2 represents the Euclidean distance between vectors; α is a weight parameter, which is usually used to balance the contribution of IoU and Wasserstein distance:
[0029]
[0030] Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes; I ijIndicates whether the j-th anchor in the i-th grid can detect an object: the value of detected is 1, and the value of undetected is 0.
[0031] As a preferred technical solution of the present invention, the confidence loss l f The calculation formula is as follows:
[0032]
[0033] Among them, λ obj and λ noobj are the weights of the confidence errors, is the confidence of the prediction box, C i is the confidence level of the true target; Indicates that if there is no real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0; It means that if there is a real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0.
[0034] As a preferred technical solution of the present invention, the process of deleting duplicate detection results according to the intersection-over-union ratio of the marked oriented frame and the recognized detection frame is as follows:
[0035] S41, the initial learning rate of network training is set to 0.001, and the batch size is set to 32;
[0036] S42, after the model training converges, the intersection-and-union ratio area of the marked oriented frame and the recognized detection frame is calculated, the rice ear detection frame with the largest intersection-and-union ratio area is saved, and the remaining detection frames are deleted; the overlapping area of the two oriented detection frames is divided into multiple triangles with the same vertices;
[0037] By counting the area of each triangle and adding them together, we can get the area of the overlapping area.
[0038] The rice ear detection frame with the largest intersection-over-union ratio is saved and the remaining detection frames are deleted; finally, the position, angle, length, width and other information of the saved rice ear detection frame are output in the form of image and text as the final recognition result.
[0039] In summary, the present invention has the following beneficial effects: by combining drone images with the angle information of rice ears and a small-scale detection layer, a directional detection frame is used to replace the traditional horizontal detection frame, thereby realizing accurate identification of rice ears. Compared with traditional ground equipment, this method has a wider coverage, lower cost, and can more accurately reflect the spatial position and direction of rice ears, especially in densely distributed or complex field environments. The microscale detection layer improves the ability to capture details, reduces false detections and missed detections, significantly improves the accuracy and efficiency of rice ear identification, and provides a more reliable and economical rice ear monitoring solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A flowchart of a method for identifying rice ear orientation using drone images based on deep learning provided by an embodiment of the present invention;
[0041] Figure 2 A schematic diagram of preprocessing collected rice ear image data provided by an embodiment of the present invention;
[0042] Figure 3 A schematic diagram of deleting duplicate detection results according to the intersection-and-union ratio of the marked directional frame and the recognized detection frame provided by an embodiment of the present invention; DETAILED DESCRIPTION
[0043] The present application is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present application. These all belong to the protection scope of the present application.
[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0045] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.
[0046] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.
[0047] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.
[0048] The disclosed embodiments aim to solve the problem that rice ears in drone images are often small in size and easily affected by occlusion and background interference. In view of this, the disclosed embodiments propose a method for directional recognition of rice ears in drone images based on deep learning. By combining drone images with the angle information of rice ears and a small-scale detection layer, a directional detection frame is used to replace the traditional horizontal detection frame, thereby achieving accurate recognition of rice ears.
[0049] Please refer to Figure 1 , Figure 1 The flowchart of a method for identifying rice ear orientation using drone images based on deep learning according to an embodiment of the present disclosure is shown. The overall process mainly includes the following four steps:
[0050] Step 1: Use a drone to take images of rice ears;
[0051] Specifically, drones were used to capture images of rice ears under clear skies.
[0052] Step 2: Crop the drone image of rice ears and perform data enhancement on the cropped image; manually annotate the rice ears in the image and generate an annotation file, with one image corresponding to one annotated text, and classify them into training set and test set in a ratio of 8:2.
[0053] Specifically, the acquired UAV images were cropped into 600×600 sub-images; the sub-images were rotated by 90°, 180°, and 270°, and enhanced by image mosaicking and brightness balancing.
[0054] The image annotation software Ro-labelimg was used to annotate the rice ears in the sub-image and generate an annotation file, which contained the position, length, width and angle information of the rice ears. The sub-image was connected with the annotation file to construct the training set and test set of the model in a ratio of 8:2.
[0055] Step 3: Construct a rice ear orientation recognition model, extract and classify the orientation information of rice ears through the rice ear orientation recognition model, and complete the processing and expression of rice ear orientation features; based on the weighted intersection-over-union positioning loss, improve the network loss of the rice ear bounding box and prediction box in complex scenes.
[0056] Specifically, the steps for constructing the rice ear recognition model are as follows:
[0057] S3-1, initialize the YOLOv8 network, and classify the angle information of the rice ears in combination with the circular smooth label. Each category is 1°, with a total of 360 categories, thereby providing angle information for the rice ear prediction box.
[0058] The distance and periodicity between the angles of rice ears are processed based on the circular smoothing label. The expression of the circular smoothing label is as follows:
[0059]
[0060] Among them, θ is the angle of the rice ear orientation frame, g(x) is the Gaussian function, r is the window radius of the Gaussian function, and x is the manually marked angle.
[0061] S3-2, based on the rice ear prediction frame generated by the circular smooth label and the multi-scale feature map, the position, length, width and angle information of the prediction frame are compared with the position, length, width and angle information of the annotated rice ear frame to construct the network loss function L.
[0062] Using the localization loss l p , classification loss l c and confidence loss l f Describe the error l of the network:
[0063] l=l p +l c +l f
[0064] Among them, the classification loss l c The angle information quantified by the circular smooth label is added to the model so that the model can measure the angular distance between the detection box and the bounding box:
[0065]
[0066] Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes, and θ is the angle category; I ij Indicates whether the jth anchor in the i-th grid can detect an object: the detected value is 1, and the undetected value is 0; P i (c) represents the probability that the object is a rice ear, P i (θ) represents the probability that the object’s angle is θ; represents the probability that the object is not a rice ear; Represents the probability that the object angle is not θ; c∈class represents the total number of object categories.
[0067] Where, the positioning loss of the rice ear orientation recognition model is p By combining the Wasserstein distance and the traditional IoU, the model not only focuses on the overlapping area when evaluating the spatial overlap between bounding boxes, but also considers the differences between the shapes and positions of the bounding boxes. Its goal is to address the shortcomings of the traditional IoU when the bounding boxes do not overlap, so that the detection model can give a reasonable evaluation value even in the case of incomplete overlap. The calculation formula is as follows:
[0068]
[0069] IoU(M1,M2)=∣M1∪M2∣M1∩M2∣
[0070] W(M1,M2)=||c(M1)-c(M2)||2+||s(M1)-s(M2)||2
[0071] Among them, W(M1,M2) is used to calculate the Wasserstein distance of bounding boxes M1 and M2, c(M1) and c(M2) are the centroids of bounding boxes M1 and M2 respectively, s(M1) and s(M2) are the sizes of bounding boxes, ||*||2 represents the Euclidean distance between vectors; α is a weight parameter, which is usually used to balance the contribution of IoU and Wasserstein distance.
[0072]
[0073] Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes; I ij Indicates whether the j-th anchor in the i-th grid can detect an object: the value of detected is 1, and the value of undetected is 0.
[0074] In the formula, the confidence loss l f The calculation formula is as follows:
[0075]
[0076] Among them, λ obj and λ noobj are the weights of the confidence errors, is the confidence of the prediction box, C i is the confidence level of the true target; Indicates that if there is no real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0; It means that if there is a real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0.
[0077] Step 4: Identify rice ears in the drone image through the rice ear orientation recognition model; delete duplicate detection results based on the intersection-over-union ratio of the marked orientation frame and the identified detection frame, and output and save the position, angle, length, width and other information of the rice ear detection frame.
[0078] Specifically, S41, the initial learning rate of network training was set to 0.001, and the batch size was set to 32;
[0079] S42. After the model training converges, the intersection-and-union ratio of the labeled oriented frame and the identified detection frame is calculated, the rice ear detection frame with the largest intersection-and-union ratio is saved, and the remaining detection frames are deleted; the overlapping area of the two oriented detection frames is divided into multiple triangles with the same vertices.
[0080] By counting the area of each triangle and adding them together, the area of the overlapping area is obtained.
[0081] The rice ear detection frame with the largest intersection-over-union ratio is saved, and the rest of the detection frames are deleted. Finally, the position, angle, length, width and other information of the saved rice ear detection frame are output in the form of image and text as the final recognition result.
[0082] The calculation process is as follows:
[0083]
[0084] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A method for rice ear orientation recognition based on drone images based on deep learning, characterized in that: The method comprises the following steps: S1. Collect rice ear image data using drones; S2, preprocessing the collected rice ear image data; S3. First, a rice ear orientation recognition model is constructed. The preprocessed rice ear image data is extracted and classified through the rice ear orientation recognition model to complete the processing and expression of the rice ear orientation features. Then, based on the positioning loss of the weighted intersection-over-union ratio, the network loss of the rice ear boundary box and the prediction box in complex scenes is improved. S4. Identify rice ears in the UAV image through the rice ear orientation recognition model; delete duplicate detection results according to the intersection-over-union ratio of the marked orientation frame and the identified detection frame, and output and save the position, angle, length, width and other information of the rice ear detection frame.
2. The method for rice ear orientation recognition based on drone images according to claim 1 is characterized in that: The preprocessing process of the collected rice ear image data is as follows: The acquired rice ear image data is cropped and the cropped image data is enhanced; the data enhancement includes: image mosaicking and brightness balancing; The enhanced rice ear image data is manually annotated with rice ears in the image and a labeling file is generated; one image corresponds to one labeling text, and the images are classified into a training set and a test set in a ratio of 8:
2.
3. The method for rice ear orientation recognition based on drone images according to claim 1 is characterized in that: The specific steps for constructing the rice ear orientation recognition model are as follows: S3-1, initialize the YOLOv8 network, combine the circular smooth label to classify the angle information of the rice ear, each category is 1°, and there are 360 categories in total, so as to provide angle information for the rice ear prediction frame; S3-2, based on the rice ear prediction frame generated by the circular smooth label and the multi-scale feature map, the position, length, width and angle information of the prediction frame are compared with the position, length, width and angle information of the annotated rice ear frame to construct the network loss function L.
4. The method for rice ear orientation recognition based on drone images according to claim 3 is characterized in that: The distance and periodicity between the angles of rice ears are processed based on a circular smooth label, and the expression of the circular smooth label is as follows: Among them, θ is the angle of the rice ear orientation frame, g(x) is the Gaussian function, r is the window radius of the Gaussian function, and x is the manually marked angle.
5. The method for rice ear orientation recognition based on drone images according to claim 3 is characterized in that: Using the localization loss l p , classification loss l c and confidence loss l f Describe the error l of the network: l＝l p +l c +l f Among them, the classification loss l c The angle information quantified by the circular smooth label is added to the model so that the model can measure the angular distance between the detection box and the bounding box: Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes, and θ is the angle category; I ij Indicates whether the jth anchor in the i-th grid can detect an object: the detected value is 1, and the undetected value is 0; P i (c) represents the probability that the object is a rice ear, P i (θ) represents the probability that the object’s angle is θ; represents the probability that the object is not a rice ear; Represents the probability that the object angle is not θ; c∈class represents the total number of object categories.
6. The method for rice ear orientation recognition based on drone images according to claim 5 is characterized in that: The localization loss of the rice ear orientation recognition model p By combining the Wasserstein distance and the traditional IoU, the evaluation model not only focuses on the overlapping area when evaluating the spatial overlap between bounding boxes, but also considers the differences between the shapes and positions of the bounding boxes. Its goal is to solve the shortcomings of the traditional IoU when the bounding boxes do not overlap, so that the detection model can give a reasonable evaluation value even in the case of incomplete overlap. The calculation formula is as follows: IoU(M1,M2)=∣M1∪M2∣M1∩M2∣ W(M1,M2)=||c(M1)-c(M2)||2+||s(M1)-s(M2)||2 Among them, W(M1,M2) is used to calculate the Wasserstein distance of bounding boxes M1 and M2, c(M1) and c(M2) are the centroids of bounding boxes M1 and M2 respectively, s(M1) and s(M2) are the sizes of bounding boxes, ||*||2 represents the Euclidean distance between vectors; α is a weight parameter, which is usually used to balance the contribution of IoU and Wasserstein distance: Among them, S 2 is the feature map size output by the detection layer, B is the number of anchor boxes; I ij Indicates whether the j-th anchor in the i-th grid can detect an object: the value of detected is 1, and the value of undetected is 0.
7. The method for rice ear orientation recognition based on drone images according to claim 5 is characterized in that: Confidence loss l f The calculation formula is as follows: Among them, λ obj and λ noobj are the weights of the confidence errors, is the confidence of the prediction box, C i is the confidence level of the true target; Indicates that if there is no real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0; It means that if there is a real target in the j-th prediction box in the i-th grid of the training image, it is 1, otherwise it is 0.
8. The method for rice ear orientation recognition based on drone images according to claim 1 is characterized in that: According to the intersection-over-union ratio of the annotated oriented box and the identified detection box, the process of deleting duplicate detection results is as follows: S41, the initial learning rate of network training is set to 0.001, and the batch size is set to 32; S42, after the model training converges, the intersection-and-union ratio area of the labeled oriented frame and the recognized detection frame is calculated, the rice ear detection frame with the largest intersection-and-union ratio area is saved, and the remaining detection frames are deleted; The overlapping area of two oriented detection frames is divided into multiple triangles with the same vertices; By counting the area of each triangle and adding them together, we can get the area of the overlapping area. The rice ear detection frame with the largest intersection-over-union ratio is saved and the remaining detection frames are deleted; finally, the position, angle, length, width and other information of the saved rice ear detection frame are output in the form of image and text as the final recognition result.