A method for online detection and counting of orchard fruits using drones

Abstract

This invention discloses a method for large-scale online detection and counting of orchard fruits using drones. It utilizes drones to collect video footage of orchards; establishes a drone live-streaming platform for online, real-time video transmission; builds a fruit detection model to perform real-time detection of the drone video footage, obtaining the center point and detection frame position information of each fruit; employs the DeepSORT algorithm to assign a specific ID to each fruit in the drone video footage and tracks them across frames; and constructs a fruit counter to achieve global calculation of the orchard fruit count. This invention overcomes the limitation that drones can only perform offline operations in orchard scenarios, and simultaneously solves the problems of missing small-sized fruits and severely occluded fruits within the drone's field of view, providing farmers with effective information for subsequent decision-making in a short period of time.

Description

Technical Field

[0001] This invention belongs to the field of fruit detection and counting, and specifically relates to an online detection and counting method for orchard fruits using drones. Background Technology

[0002] In the process of fruit production and processing, pre-harvest yield estimation is an extremely important part. Accurate orchard yield forecasting can not only provide effective information for managers and farmers to make decisions at different stages of the year, but also help them make wise adjustments to management in the following year, further improving the yield and quality of orchard fruits. Among them, the accurate detection and counting of fruits in natural environment is a key step in early yield estimation of orchards and the realization of smart orchard production management. There are many existing fruit counting methods, but they often have the following drawbacks: (1) In existing fruit counting studies, people usually use handheld cameras or small vehicles equipped with cameras to collect fruit videos in advance, and then use algorithms to process offline videos to obtain the number of fruits. This type of method requires additional economic costs (laying guide rails), cannot estimate yield in real time, and cannot achieve online yield estimation, that is, it is impossible to make timely adjustments on site to obtain accurate results. In addition, the Apolo-Apolo & Pérez-Ruiz et al., 2020 study showed that the above photography method requires a specific platform, which may not only cause the inability to operate due to obstacles, but may also aggravate soil compaction problems. (2) Due to the high altitude of drone photography, the size of fruits in the image is often small. At the same time, the fruits grow densely and cause severe occlusion. Therefore, the detection of small objects and densely occluded objects has always been a challenge in fruit detection. According to the definition of the COCO dataset, when the number of pixels of an object is less than 32×32, the object is considered a small object. Figure 2 The image of the drone input to the counting system shows that the size of the complete fruit on the front of the fruit tree is concentrated at 15 pixels × 13 pixels (the rightmost tree), the size of the occluded fruit is concentrated at 12 pixels × 12 pixels (the middle tree), and the size of the fruit on the back of the fruit tree is concentrated at 10 pixels × 9 pixels (the leftmost tree). In all three cases, the fruit is within the small target range. (3) At present, most scholars have only realized the counting of a single image and cannot apply it to the counting of the entire orchard. Some scholars obtain the total number of fruits in the orchard by repeatedly shooting the front and back of each fruit tree, such as Apolo-Apolo & Martínez-Guanter et al., 2020. This type of method is complicated and time-consuming. Some scholars have also proposed to realize the counting of orchard fruits based on neural network technology combined with three-dimensional spatial information, such as patent CN112233121A. However, the acquisition of three-dimensional spatial information by this type of method often requires a depth camera or additional three-dimensional reconstruction algorithm, which not only increases the cost, but also makes it difficult to guarantee the real-time performance of the algorithm. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an online detection and counting method for orchard fruits using drones, which can perform real-time counting and small-target fruit counting, and realize accurate detection and counting of fruits in large-scale orchards.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A method for online detection and counting of orchard fruits using drones includes the following steps:

[0006] (1) Use drones to collect drone videos of the orchard;

[0007] (2) Build an orchard drone live streaming platform to transmit drone videos online and in real time;

[0008] (3) Establish a fruit detection model to perform real-time detection on UAV video footage and obtain the position information of the center point and detection frame of each fruit;

[0009] (4) The DeepSORT algorithm is used to assign a specific ID to each fruit in the UAV video footage and track them across frames;

[0010] (5) Construct a fruit counter to realize the global calculation of the number of fruits in the orchard.

[0011] In step (1), the process of using a drone to collect drone video of the orchard specifically includes the following steps:

[0012] (1-1) Use the camera on the drone to obtain drone video of the fruit trees;

[0013] (1-2) Frame extraction is performed on the acquired UAV videos to remove images of poor quality, an image database is constructed, and a training set is created. train Validation Set val and test set test ;

[0014] (1-3) Use the DarkLabel annotation tool to annotate the images.

[0015] In step (2), the construction of the orchard drone live streaming platform is based on the mainstream open-source architecture Nginx-RTMP-module, and specifically includes the following steps:

[0016] (2-1) Connect the computing platform and the mobile phone to the same network so that they are in the same intranet;

[0017] (2-2) Perform flight control on the drone and push the drone video stream to the Nginx server online via the RTMP push streaming protocol;

[0018] (2-3) The computing platform starts up and accesses the Nginx server to obtain the drone video stream in real time.

[0019] In step (2-2), the flight control of the drone is based on secondary development using DJI's official Mobile SDK, and includes the following steps:

[0020] (2-2-1) Based on the LiveSteamView class in Mobile-SDK, the video streaming function is implemented. The video is re-encoded into a binary stream for transmission. At the same time, the transmission data is optimized by removing audio information that is not useful for image recognition. Then, the destination address of the streaming is set to the address of the backend server in the same network segment to complete the streaming.

[0021] (2-2-2) Set up a backend server to receive RTMP streams. Use an Nginx server with the rtmp-module plugin to set up the server. This server is used to receive RTMP binary streams pushed from the front end and re-encode them into video. When starting the server, you need to specify parameters such as the address, port and data packet size of the local machine in the configuration file. After the configuration is correct, you can play the video through video playback software such as VLC Media Player and count the frame rate and packet loss.

[0022] (2-2-3) The autonomous flight control function of the drone is implemented based on the FlightController class. The user sets parameters such as flight direction, speed and distance in the operation interface. Then, according to the set flight direction, speed and distance of the drone, the time required to fly a specified distance at the current speed is calculated, and flight commands are continuously sent through a timer to realize the autonomous flight of the drone and stable shooting of orchard fruit trees.

[0023] In step (3), the establishment of the fruit detection model is achieved by using the Fruits-YOLO network. The Fruits-YOLO network is based on the YOLOv5 target detection algorithm as the basic framework of the fruit detection model, and the network is optimized and improved, including the following:

[0024] (3-1) The YOLOV5 feature extraction backbone network contains one Focus module and three CSP modules, namely Focus, CSP1_3, CSP1_9 and CSP1_9; a CSP1_3 module is added at the end of its backbone network to obtain feature maps with high-level semantic information and corresponding sizes.

[0025] (3-2) In the feature enhancement fusion network, the PANE network is replaced with the BiFPN network to splice the shallow feature maps and deep feature maps, and reuse and fuse multi-layer features.

[0026] (3-3) In the prediction network, an additional detection head is added to the original three detection heads to detect small-sized fruits;

[0027] (3-4) Reduce the pixel size of the image input to the network and optimize and accelerate the model using model deployment frameworks (such as TensorRT, NCNN, MNN and Tengine).

[0028] In step (4), the cross-frame tracking specifically includes the following steps:

[0029] (4-1) The Kalman filter predicts the trajectory of the fruit in the (k-1)th frame to obtain the trajectory Tracks in the kth frame;

[0030] (4-2) Use the Hungarian algorithm to match the predicted tracks with the detections in the current k-th frame. First, perform concatenated matching, then perform IOU matching.

[0031] (4-3) The Kalman filter updates the fruit trajectory in the kth frame.

[0032] In step 5, the construction of the fruit counter is carried out using a non-uniform distribution counter counting method, specifically including the following steps:

[0033] (5-1) During the flight of the UAV, the non-uniformly distributed counter performs the counting task. If the ID value of a fruit in the picture does not exist in the fruit database D, it is a newly appeared fruit. It is counted and its ID value is added to the fruit database D; otherwise, it indicates that the fruit has been counted and no operation is performed on it.

[0034] (5-2) By controlling the drone to photograph all the fruit trees in the orchard, the total number of fruits in the orchard can be obtained.

[0035] In step (5-1), the non-uniform distribution counter includes two counters_a, two counters_b, and one counter_c; counters_a are used to count the fruits at the image boundary; counters_b, which have a smaller working range, are placed on both sides of the image to count the clear and obvious fruits; counters_c are used to count most of the fruits in the middle of the shot.

[0036] The principle of this invention is as follows: (1) When the UAV is operating, the UAV control application first encodes the video stream into a binary stream and transmits it to the specified RTMP push address; sets parameters such as flight direction, speed and distance, controls the UAV to fly and shoot the fruit trees in the orchard; the proposed counting system obtains the video stream online by accessing the Nginx server. (2) The fruit detection model adopts the BiFPN network. Based on PANet, BiFPN has two main improvements: First, it simplifies PANet by deleting redundant nodes at both ends of the network to reduce unnecessary calculations; Second, it increases the jump connection between the original feature nodes and the output nodes of the same layer of the network to strengthen the reuse and fusion of multi-layer features; In addition, after obtaining the feature map fused by the shallow network and the deep network, we add a small object detection head to detect it to enhance the robustness to small-sized fruits; At the same time, we reduce the size of the image pixels input to the network and use the model deployment framework to quantize and accelerate the trained model (e.g., Int 8, FP16 and FP32). (3) While the wide field of view of drones allows us to quickly and efficiently complete the counting of orchards, it also brings the problem of fruit ID variations, leading to duplicate counting of fruits. When using drones to count orchards, we found that if the maximum value of the fruit ID is used as the final counting result, a large number of fruits will be counted repeatedly, resulting in serious counting errors, such as... Figure 6 As shown in (a), the maximum fruit ID obtained using this algorithm is 1274, while the actual number of fruits is 318; therefore, we propose a fruit counter, such as... Figure 6 As shown in (b), the non-uniform distribution counter of the present invention consists of two counters_a, two counters_b, and one counter_c. Because the fruit in the middle of the video (which is often clear and occupies a large proportion of the image pixels) is easy to identify and track when the drone is flying stably, while the fruit on both sides of the image (far away from the drone lens) is often blurry, and the proportion of all images pixels is low, which easily leads to the phenomenon of fruit ID change, resulting in repeated counting of fruit; therefore, counter_a is responsible for counting the fruit at the image boundary; counter_b, which has a smaller working range, is placed on both sides of the image and is responsible for counting clear and obvious fruit, reducing the counting of fruit that fails to be tracked; counter_c is responsible for counting most of the fruit in the middle of the lens image.

[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0038] (1) This invention uses drones to achieve rapid and accurate counting of fruits in large-scale orchards, reducing economic costs and has broad application prospects and value.

[0039] (2) This invention, by building an orchard drone live streaming platform and combining it with a counting algorithm, has for the first time realized online and real-time counting of orchard fruits, reducing time costs.

[0040] (3) This invention proposes a fruit detection model Fruits-YOLO for UAV application scenarios. By adding a small target detection layer and replacing PAN-Net with Bi-FPN, it can effectively prevent the missed detection of small-sized fruits and severely occluded fruits. At the same time, this invention reduces the image size and uses a deployment framework to accelerate the process, thereby improving the real-time performance and practicality of this invention.

[0041] (4) This invention proposes a non-uniformly distributed counter, which combines a tracking algorithm to achieve continuous tracking and global counting of fruits in the UAV screen, and can effectively suppress repeated counting caused by UAV flight instability or tracking algorithm failure.

[0042] (5) The method of the present invention has the advantages of low cost, strong robustness and high real-time performance, and can also be extended to the detection, tracking and counting of various other objects of interest in UAV application scenarios. In addition, through simple extensions, the present invention can realize the simultaneous operation of multiple UAVs, which has high practicality and promotional value. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention.

[0044] Figure 2 This is a diagram showing the target size of the fruit.

[0045] Figure 3 This is the architecture diagram of the orchard drone live streaming platform of this invention.

[0046] Figure 4 This is a diagram of the Fruits-YOLO network structure of the present invention.

[0047] Figure 5 This is a diagram illustrating the fruit detection, tracking, and counting effects of the present invention.

[0048] Figure 6 This is a structural diagram of the non-uniform distribution counter of the present invention; wherein, (a) is a counting based on the maximum fruit ID value; and (b) is a counting based on a non-uniform distribution counter.

[0049] Figure 7 This is a comparison chart of the detection performance of the YOLOV5 network (a) and the Fruits-YOLO network of this invention (b). Detailed Implementation

[0050] To facilitate understanding of the present invention, specific embodiments will be described in detail below. These embodiments will help those skilled in the art to further understand the present invention; however, they are not intended to limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements to the present invention without departing from its conceptual framework, and these modifications and improvements all fall within the scope of protection of the present invention.

[0051] Example 1

[0052] like Figure 1 As shown, taking citrus as the object, this embodiment provides a method for online detection and counting of orchard fruits using drones. The specific implementation steps are as follows:

[0053] Videos of the fruit trees were acquired using DJI PHANTOM 4 PRO v2 and PHANTOM 4 ADVANCED drones. The drone's onboard camera had a controllable rotation range of -90° to +30°, and the built-in camera had 20 megapixels, producing an image resolution of 4096×2160. The flight altitude was 5.4m. The filming location was an orchard in Zengcheng District, Guangzhou City, Guangdong Province, China (23°16′N, 113°51′E). The trees bore a large number of small fruits, a variety known as Emperor Mandarin oranges, which were yellow in color. A total of 30 drone videos were collected, each containing 5 to 10 trees and 300 to 600 fruits, with video lengths ranging from 2 to 5 minutes. Since the fruit detection network was trained using image data, frame extraction from the videos was employed, resulting in 12,640 training images. To reduce the time and manpower required for image annotation, the DarkLabel annotation tool was used for manual annotation.

[0054] Enable a Wi-Fi hotspot on your mobile phone to ensure the computing platform and the phone are on the same internal network. Use the DJI GO 4 or Fly4Citrus application installed on your smartphone for drone flight control, and push the drone's video stream online to the Nginx server via the RTMP streaming protocol. The computing platform starts up and accesses the Nginx server to retrieve the drone video stream online. The architecture diagram of the orchard drone live streaming platform is shown below. Figure 3 As shown.

[0055] In previous experiments in the orchard, we mainly controlled the drone's flight using the official DJI GO 4 application installed on a smartphone. This method has the following disadvantages: (1) The drone is manually controlled, which is not conducive to intelligent counting in the orchard. (2) It is difficult for operators to maintain absolute stability during operation, and changes in drone speed will affect the algorithm. (3) The narrow space and uneven ground in the orchard will have a certain impact on the stable flight of the drone, and in severe cases, it may crash, causing economic losses. To solve the above problems, based on DJI's official MobileSDK, the inventor carried out secondary development and implemented an application called Fly4Citrus. The interface of the Fly4Citrus application adopts the Merge layout design. The flight function of the Fly4Citrus application is mainly implemented based on the LiveSteamView class and the FlightController class. When the UAV is operating, Fly4Citrus first encodes the video stream into a binary stream and transmits it to the specified RTMP push address. Then, the user sets parameters such as flight direction, speed, and distance in the interface. The Fly4Citrus application then calculates the time required to fly a specified distance at the current speed by obtaining the flight direction, speed, and distance, and continuously sends flight commands through a timer to enable the UAV to fly autonomously and stably film the orchard fruit trees. Finally, the proposed counting pipeline obtains the video stream online by accessing the Nginx server.

[0056] The YOLOv5 object detection algorithm was chosen as the basic framework for the fruit detection model. Network optimization and improvement were performed, ultimately leading to the proposed Fruits-YOLO network. Figure 4 As shown. The reason is that, under the drone's field of view, the fruits in the image not only have high density and severe occupancy and overlap, but also occupy a small number of pixels, such as... Figure 2As shown, this scenario presents a significant challenge for the YOLOv5 algorithm. Specifically: The YOLOv5 feature extraction backbone network includes one Focus module and three CSP modules: Focus, CSP1_3, CSP1_9, and CSP1_9. A CSP1_3 module is added at the end of the backbone network to obtain feature maps with high-level semantic information and corresponding dimensions. In the feature enhancement and fusion network, the PANE network is replaced with a BiFPN network to concatenate shallow and deep feature maps, reusing and fusing multi-layer features. In the prediction network, based on the original three detection heads of YOLOv5... An additional detection head was added to detect small fruits, enhancing the network's robustness to small fruits. During model training, the fruit detection model was trained using the training sets `Settrain` and `Setval`. To accelerate training, the training image size was set to 1280*1280. In the actual orchard fruit counting operation, the computing platform first started the Nginx server. Then, the Fruits-YOLO network pulled drone video streams in real time via the RTMP protocol for detection. Finally, the location information of the center point and detection box of each fruit was obtained. (See fruit detection details...) Figure 5 As shown; where, to speed up inference, the detection image size is set to 640*640 and TensorRT is used to accelerate the model.

[0057] The Kalman filter predicts the trajectory of the fruit in the (k-1)th frame, obtaining the trajectory Tracks in the kth frame. The Hungarian algorithm is used to match the predicted trajectory Tracks with the Detections in the current kth frame, first performing cascade matching, then performing IOU matching. The Kalman filter updates the fruit trajectory in the kth frame.

[0058] This invention employs a counting scheme for a non-uniformly distributed counter, as detailed below: Figure 6 As shown in (b), a non-uniformly distributed counter includes two counters_a, two counters_b, and one counter_c. Counters_a are responsible for counting fruits at the image boundaries; counters_b, with a smaller working range, are placed on both sides of the image to count clearly visible fruits, reducing the number of fruits that fail to be tracked; counters_c are responsible for counting most fruits in the middle of the image. During the drone's flight, the non-uniformly distributed counter performs the counting task. If the ID value of a fruit in the image does not exist in the fruit database D, it is a newly appearing fruit, and its ID value is counted and added to the fruit database D; otherwise, it indicates that the fruit has already been counted, and no operation is performed on it. By controlling the drone to photograph all fruit trees in the orchard, the total number of fruits in the orchard is finally obtained, see... Figure 5 As shown.

[0059] Test Example 1

[0060] The experimental results of this invention are described in detail below:

[0061] This invention uses three industry-recognized evaluation metrics to statistically analyze the detection results of the test video: Precision (P), Recall (R), F1, and FPS. Their formulas are defined as shown in formulas (1)-(3):

[0062]

[0063]

[0064]

[0065] In this model, TP represents the number of correctly detected fruits, FP represents the number of false positives, and FN represents the number of missed detections. P and R represent precision and recall, respectively. F1 is the harmonic mean of precision and recall, which is a metric reflecting model performance.

[0066] The test video was calculated using the above formula, and the statistical results are shown in Table 1.

[0067] Table 1 Comparison of detection results between YOLOv5 network and the Fruits-YOLO network of this invention.

[0068]

[0069] The comparison results between the YOLOv5 network and the Fruits-YOLO network proposed in this invention are as follows: Figure 7 As shown, the method proposed in this invention effectively overcomes the problem of the YOLOv5 network missing small-sized fruits or occluded fruits. Specific quantitative results are shown in Table 1. In terms of accuracy, the Fruits-YOLO network's precision, recall, and F1 score are all higher than YOLO v5, by 0.55 percentage points, 1.34 percentage points, and 0.96 percentage points respectively. Regarding real-time performance, the accelerated Fruits-YOLO network runs significantly faster than the YOLOv5 network, with a 92.06 FPS improvement. Experimental results demonstrate that the Fruits-YOLO network of this invention has superior detection performance.

[0070] The counting results of this invention are statistically analyzed using four commonly used evaluation indicators in the industry: latency, packet loss rate, IDs, and mean absolute percentage error (MAPE), which are defined as shown in formula (4):

[0071]

[0072] Where n represents the number of fruit trees used for testing, which is 33 in this experiment. i The final calculation result of the algorithm representing the i-th video, round Truth i This represents the total number of fruits manually counted for the i-th tree.

[0073] The experiment used two methods: offline video counting and online counting in the orchard as described in this invention. The comparison results of the two methods are shown in Table 2.

[0074] Table 2 Comparison of offline and online counting results

[0075]

[0076] As shown in Table 2, in terms of counting performance, the average IDs and MAPE of the offline counting method are 18% and 5.28%, respectively, while the average IDs and MAPE of the online counting method are improved to 39.67% and 9.67%, respectively. The final experimental results show that the error of the proposed UAV online counting system is 14.95%, and its counting accuracy is within an acceptable range.

[0077] The above description is merely an embodiment of the present invention, but the implementation of the present invention is not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention are equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A method for online detection and counting of orchard fruits using unmanned aerial vehicles (UAVs), characterized in that... Includes the following steps: (1) Use drones to collect drone videos of the orchard; (2) Establish an orchard drone live streaming platform to transmit drone videos online and in real time, including the following: (2-1) Connect the computing platform and the mobile phone to the same network so that they are in the same intranet; (2-2) Perform flight control on the UAV and push the UAV video stream to the Nginx server online via the RTMP push protocol; (2-3) The computing platform starts up and accesses the Nginx server to obtain the drone video stream in real time; (3) Establish a fruit detection model to detect the UAV video footage in real time and obtain the position information of the center point of each fruit and the detection frame; The fruit detection model is established using the Fruits-YOLO network. The Fruits-YOLO network uses the YOLOv5 target detection algorithm as the basic framework of the fruit detection model, and then optimizes and improves the network, including the following: (3-1) The YOLOV5 feature extraction backbone network contains one Focus module and three CSP modules, namely Focus, CSP1_3, CSP1_9 and CSP1_9; a CSP1_3 module is added at the end of its backbone network to obtain feature maps with high-level semantic information and corresponding sizes. (3-2) In the feature enhancement fusion network, the PANE network is replaced with the BiFPN network to splice the shallow and deep feature maps, reuse and fuse multi-layer features; (3-3) In the prediction network, an additional detection head is added to detect small-sized fruits, based on the original three detection heads; (3-4) Reduce the pixel size of the image input to the network and optimize and accelerate the model using a model deployment framework; (4) The DeepSORT algorithm is used to assign a specific ID to each fruit in the UAV video footage and track them across frames; (5) Construct a fruit counter using a non-uniform distribution counter method, including the following: (5-1) During the flight of the UAV, the non-uniformly distributed counter performs the counting task. If the ID value of a certain fruit in the picture does not exist in the fruit database D, it is a newly appeared fruit. It is counted and its ID value is added to the fruit database D; otherwise, it indicates that the fruit has been counted and no operation is performed on it. The non-uniformly distributed counter includes two counters_a, two counters_b and one counter_c. Counters_a are used to count the fruits at the edge of the image. Counters_b are placed on both sides of the image and are used to count the fruits that are clear and obvious. Counters_c are used to count the fruits in the middle of the lens picture. (5-2) By controlling the drone to photograph all the fruit trees in the orchard, the total number of fruits in the orchard can be obtained.

2. The method for online detection and counting of orchard fruits using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: In step (1), the process of using a drone to collect drone video of the orchard includes the following steps: (1-1) Use the camera mounted on the drone to obtain drone video of the fruit trees; (1-2) Perform frame extraction processing on the collected UAV videos, remove images with poor quality, construct an image database, and divide the training set Set. train Validation Set val and test set test ; (1-3) Use the DarkLabel annotation tool to annotate the images.

3. The method for online detection and counting of orchard fruits using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: In step (2-2), the flight control of the UAV includes the following steps: (2-2-1) Based on the LiveSteamView class in Mobile-SDK, the video streaming function is implemented. The video is re-encoded into a binary stream for transmission. At the same time, the transmission data is optimized by removing audio information that is not useful for image recognition. Then, the destination address of the streaming is set to the address of the backend server in the same network segment to complete the streaming. (2-2-2) Set up a backend server to receive RTMP streams. Use an Nginx server with the rtmp-module plugin to set up the server. This server is used to receive RTMP binary streams pushed from the front end and re-encode them into video. When starting the server, you need to specify the address, port and data packet size of the local machine in the configuration file. Then play the video through video playback software and count the frame rate and packet loss. (2-2-3) The autonomous flight control function of the drone is implemented based on the FlightController class. The user sets the flight direction, speed and distance parameters in the operation interface. Then, according to the set flight direction, speed and distance of the drone, the time required to fly a specified distance at the current speed is calculated, and flight commands are continuously sent through a timer to realize the autonomous flight of the drone and stable shooting of orchard fruit trees.

4. The method for online detection and counting of orchard fruits using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: In step (4), the cross-frame tracking includes the following steps: (4-1) The Kalman filter predicts the trajectory of the fruit in the (k-1)th frame to obtain the trajectory Tracks in the kth frame; (4-2) Use the Hungarian algorithm to match the predicted tracks with the detections in the current k-th frame. First, perform concatenated matching, then perform IOU matching. (4-3) The Kalman filter updates the fruit trajectory in the kth frame.

Images