Farmland pest real-time detection device, method and system based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of farmland pest real-time detection, in particular to a farmland pest real-time detection device, method and system based on an unmanned aerial vehicle, and the device comprises the unmanned aerial vehicle, a flight control system, a remote controller, a camera, a camera remote controller, an airborne computer, a 4G communication module and a ground workstation. A detection result is transmitted to a ground work station in real time through the 4G communication module, so that the real-time performance of data is ensured, the lagging of information processing during pest outbreak is reduced, and pest diffusion caused by information delay is avoided; according to the method, pests are automatically identified based on an improved YOLOv10 network model, a weight file is deployed on an airborne computer of an unmanned aerial vehicle and real-time detection is carried out, images can be directly processed in the flight process, a detection result can be rapidly obtained, information lag is avoided, it is ensured that prevention and control measures can be taken in time when pests burst out, and the control efficiency is improved. Therefore, crop loss is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time detection of farmland pests, and in particular to a device, method and system for real-time detection of farmland pests based on an unmanned aerial vehicle. Background Art

[0002] In the field of pest control, traditional detection methods mainly rely on on-site technicians to directly observe the distribution of foliar pests on crops. However, the above detection system faces two challenges: first, the accuracy of pest identification is limited by the professional knowledge and experience of the staff, especially when dealing with pests with similar appearance or dense distribution, the detection work has a certain degree of subjectivity; second, agricultural technicians have limited resources, high work intensity and low efficiency, making it difficult to cover the national pest detection and control needs. Therefore, new ideas should be used to explore pest prediction and forecasting strategies.

[0003] In recent years, with the continuous advancement of computer vision and deep learning technologies, the concepts of precision agriculture and smart agriculture have gradually emerged and developed rapidly. Compared with traditional detection methods, deep learning algorithms have shown higher accuracy in the identification and detection of agricultural pests. At the same time, with the rise of Unmanned Aerial Vehicle (UAV) technology, it plays an important role in agricultural production, including assisting in tasks such as crop sowing, fertilization, plant protection, and harvesting. In addition, by carrying high-resolution cameras, it can also monitor crop growth, detect pests and diseases, and manage precision agriculture. The mature application of drones in the agricultural field indicates that it has become a new trend in the development of smart agriculture.

[0004] However, in the process of pest detection, drones mostly undertake the image collection function and lack the ability to process real-time information. When faced with the outbreak and spread of pests, the lag in information processing may cause the pests to spread rapidly and miss the best time for prevention and control.

[0005] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0006] The present invention provides a device, method and system for real-time detection of farmland pests based on unmanned aerial vehicles, thereby effectively solving the problems pointed out in the background technology.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A real-time detection device for farmland pests based on a drone, comprising a drone, a flight control system, a remote controller, a camera, a camera remote controller, an onboard computer, a 4G communication module and a ground workstation;

[0009] The drone is used to carry the camera and the onboard computer and perform inspection tasks in the farmland area;

[0010] The flight control system is arranged on the UAV and is used to control the flight attitude of the UAV and ensure the flight stability;

[0011] The remote controller communicates with the flight control system and is used to issue instructions to the flight control system to control the take-off, landing and flight of the UAV;

[0012] The camera is arranged on the drone and is used to obtain image information of the farmland;

[0013] The camera remote controller is connected to the camera and is used to adjust the focal length of the camera lens;

[0014] The onboard computer is disposed on the drone and is used to receive image information acquired by the camera, perform image processing to identify pests in the farmland, and count the identified pests;

[0015] The 4G communication module is connected to the onboard computer and is used to transmit the identified pest detection results from the onboard computer to the ground workstation;

[0016] The ground workstation is used to receive the pest detection results from the onboard computer and to display and analyze the detection results.

[0017] Furthermore, the camera is arranged at the bottom of the drone, and the flight control system and the 4G communication module are arranged at the top of the drone.

[0018] The present invention also includes a method for real-time detection of farmland pests based on a drone, which is used in the above-mentioned real-time detection device for farmland pests based on a drone, comprising:

[0019] S10: inspecting the farmland area by using the camera carried by the drone to collect images of farmland pests, segmenting and labeling the images of the farmland pests, and constructing a farmland pest image dataset;

[0020] S20: Based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection;

[0021] S30: deploying the weight file for farmland pest detection on the onboard computer of the drone, and performing pest detection on the real-time video stream of the farmland captured by the camera through the onboard computer;

[0022] S40: Utilizing the 4G communication module on the UAV, the farmland pest detection results generated by the onboard computer are transmitted to a ground workstation in real time, and the ground workstation receives and displays the farmland pest detection results on the UAV in real time.

[0023] Further, in step S20, based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection, the steps comprising:

[0024] S21: Optimize the loss function of the YOLOv10 network and use the PIoU loss function instead of the CIoU loss function to improve the convergence speed and accuracy of the model;

[0025] S22: Integrate the deformable large convolution kernel DLK into the SPPF module to form SPPF-DLK to enhance the model's ability to recognize objects of different scales;

[0026] S23: Integrate the hybrid local channel attention MLCA into the C2f module to form C2f-MLCA to improve the model's ability to detect small targets in complex backgrounds;

[0027] S24: Based on the optimized PIoU loss function, the SPPF-DLK and the C2f-MLCA, an improved YOLOv10 network training model is constructed, and a weight file for farmland pest detection is generated.

[0028] Furthermore, in step S21, the loss function of the YOLOv10 network is optimized, and the PIoU loss function is used instead of the CIoU loss function. The model includes:

[0029]

[0030]

[0031] Where P is the penalty term for the distance between the predicted box and the real box boundary; L PIoU represents the PIoU loss function; U represents the union area of ​​the predicted box and the true box; Dh r With Dh p They represent the distances between the upper and lower edges of the intersection of the predicted box and the true box, respectively. p With Dw r It represents the distance between the left and right corresponding edges of the intersection of the predicted box and the real box; Wr and Hr represent the length and height of the real box respectively.

[0032] Further, in step S23, the hybrid local channel attention MLCA is integrated into the C2f module to form C2f-MLCA, and the steps include:

[0033] S231: extracting local spatial information from the input feature map by local pooling extraction;

[0034] S232: Divide the input feature map into two branches, one branch contains local spatial information through one-dimensional convolution processing, and the other branch processes global information through global pooling combined with one-dimensional convolution;

[0035] S233: Restoring the feature maps of the local spatial information and the global information to their original sizes through de-pooling, and fusing the two;

[0036] S234: Through the integrated C2f-MLCA module, the network's feature extraction capability for targets of different scales is improved while keeping the original computational burden low.

[0037] Further, in step S30, the weight file for the farmland pest detection is deployed on the onboard computer of the drone, and the real-time video stream of the farmland captured by the camera is used for pest detection by the onboard computer. The steps include:

[0038] S31: Build an NVIDIA Jetson TX2 NX embedded computing environment to support efficient deployment and real-time inference operations of deep learning models;

[0039] S32: Based on the NVIDIA Jetson TX2 NX embedded computing environment, the deep learning model is accelerated and optimized by using TensorRT. The accelerated reasoning optimization improves the reasoning speed and detection efficiency of the deep learning model in real-time video streams through precision conversion of network layer weights, memory access optimization, and parallel computing processing.

[0040] Further, in step S40, the farmland pest detection result generated by the onboard computer is transmitted to a ground workstation in real time by using the 4G communication module on the drone, and the ground workstation receives and displays the farmland pest detection result on the drone in real time, and the steps include:

[0041] S41: Through 4G wireless communication technology and Tailscale virtual LAN technology, the NVIDIA Jetson TX2NX device and the ground workstation are deployed in the same LAN environment, and the real-time detection data is transmitted between the embedded onboard computer of the drone and the ground server using the 4G mobile communication network to ensure the reliability and low latency of data transmission;

[0042] S42: Based on the same LAN environment, switch the computer operating system from Windows to Ubuntu, log in to the operation interface of NVIDIA Jetson TX2 NX through the remote control software VNC, enter the command to start ROS, run the pest detection node and publish the detection results, use Theora encoding technology to compress and resize the video stream, the ground workstation synchronously subscribes to and receives the real-time detection video, and stores and records the received video stream for subsequent analysis.

[0043] Further, in step S41, the NVIDIA Jetson TX2 NX device and the ground workstation are deployed in the same local area network environment through 4G wireless communication technology and Tailscale virtual local area network technology, and the steps include:

[0044] S411: Use a plug-and-play, driver-free and dial-free Qualcomm 4G LTE module to connect the Qualcomm 4G LTE module to the onboard computer of the drone to provide an independent cellular communication network for the onboard computer;

[0045] S412: confirming that the onboard computer has successfully accessed the network through the Qualcomm 4G LTE module and obtained a valid IP address, and confirming that the ground workstation has been connected to the network to ensure that both parties have effective network communication conditions;

[0046] S413: Install Tailscale software on the embedded onboard computer of the drone and the ground workstation respectively;

[0047] S414: starting the Tailscale software client on the ground workstation, creating and logging in to an account to access a virtual private network, and starting the Tailscale software service on the onboard computer of the drone, logging in to the same account to join the onboard computer to the virtual private network;

[0048] S415: confirming that the onboard computer and the ground workstation device are both online in the VPN network and can recognize and communicate with each other;

[0049] S416: By inputting the IP address of the embedded airborne computer in the Tailscale network on the ground workstation, the connectivity of the network link is verified and the stability of the network connection is ensured, thereby achieving stable data transmission between the embedded airborne computer and the ground workstation.

[0050] The present invention also includes a real-time detection system for farmland pests based on an unmanned aerial vehicle, comprising:

[0051] A farmland information collection module, including a drone and a camera, for collecting pest image information from the farmland area;

[0052] The pest detection module is set on the onboard computer and uses the improved YOLOv10 algorithm to process the images collected by the camera to identify pests in the farmland;

[0053] The video stream transmission module is used to transmit the detection results of the onboard computer to the ground workstation in real time through the 4G communication module.

[0054] The technical solution of the present invention can achieve the following technical effects:

[0055] The on-board computer can realize real-time image processing on site, and the detection results can be instantly transmitted to the ground workstation through the 4G communication module, which not only ensures the real-time nature of the data, but also reduces the lag in information processing when pests break out, and avoids the spread of pests caused by information delays;

[0056] Pests are automatically identified based on the improved YOLOv10 network model. The algorithm has been optimized and can accurately identify and count pests. Compared with traditional detection methods, the detection accuracy is significantly improved, especially for complex farmland scenes and small target pests. It has stronger adaptability and detection effect; by deploying weight files on the drone's onboard computer and performing real-time detection, the system can directly process images during flight and quickly obtain detection results. The real-time processing capability avoids information lag and ensures that prevention and control measures can be taken in time when pests break out, thereby reducing crop losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 This is a schematic diagram of a drone-based real-time detection device for farmland pests;

[0059] Figure 2 The figure is a flowchart of a method for real-time detection of agricultural pests based on drones;

[0060] Figure 3 The overall flow chart of the real-time detection method of agricultural pests based on drones;

[0061] Figure 4 This is the network architecture diagram of the YOLOv10 benchmark algorithm;

[0062] Figure 5 This is the network architecture diagram of the improved YOLOv10 algorithm;

[0063] Figure 6 Schematic diagram of PIoU;

[0064] Figure 7 It is the structural diagram of SPPF-DLK module;

[0065] Figure 8 It is a schematic diagram of the structure of the C2F-MLCA module;

[0066] Fig. 9 Optimize the flow chart for TensorRT fusion;

[0067] Fig.10 Deploy process graph for model inference;

[0068] Fig.11 This is a picture of the aphid detection experiment on the farm field;

[0069] Fig.12 This is a schematic diagram of the structure of the real-time detection system of agricultural pests using drones. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0072] like Figure 1 As shown: A real-time detection device for farmland pests based on drones, including: drones, flight control systems, remote controllers, cameras, camera remote controllers, onboard computers, 4G communication modules and ground workstations;

[0073] The drone is used to carry a camera and an onboard computer and perform inspection tasks in farmland areas. Since the device is used in an outdoor environment, the drone is required to have sufficient load capacity, bending resistance, wind resistance and strong endurance during stable flight. The Yuanhang ZD550 drone selected in the present invention is based on an "X"-shaped frame and is made of all-metal material, which increases the safety of flight. At the same time, the power module uses a 6S ultra-large capacity lithium battery, and the drone can reach 30 to 40 minutes of no-load flight time. The Haoying Letian series of electronic speed controllers are selected to drive the low-noise brushless motor to rotate, and the high-strength carbon fiber propeller is matched to make the drone take off smoothly.

[0074] The flight control system is installed on the drone to control the flight attitude of the drone and ensure flight stability. The flight control system uses the DJI N3 flight control system, which has a built-in dual redundant inertial measurement unit (IMU) and combines a calibration algorithm to increase flight reliability.

[0075] The remote controller communicates with the flight control system and is used to issue instructions to the flight control system to control the take-off, landing and flight of the drone;

[0076] The camera is installed on the drone to obtain image information of farmland. The camera carried by the drone is a HIKVISION DS-UVC-U168R industrial-grade camera, which has an 8-megapixel high-definition sensor and a 5.8-46.4mm zoom lens. It can collect high-definition images of tiny pests in complex environments, ensuring the detection accuracy and identification effect of farmland pests.

[0077] A camera remote controller, connected to the camera, used to adjust the focal length of the camera lens;

[0078] The onboard computer is arranged on the UAV and is used to receive image information acquired by the camera, perform image processing to identify pests in the farmland, and identify and count the identified pests. The present invention uses NVIDIA Jetson TX2 NX as a computing module, with a power consumption range of 7.5 to 15W and a size of only 100mm×80mm×29mm. In addition, NVIDIA Jetson TX2 NX uses a dual-core single-threaded NVIDIA Denver2 and a quad-core multi-threaded Cortex-A57 cluster to form a central processing unit (CPU), and carries 256 CUDA cores to form a graphics processing unit to accelerate computationally intensive detection algorithms. The low power consumption and small size of the device can minimize the impact on the endurance of the UAV.

[0079] A 4G communication module is connected to the onboard computer and is used to transmit the identified pest detection results from the onboard computer to the ground workstation; the present invention uses an integrated Qualcomm 4G LTE module and a 4G cellular network as a communication system to achieve data interaction between different components of the system;

[0080] The ground workstation is used to receive the pest detection results from the onboard computer and to display and analyze the detection results.

[0081] By using drones carrying cameras and onboard computers, it is possible to automatically and quickly obtain image information of pests in farmland, and identify and count pests in real time, reducing dependence on technicians and improving the efficiency and accuracy of detection. Especially when dealing with large-scale farmland pest detection, it can cover a larger area and reduce labor costs.

[0082] The device combines the improved YOLOv10 deep learning algorithm and the camera's high-resolution image acquisition function to more accurately identify pest species, reduce detection errors caused by subjectivity, and improve recognition accuracy, especially in complex farmland scenes (such as small target pest detection in uneven lighting and cluttered backgrounds).

[0083] On-site real-time image processing is achieved through the onboard computer, and the detection results are instantly transmitted to the ground workstation through the 4G communication module. This not only ensures the real-time nature of the data, but also reduces the lag in information processing during pest outbreaks and avoids the spread of pests caused by information delays.

[0084] By using drones to automatically perform farmland inspection tasks and remotely controlling the drones in combination with the flight control system and remote controller, the complexity of manual operations is reduced. In addition, the camera focal length can be remotely adjusted using the camera remote controller, which improves the flexibility and adaptability of the system and greatly reduces the workload of agricultural technicians.

[0085] As a preferred embodiment of the above embodiment, the camera is arranged at the bottom of the drone, and the flight control system and the 4G communication module are arranged at the top of the drone; on the one hand, by setting the camera at the bottom of the drone with the lens pointing vertically downward, a better viewing angle can be obtained, ensuring that the camera can effectively cover the entire farmland area, and improving the integrity and clarity of image acquisition, avoiding the camera from being blocked by drone components or the external environment, and ensuring high-quality input of images required for pest detection; on the other hand, by setting the camera at the bottom of the drone and the flight control system and the 4G communication module at the top, this up and down distribution method can optimize the center of gravity balance of the drone, help improve the stability and controllability of the drone during flight, especially maintain smooth flight in complex farmland environments, and ensure clear and accurate image acquisition.

[0086] Among them, the camera and 4G communication module are both connected to the onboard computer using a USB interface. Specifically, by using the USB interface to connect the camera and the 4G communication module to the onboard computer, good compatibility between devices of different models or brands can be ensured, reducing the complexity and cost of equipment selection and improving the flexibility of the system.

[0087] The present invention also includes a method for real-time detection of farmland pests based on drones, which is used in the above-mentioned real-time detection device for farmland pests based on drones, such as Figure 2 , 3 As shown, including:

[0088] S10: Use the camera carried by the drone to inspect the farmland area to collect images of farmland pests, segment and annotate the images of farmland pests, and construct a data set of farmland pest images; specifically, use the farmland pest images taken by the camera, the images collected are sugarcane aphids, and the image resolution is 2448×3264. The original aphid images are segmented, and the 2448×3264 resolution images are cut into 4×4 sub-images; after the image is segmented, 860 sub-images with sugarcane aphid targets are selected from all images to form an aphid dataset; the open source tool LabelImg is used to manually annotate the images, and the location information and category information of the target aphids are annotated; an aphid dataset in complex scenes is constructed based on the collected and annotated images, and the dataset is unified in YOLO format; the dataset is randomly divided into training set, validation set and test set, and the division ratio is 8:1:1;

[0089] S20: Based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection; the target detection benchmark model specifically adopts the YOLOv10 model. YOLO is an end-to-end target detection model, such as Figure 4 As shown in the figure, the network architecture diagram of the YOLOv10 benchmark algorithm is as follows: Figure 5 As shown in the figure, the network architecture diagram of the improved YOLOv10 algorithm. YOLOv10 is lightweight and high-speed. It has high detection accuracy while performing fast detection, and is very suitable as an identification model for farmland pest detection.

[0090] S30: deploy the weight file of farmland pest detection to the onboard computer of the drone, and use the onboard computer to perform pest detection on the real-time video stream of the farmland captured by the camera;

[0091] S40: Using the 4G communication module on the drone, the farmland pest detection results generated by the onboard computer are transmitted to the ground workstation in real time. The ground workstation receives and displays the farmland pest detection results on the drone in real time.

[0092] By using drones equipped with cameras to inspect farmland areas, the labor intensity and time consumption of traditional manual inspections are reduced. Drones can quickly cover large areas of farmland and collect high-resolution pest images in real time, ensuring the integrity and quality of image data. Pests are automatically identified based on the improved YOLOv10 network model. The algorithm has been optimized and can accurately identify and count different types of pests. Compared with traditional detection methods, the accuracy of detection is significantly improved, especially for complex farmland scenes and small target pests. It has stronger adaptability and detection effect. By deploying weight files on the drone's onboard computer and performing real-time detection, the system can directly process images during flight and quickly obtain detection results. The real-time processing capability avoids information lag and ensures that prevention and control measures can be taken in time when pests break out, thereby reducing crop losses.

[0093] The test results are transmitted to the ground workstation in real time through the 4G communication module, ensuring the immediacy of the test data and the stability of the transmission. Compared with the traditional method of downloading data after drone inspection, this method eliminates the data backhaul and delay issues and improves the timeliness of information processing.

[0094] The combination of drone inspection and the automatic identification function of the deep learning model has achieved full automation of pest detection and reduced the need for human intervention. At the same time, through the application of the improved YOLOv10 model, the system has a higher level of intelligence and can automatically respond to pest detection tasks in different environments.

[0095] As a preferred embodiment of the above, in step S20, based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection, wherein the network improved based on YOLOv10 mainly includes model lightweighting and performance optimization, and the steps include:

[0096] S21: Optimize the loss function of the YOLOv10 network and use the PIoU (Powerful-IoU) loss function instead of the CIoU (Complete Intersection over Union) loss function to improve the convergence speed and accuracy of the model and reduce training time;

[0097] S22: The deformable large kernel DLK (Deformable Large Kernel) is integrated into the SPPF (Spatial Pyramid Pooling-Fast) module to form SPPF-DLK, which can enhance the model's ability to recognize targets of different scales. The introduction of DLK (Deformable Large Kernel) improves the generalization performance of the model by increasing the model's adaptability to complex scenes. The combination of the SPPF module and DLK helps to process pest targets of different scales and improves the accuracy of detection, especially for small targets.

[0098] The deformable large convolution kernel (DLK) in the deformable large kernel attention mechanism Deformable-LKA (D-LKA) is introduced into the SPPF module of the YOLOv10 backbone. DLK combines the wide receptive field of the large convolution kernel and the flexibility of the deformable convolution, which can fully capture the contextual information in the image and process complex visual information.

[0099] This convolution kernel can adapt to different image features by dynamically adjusting the shape and size of the convolution kernel, improving the performance of the model when facing multi-scale targets, and to a certain extent improving the problems of uneven lighting and cluttered background in the data set. It also has strong adaptability to small targets such as farmland pests.

[0100] The following is a detailed description of SPPF-DLK. Figure 7 This is the structural diagram of the SPPF-DLK module:

[0101] The input features of the SPPF-DLK module first pass through a cascade structure composed of a convolution module and multiple maximum pooling to obtain the fourth scale feature, the fifth scale feature, the sixth scale feature and the seventh scale feature. Among them, through the cascade of three maximum pooling, the feature map is converted into three different scales of 5×5, 9×9, and 13×13, which are the fifth scale feature, the sixth scale feature and the seventh scale feature, respectively, and the fourth scale feature is the feature before pooling. The features of these four scales are spliced ​​to obtain the spliced ​​features, and then the spliced ​​features are connected through a residual connection including a deformable depth separable two-dimensional convolution (Deform-DWConv2D), a deformable depth separable scalable two-dimensional convolution (Deform-DW-D Conv2D) and a standard 2D convolution, and then the Hadamard product is performed with the spliced ​​features to obtain the output features of the SPPF-LSKA module;

[0102] S23: Integrate Mixed Local Channel Attention (MLCA) into the C2f module to form C2f-MLCA to improve the model's ability to detect small targets in complex backgrounds. MLCA can enhance the model's ability to extract target features, especially for scenes with small targets and complex backgrounds. Integrating it into the C2f module (the lightweight layer in YOLO) helps further improve the model's detection accuracy.

[0103] S24: Based on the optimized PIoU loss function, SPPF-DLK and C2f-MLCA, an improved YOLOv10 network training model is constructed, and a weight file for farmland pest detection is generated.

[0104] Use CUDA11.3, NVIDIA RTX 4090, and PyTorch 1.11.0 for model training. The initial learning rate is set to 0.01 and the learning rate is adjusted by the cosine annealing algorithm; the momentum factor is set to 0.937 to help accelerate the direction of the gradient vector and suppress the oscillation phenomenon; the weight decay coefficient is 0.0005; Stochastic Gradient Descent (SGD) is used as the parameter update strategy; the batch size is set to 8; COCO pre-trained weight yolov10n.pt is used for transfer learning, iterate 500 rounds, and the image is preprocessed online, mainly including pixel value normalization, random rotation, random flipping, random color transformation, Mosaic enhancement and other operations to enrich the data set.

[0105] Training aphid dataset: Feed the dataset into the improved YOLOv10 deep learning network and use GPU to accelerate the training model;

[0106] like Fig.11 As shown in the figure, the aphid detection experiment diagram in the farmland verifies the training effect and displays the detection results: in the training process of the improved YOLOv10 network model, the network converges through repeated iterations and optimization; in order to verify the robustness and accuracy of the training algorithm, the model is verified and evaluated in real time through the aphid test set, and finally the optimal model is obtained; the average detection accuracy, storage space and FPS are used as evaluation indicators to represent the detection effect, and the test results are output.

[0107]

[0108]

[0109] As shown in the table above, using PIoU to replace CIoU increases its mAP by 1.8% compared with the original YOLOv10. Using SPPF_DLK to replace SPPF increases its percentage by 2.4% compared with the original YOLOv10, but the FPS drops to 78.23. Finally, using C2f-MLCA to replace the fourth C2f of the backbone increases its percentage to 91.8%, which is a slight improvement over the original YOLOv10. The FPS is also increased to 114.18, which is not much different from the FPS of the original YOLOv10.

[0110] Among them, in step S21, the loss function of the YOLOv10 network is optimized, and the PIoU loss function is used instead of the CIoU loss function. PIoU abandons the use of the minimum diagonal length of the enclosing rectangle between the predicted box and the true box as the denominator of the penalty function, and instead selects the boundary distance between the predicted box and the true box as the penalty function variable, such as Figure 6 As shown, the model includes:

[0111]

[0112]

[0113] Where P is the penalty term for the distance between the predicted box and the real box boundary; L PIoU represents the PIoU loss function; U represents the union area of ​​the predicted box and the true box; Dh r With Dh p They represent the distances between the upper and lower edges of the intersection of the predicted box and the true box, respectively. p With Dw r It represents the distance between the left and right corresponding edges of the intersection of the predicted box and the real box; Wr and Hr represent the length and height of the real box respectively.

[0114] It can be seen from the above model that the penalty value of PIoU is only related to P, and the size of P has no direct relationship with the size of the minimum wrapping rectangle. The gradient of P will disappear only when the predicted box completely overlaps with the true box. Therefore, it is adaptable to target sizes of different sizes.

[0115] In this embodiment, in step S23, the hybrid local channel attention MLCA is integrated into the C2f module to form C2f-MLCA, and the steps include:

[0116] S231: extracting local spatial information from the input feature map through local pooling extraction (LAP);

[0117] S232: Divide the input feature map into two branches, one branch contains local spatial information through one-dimensional convolution processing, and the other branch processes global information through global pooling (GAP) combined with one-dimensional convolution;

[0118] S233: restore the feature maps of local spatial information and global information to their original sizes through unpooling (UNAP), and fuse the two;

[0119] S234: Through the integrated C2f-MLCA module, the network's feature extraction capability for targets of different scales is improved while keeping the original computational burden low, especially the detection accuracy of small targets.

[0120] In order to improve the model's ability to detect small targets, the present invention considers adding the attention mechanism to the convolution. On the basis of retaining the original C2f module, a hybrid local channel attention mechanism MLCA module is added after the second CBS module of the Bottleneck module in the C2f module to construct a new C2F-MLCA module. The structure of the C2F-MLCA module is as follows: Figure 8 shown.

[0121] Among them, the principle of the hybrid local channel attention mechanism (MLCA) is to first extract local spatial information by local pooling (LAP), and then use two branches to convert the input into a one-dimensional vector. The first branch contains local information after one-dimensional convolution, and the second branch contains global information after global pooling (GAP) and one-dimensional convolution. The original size of the vector is restored through unpooling (UNAP) to perform information fusion. The fused vector is restored to the same size as the input through unpooling (UNAP) to perform information fusion. On the one hand, such a structural design can avoid the image distortion problem caused by scaling the image. On the other hand, with a small increase in computational burden, it improves the network's extraction of feature information of different scales of the target.

[0122] As a preferred embodiment of the above, in step S30, the weight file for farmland pest detection is deployed on the onboard computer of the drone, and pest detection is performed on the real-time video stream of the farmland captured by the camera through the onboard computer, and the steps include:

[0123] S31: Build an NVIDIA Jetson TX2 NX embedded computing environment to support efficient deployment and real-time inference operations of deep learning models;

[0124] The NVIDIA Jetson TX2 NX platform environment is uniformly configured with the help of the JetPack (Jetson Development Pack) development tool kit officially provided by NVIDIA; the kit includes deep learning development toolkits such as CUDA, cuDNN, and TensorRT acceleration optimizer, and supports OpenCV computer vision library; JetPack development tools provide a reliable basic environment for building online deep learning networks; the present invention is based on the installation of JetPack4.6.4 on the Ubuntu18.04 LTS operating system in the Linux environment;

[0125] S32: Based on the NVIDIA Jetson TX2 NX embedded computing environment, it uses TensorRT to accelerate the inference optimization of deep learning models. Accelerated inference optimization improves the inference speed and detection efficiency of deep learning models in real-time video streams through precision conversion of network layer weights, memory access optimization, and parallel computing processing.

[0126] In NVIDIA Jetson TX2 NX, each layer of neural network operation requires GPU to call different CUDA cores to implement, and a lot of time is wasted on task scheduling and data reading and writing operations. To solve this problem, TensorRT significantly reduces the amount of calculation and memory access through inter-layer fusion, tensor fusion, quantization, parallel multi-stream input, memory optimization and other technologies. For the specific fusion process, see Fig. 9 ,The improved YOLOv10 algorithm is deployed on NVIDIA Jetson TX2 NX. To meet real-time detection requirements, TensorRT optimizer is used to accelerate reasoning, ,which improves the model detection efficiency while balancing the ,complexity and accuracy of the model.

[0127] During the deployment process, TensorRT optimization is divided into two parts: compilation and reasoning. For detailed structure, see Fig.10In the compilation phase, the RSCDet optimal model parameters are extracted to generate a .wts file, and the network structure is gradually built in combination with the underlying C++API to import the model. At the same time, the TensorRT acceleration component is used to optimize the network and compile to generate a high-performance inference engine; in the process of reducing deployment time and computing overhead, in order to avoid repeated compilation, the engine object is serialized and saved as an .engine file; the inference phase is the deserialization deployment process, which loads the serialized engine file from the memory buffer, restores the model structure and parameter information, and creates an execution unit according to the optimized model, including setting parallel CUDA streams, executing buffers, synchronizing information between different CUDA streams, etc., to ensure high throughput of multi-stream input processing and output results in sequence; when in use, in the deployment framework based on model deserialization, pre-processing operations are performed on the input data, and post-processing technologies such as the NMS non-maximum suppression algorithm, confidence threshold determination, and coordinate mapping are integrated for the detection results to achieve efficient inference, and finally the results are output through hardware devices.

[0128] In this embodiment, in step S40, the 4G communication module on the drone is used to transmit the farmland pest detection results generated by the onboard computer to the ground workstation in real time, and the ground workstation receives and displays the farmland pest detection results on the drone in real time. The steps include:

[0129] S41: Through 4G wireless communication technology and Tailscale virtual LAN technology, the NVIDIA Jetson TX2NX device and the ground workstation are deployed in the same LAN environment, and the 4G mobile communication network is used to transmit real-time detection data between the drone's embedded onboard computer and the ground server, ensuring data transmission reliability and low latency;

[0130] S42: Based on the same LAN environment, switch the computer operating system from Windows to Ubuntu, log in to the operation interface of NVIDIA Jetson TX2 NX through the remote control software VNC, enter the command to start ROS (Robot Operating System), run the pest detection node and publish the detection results, use Theora encoding technology to compress and resize the video stream, and the ground workstation simultaneously subscribes to and receives the real-time detection video, and stores and records the received video stream for subsequent analysis.

[0131] S421: command to start ROS;

[0132] Since the ROS operating system is a node-based architecture, each node is an executable process that can run between multiple computers and communicate through the ROS Master. The present invention uses ROS to call the industrial USB camera driver and start the node / usb_cam to subscribe to the camera data stream, and at the same time publishes the / usb_cam / image_raw image acquisition node, starts the algorithm detection node and then subscribes to / usb_cam / image_raw for processing, identifies the target and publishes the detection results. Topic asynchronous communication is adopted between nodes, and the real-time pest detection function is realized by subscribing to topics.

[0133] S422: video compression;

[0134] In the 4G mobile network environment, since ROS uses the TCP / IP protocol to transmit images, and the high-resolution video image data volume is large, it is easy to cause network bandwidth limitations. This limitation will further aggravate the network congestion problem and cause significant transmission delays, affecting the performance and responsiveness of the entire UAV system. In order to alleviate the challenges faced by image data transmission, the video size is adjusted and the Theora encoding technology is applied to compress the video.

[0135] As a preferred embodiment of the above, in step S41, the NVIDIA Jetson TX2 NX device and the ground workstation are deployed in the same local area network environment through 4G wireless communication technology and Tailscale virtual local area network technology, and the steps include:

[0136] S411: Use the plug-and-play, driver-free and dial-free Qualcomm 4G LTE module to connect the Qualcomm 4G LTE module to the drone's onboard computer to provide an independent cellular communication network for the onboard computer;

[0137] S412: Confirm that the onboard computer has successfully connected to the network through the Qualcomm 4G LTE module and obtained a valid IP address, and confirm that the ground workstation has been connected to the network to ensure that both parties have effective network communication conditions;

[0138] S413: Install Tailscale software on the drone's embedded onboard computer and the ground workstation respectively;

[0139] S414: Start the Tailscale software client on the ground workstation, create and log in to an account to access the virtual private network (VPN), start the Tailscale software service on the drone's onboard computer, and log in to the same account to join the onboard computer to the virtual private network (VPN);

[0140] S415: Confirm that the onboard computer and the ground workstation equipment are both online in the VPN network and can recognize and communicate with each other;

[0141] S416: By entering the IP address of the embedded airborne computer in the Tailscale network on the ground workstation, the connectivity of the network link is verified and the stability of the network connection is ensured, thereby achieving stable data transmission between the embedded airborne computer and the ground workstation.

[0142] This solution provides an independent cellular network for drones through 4G LTE modules, allowing drones to maintain stable communication with ground workstations over long distances and ensure that data can be transmitted back to the ground workstations in real time. It is suitable for operations in large areas of farmland. The use of Qualcomm 4G LTE modules provides plug-and-play functions without driver or dialing, which simplifies the deployment and use of equipment and improves the ease of operation of the system.

[0143] Through Tailscale virtual LAN technology, the solution creates a secure and stable network communication channel, connecting the drone's onboard computer and the ground workstation to the same virtual LAN, ensuring the security and integrity of data transmission. This technology can effectively avoid external network interference and data leakage.

[0144] Through Tailscale virtual LAN technology, the solution creates a secure and stable network communication channel, connecting the drone's onboard computer and ground workstation to the same virtual LAN, ensuring the security and integrity of data transmission, and effectively avoiding external network interference and data leakage.

[0145] The present invention also includes a real-time detection system for farmland pests based on drones, such as Fig.12 As shown, including:

[0146] A farmland information collection module, including a drone and a camera, for collecting pest image information from the farmland area;

[0147] The pest detection module is set on the onboard computer and uses the improved YOLOv10 algorithm to process the images collected by the camera to identify pests in the farmland;

[0148] The video stream transmission module is used to transmit the detection results of the onboard computer to the ground workstation in real time through the 4G communication module.

[0149] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A real-time detection device for farmland pests based on drones, characterized in that: Including drone, flight control system, remote controller, camera, camera remote controller, onboard computer, 4G communication module and ground workstation; The drone is used to carry the camera and the onboard computer and perform inspection tasks in the farmland area; The flight control system is arranged on the UAV and is used to control the flight attitude of the UAV and ensure the flight stability; The remote controller communicates with the flight control system and is used to issue instructions to the flight control system to control the take-off, landing and flight of the UAV; The camera is arranged on the drone and is used to obtain image information of the farmland; The camera remote controller is connected to the camera and is used to adjust the focal length of the camera lens; The onboard computer is disposed on the drone and is used to receive image information acquired by the camera, perform image processing to identify pests in the farmland, and count the identified pests; The 4G communication module is connected to the onboard computer and is used to transmit the identified pest detection results from the onboard computer to the ground workstation; The ground workstation is used to receive the pest detection results from the onboard computer and to display and analyze the detection results.

2. The real-time detection device for farmland pests based on drone according to claim 1 is characterized in that: The camera is arranged at the bottom of the drone, and the flight control system and the 4G communication module are arranged at the top of the drone.

3. A real-time detection method for farmland pests based on drones, characterized in that: The real-time detection device for farmland pests based on an unmanned aerial vehicle as claimed in any one of claims 1 to 2 comprises: S10: inspecting the farmland area by using the camera carried by the drone to collect images of farmland pests, segmenting and labeling the images of the farmland pests, and constructing a farmland pest image dataset; S20: Based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection; S30: deploying the weight file for farmland pest detection on the onboard computer of the drone, and performing pest detection on the real-time video stream of the farmland captured by the camera through the onboard computer; S40: Utilizing the 4G communication module on the UAV, the farmland pest detection results generated by the onboard computer are transmitted to a ground workstation in real time, and the ground workstation receives and displays the farmland pest detection results on the UAV in real time.

4. The method for real-time detection of farmland pests based on drones according to claim 3, characterized in that: In step S20, based on the farmland pest image dataset, an improved YOLOv10 network training model is constructed to generate a weight file for farmland pest detection, the steps comprising: S21: Optimize the loss function of the YOLOv10 network and use the PIoU loss function instead of the CIoU loss function to improve the convergence speed and accuracy of the model; S22: Integrate the deformable large convolution kernel DLK into the SPPF module to form SPPF-DLK to enhance the model's ability to recognize objects of different scales; S23: Integrate the hybrid local channel attention MLCA into the C2f module to form C2f-MLCA to improve the model's ability to detect small targets in complex backgrounds; S24: Based on the optimized PIoU loss function, the SPPF-DLK and the C2f-MLCA, an improved YOLOv10 network training model is constructed, and a weight file for farmland pest detection is generated.

5. The method for real-time detection of farmland pests based on drones according to claim 4, characterized in that: In step S21, the loss function of the YOLOv10 network is optimized, and the PIoU loss function is used instead of the CIoU loss function. The model includes: Where P is the penalty term for the distance between the predicted box and the real box boundary; L PIoU represents the PIoU loss function; U represents the union area of ​​the predicted box and the true box; Dh r With Dh p They represent the distances between the upper and lower edges of the intersection of the predicted box and the true box, respectively. p With Dw r It represents the distance between the left and right corresponding edges of the intersection of the predicted box and the real box; Wr and Hr represent the length and height of the real box respectively.

6. The method for real-time detection of farmland pests based on drones according to claim 4, characterized in that: In step S23, the hybrid local channel attention MLCA is integrated into the C2f module to form C2f-MLCA, and the steps include: S231: extracting local spatial information from the input feature map by local pooling extraction; S232: Divide the input feature map into two branches, one branch contains local spatial information through one-dimensional convolution processing, and the other branch processes global information through global pooling combined with one-dimensional convolution; S233: Restoring the feature maps of the local spatial information and the global information to their original sizes through de-pooling, and fusing the two; S234: Through the integrated C2f-MLCA module, the network's feature extraction capability for targets of different scales is improved while keeping the original computational burden low.

7. The method for real-time detection of farmland pests based on drones according to claim 3, characterized in that: In step S30, the weight file for the farmland pest detection is deployed on the onboard computer of the drone, and pest detection is performed on the real-time farmland video stream captured by the camera through the onboard computer. The steps include: S31: Build an NVIDIA Jetson TX2 NX embedded computing environment to support efficient deployment and real-time inference operations of deep learning models; S32: Based on the NVIDIA Jetson TX2 NX embedded computing environment, the deep learning model is accelerated and optimized by using TensorRT. The accelerated reasoning optimization improves the reasoning speed and detection efficiency of the deep learning model in real-time video streams through precision conversion of network layer weights, memory access optimization, and parallel computing processing.

8. The method for real-time detection of farmland pests based on drones according to claim 3, characterized in that: In step S40, the 4G communication module on the drone is used to transmit the farmland pest detection results generated by the onboard computer to a ground workstation in real time, and the ground workstation receives and displays the farmland pest detection results on the drone in real time. The steps include: S41: Through 4G wireless communication technology and Tailscale virtual LAN technology, the NVIDIA Jetson TX2 NX device and the ground workstation are deployed in the same LAN environment, and the real-time detection data is transmitted between the embedded onboard computer of the drone and the ground server using the 4G mobile communication network to ensure the reliability and low latency of data transmission; S42: Based on the same LAN environment, switch the computer operating system from Windows to Ubuntu, log in to the operation interface of NVIDIA Jetson TX2 NX through the remote control software VNC, enter the command to start ROS, run the pest detection node and publish the detection results, use Theora encoding technology to compress and resize the video stream, the ground workstation synchronously subscribes to and receives the real-time detection video, and stores and records the received video stream for subsequent analysis.

9. The method for real-time detection of farmland pests based on drones according to claim 8, characterized in that: In step S41, the NVIDIA Jetson TX2NX device and the ground workstation are deployed in the same local area network environment through 4G wireless communication technology and Tailscale virtual local area network technology, and the steps include: S411: Use a plug-and-play, driver-free and dial-free Qualcomm 4G LTE module to connect the Qualcomm 4G LTE module to the onboard computer of the drone to provide an independent cellular communication network for the onboard computer; S412: confirming that the onboard computer has successfully accessed the network through the Qualcomm 4G LTE module and obtained a valid IP address, and confirming that the ground workstation has been connected to the network to ensure that both parties have effective network communication conditions; S413: Install Tailscale software on the embedded onboard computer of the drone and the ground workstation respectively; S414: starting the Tailscale software client on the ground workstation, creating and logging in to an account to access a virtual private network, and starting the Tailscale software service on the onboard computer of the drone, logging in to the same account to join the onboard computer to the virtual private network; S415: confirming that the onboard computer and the ground workstation device are both online in the VPN network and can recognize and communicate with each other; S416: By inputting the IP address of the embedded airborne computer in the Tailscale network on the ground workstation, the connectivity of the network link is verified and the stability of the network connection is ensured, thereby achieving stable data transmission between the embedded airborne computer and the ground workstation.

10. A real-time detection system for farmland pests based on drones, characterized in that: include: A farmland information collection module, including a drone and a camera, for collecting pest image information from the farmland area; The pest detection module is set on the onboard computer and uses the improved YOLOv10 algorithm to process the images collected by the camera to identify pests in the farmland; The video stream transmission module is used to transmit the detection results of the onboard computer to the ground workstation in real time through the 4G communication module.

Images