Plant disease and insect pest detection equipment and method based on UAV high-altitude texture imaging

Abstract

This invention discloses a plant disease and insect pest detection device and method based on high-altitude texture imaging from an unmanned aerial vehicle (UAV), belonging to the field of smart agriculture technology. The device comprises an unmanned aerial vehicle (UAV) main body, a gimbal, a high-altitude texture camera, and a small host computer. The high-altitude texture camera is located below the gimbal, which is located below the small host computer, which is located below the UAV main body. The UAV main body is equipped with a front-facing high-definition camera, a battery, and a light sensor. The high-altitude texture camera is wirelessly connected to a handle and a flight remote control via Bluetooth. The plant disease and insect pest detection method is based on high-altitude texture feature fusion, combined with an environmental perception-driven data enhancement module, an adaptive multi-scale graph convolution feature extraction and fusion module, and a pest and disease region attention fusion module, achieving high-precision pest and disease identification. This technology effectively supports the precise detection of pests and diseases in agricultural production and provides a scientific basis for agricultural management and pesticide application.

Description

Technical Field

[0001] The present invention belongs to the field of smart agriculture technology, and in particular relates to plant disease and insect pest detection equipment and methods based on unmanned aerial vehicle (UAV) high-altitude texture imaging. Background Art

[0002] Traditional imaging technologies, used in agriculture and other fields, have numerous limitations, including limited field of view and coverage, reliance on manual operation, high costs, and difficulty in dynamic monitoring and multi-angle observation. The limitations of field of view and coverage are particularly critical, hindering effective data collection and accurate analysis. However, the emergence of drone-based high-altitude texture imaging technology offers a novel approach to addressing these issues. This innovative image acquisition and processing method can accurately capture a wide range of surface texture information. Furthermore, high-quality high-altitude texture images can significantly enhance the object recognition capabilities of deep learning models.

[0003] In agriculture, drone-mounted texture imaging technology can capture rich texture information on vegetation surfaces from a bird's-eye view. This information includes key features such as leaf morphology, texture, and color. This texture information provides crucial data support and decision-making for agricultural production. Subsequently, using image processing and deep learning algorithms, we can precisely analyze this collected data to accurately identify and locate potential plant pests and diseases. This comprehensive data coverage helps promptly identify plant health issues and implement appropriate measures to reduce crop losses caused by pests and diseases, thereby improving production efficiency and quality. It can also promote the development of macrophenotyping in plant pest and disease research. Therefore, this technology is of great significance in the field of smart agriculture, enabling applications in plant health monitoring, pest and disease monitoring and control, precision fertilization and irrigation, and farmland planning and management.

[0004] The implementation of drone-based high-altitude texture imaging and plant pest and disease identification methods primarily relies on the synergy of a series of devices, including drones, gimbals, small hosts, touch screens, accessory interfaces, and high-definition cameras. Pest and disease identification is accomplished through a multi-scale feature-based high-altitude texture feature fusion algorithm. The parameters of high-definition cameras and lenses are crucial for acquiring high-altitude texture images, requiring analysis of complex physical imaging principles. Furthermore, identifying pests and diseases in high-altitude texture images is also challenging. Therefore, there is an urgent need for a plant pest and disease detection device and method based on drone-based high-altitude texture imaging that can overcome the difficulties inherent in the aforementioned technologies, thereby better identifying plant pests and diseases in the field and providing more accurate management and decision-making support for agricultural production. Summary of the Invention

[0005] The purpose of this invention is to provide a plant pest and disease detection device and method based on high-altitude texture imaging from drones. This device, which uses a plant pest and disease identification method that integrates high-altitude texture features, enables precise pest and disease detection. The device can capture high-definition texture images of plant surfaces at high altitudes. Combined with data enhancement, adaptive multi-scale feature extraction and fusion, and pest and disease area attention fusion, this improves detection accuracy and real-time performance.

[0006] To achieve the above-mentioned objectives, the present invention provides a plant disease and pest detection device based on drone high-altitude texture imaging, comprising a drone body, a gimbal, a high-altitude texture camera and a small host; the high-altitude texture camera is arranged below the gimbal, the gimbal is arranged below the small host, and the small host is arranged below the drone body; the drone body is equipped with a front-facing high-definition camera, a battery, and a light sensor, and the high-altitude texture camera is wirelessly equipped with a handle and a flight remote control via Bluetooth.

[0007] Preferably, the high-altitude texture camera is composed of a 61-megapixel camera and a lens with a focal length of 70 mm to 200 mm. The 61-megapixel camera and the lens with a focal length of 70 mm to 200 mm are determined by the Gaussian imaging formula, and the specific expression is as follows:

[0008]

[0009] Among them, u is the object distance, v is the image distance, and f is the focal length.

[0010] The present invention also provides a plant disease and insect pest detection method based on drone high-altitude texture imaging, comprising the following steps:

[0011] Step 1: Train the plant disease and insect pest recognition model;

[0012] Step 2: Deploy the trained plant disease and insect pest recognition model to the small host;

[0013] Step 3: Capture images using a high-altitude texture camera, and transmit the captured image data to the small host computer deployed with the plant disease and insect pest identification model in step 2 for detection.

[0014] Preferably, the specific process of training the plant disease and insect pest recognition model in step 1 is as follows:

[0015] Step 101: adjusting the brightness, contrast, and saturation of the image according to the real-time light intensity, collecting plant disease and insect pest image data, and performing data preprocessing;

[0016] Step 102: input the preprocessed data obtained in step 101 into the ResNet-101 model, and train the ResNet-101 model to learn and extract features of plant pests and diseases texture, shape, and structure from the plant pests and diseases image, and convert the extracted features into a high-dimensional feature vector f0;

[0017] Step 103: Input the high-dimensional feature vector f0 extracted in step 102 into the 3×3 convolution layer and the normalization layer respectively to obtain the attention map A focusing on the adaptive nearest neighbor K. opt ;

[0018] Step 104: Input the high-dimensional feature vector f0 extracted in step 102 into the graph convolution layer and the global average pooling layer with the nearest neighbor set {K1=3, K2=5, K3=7} to obtain the multi-scale feature set {f1, f2, f3, f a};

[0019] Step 105: Through the multi-scale feature set {f1,f2,f3,f a} and the attention map A of the adaptive nearest neighbor K opt Fusion is performed to obtain a multi-scale fusion feature map f m ;

[0020] Step 106: The multi-scale fusion feature map f obtained in step 105 is m After 3×3 convolution layer, fully connected layer and activation layer and f m Add itself to obtain the spatial-channel feature enhancement map F;

[0021] Step 107: Map the spatial-channel feature enhancement map F obtained in step 106 to the category space through a fully connected layer, and use the softmax classifier to calculate the probability distribution of each category and output the final classification result.

[0022] Preferably, the specific expression for enhancing illumination perception in step 101 is as follows:

[0023] I aug =AL(I,LI);

[0024] Among them, I is the plant disease image, I aug is the enhanced image, LI is the real-time illumination intensity, and AL(·) is the adaptive function that adjusts the image brightness according to the illumination intensity.

[0025] Preferably, the specific expression of the feature extraction process in step 102 is as follows:

[0026] f0=R(I aug ;φ);

[0027] Among them, φ is the parameter of the ResNet-101 model, R represents the ResNet-101 model, and f0 is the output feature vector.

[0028] Preferably, the specific expression of the attention map of the adaptive nearest neighbor K in step 103 is as follows:

[0029] A opt =Conv(Norm(Conv(f0)));

[0030] Among them, Conv(·) represents the convolution operation on image I, and Norm(·) is the normalization operation, which is used to standardize the feature map after convolution to ensure its numerical stability during training.

[0031] Preferably, in step 105, a multi-scale fusion feature map f is obtained. m The specific expression is as follows:

[0032] f m =(f1+f2+f3+f a )×A opt .

[0033] Preferably, the specific expression of the space-channel feature enhancement map F in step 106 is as follows:

[0034] F=σ(Conv(f m ))*σ(FC(f m ))+f m ;

[0035] Among them, FC(.) is the fully connected layer and σ is the activation function.

[0036] Preferably, the specific expression of mapping the spatial-channel feature enhancement map F to the category space through the fully connected layer in step 107 is as follows:

[0037]

[0038] Among them, P(a i |b) indicates that the given input image b belongs to category a i The probability of is the i-th element of the j-th category output by the fully connected layer, and n is the total number of categories.

[0039] Therefore, the present invention adopts the above-mentioned plant disease and insect pest detection equipment and method based on drone high-altitude texture imaging, which has the following beneficial effects:

[0040] (1) Comprehensive application areas: The use of UAV high-altitude texture imaging equipment and methods can quickly and efficiently detect plant diseases and insect pests in the field, and provide accurate and detailed data support for research, monitoring and decision-making in agriculture, forestry, environmental monitoring, urban planning and other fields. It has broad application prospects and market potential.

[0041] (2) High-quality texture image acquisition: The professional combination of a 61-megapixel full-frame micro-single digital camera and a lens with a focal length of 70mm to 200mm greatly improves the quality of high-altitude imaging.

[0042] (3) Efficient pest and disease identification method: The pest and disease detection method uses an environmental perception-driven data enhancement module, an adaptive multi-scale graph convolution feature extraction and fusion module, and a pest and disease regional attention fusion module to enhance the ability to extract plant pest and disease features. Compared with traditional single-scale extraction methods, this model can achieve higher resolution in detail feature extraction, thereby improving the accuracy of pest and disease detection. The plant pest and disease identification model is trained using the above method and deployed on a small host, thereby achieving real-time image processing and recognition without relying on external network connections or cloud servers, greatly improving the reliability and practicality of the system.

[0043] (4) Real-time recognition capability with rapid response: Deploy the trained model to the drone equipment to achieve real-time recognition of plant diseases and pests in images, helping farmers to detect and deal with diseases and pests in a timely manner and reduce crop losses.

[0044] (5) Accurate shooting angle adjustment: The drone's front-mounted HD camera can monitor the images sent back in real time. Through the flight controller and Ronin 4D handle combined with the DJI RS 3Pro gimbal, the flight path and shooting angle can be accurately adjusted. Combined with the autofocus function, the clarity and quality of the captured images can be ensured.

[0045] (6) Flexible and convenient equipment matching: The DJI RS 3Pro gimbal used has a quick-release design and is equipped with compatible accessories, making the installation, storage and use of the equipment more flexible and convenient, improving the convenience and comfort of operation.

[0046] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is the overall structural diagram of the plant disease and insect pest detection equipment based on UAV high-altitude texture imaging of the present invention;

[0048] Figure 2 A top view of the plant disease and insect pest detection device based on drone high-altitude texture imaging of the present invention;

[0049] Figure 3 A schematic diagram of a high-altitude texture camera and its imaging process according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic structural diagram of a handle according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of the structure of a flight remote controller according to an embodiment of the present invention;

[0052] Figure 6 Schematic diagram of the framework of the plant disease and insect pest identification method based on the fusion of multi-scale features and high-altitude texture features of the present invention;

[0053] Figure 7 Schematic diagram of feature visualization of the plant disease and insect pest identification method based on multi-scale high-altitude texture feature fusion of the present invention;

[0054] Among them, 1. Small host; 2. Drone body; 3. High-altitude texture camera; 4. Gimbal; 5. Front HD camera; 6. Photo button; 7. Shooting angle selection button; 8. Flight mode button; 9. Left joystick; 10. Zoom button; 11. Right joystick; 12. Light sensor. DETAILED DESCRIPTION

[0055] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0056] See also Figure 1-7 A plant pest and disease detection device based on drone high-altitude texture imaging includes a drone body 2, a gimbal 4, a high-altitude texture camera 3, and a small host 1. The high-altitude texture camera 3 is located below the gimbal 4, which is located below the small host 1. The small host 1 is located below the drone body 2. The drone body 2 is equipped with a front-facing high-definition camera 5, a battery, and a light sensor 12. The high-altitude texture camera 3 is wirelessly equipped with a handle and a flight remote control via Bluetooth. The high-altitude texture camera 3 is composed of a 61-megapixel camera and a lens with a focal length of 70mm to 200mm. The 61-megapixel camera and the lens with a focal length of 70mm to 200mm are determined by the Gaussian imaging formula. The Gaussian imaging formula can describe the relationship between lens imaging and determine the positional relationship between the object and the image, which can help us choose the best camera and lens. The specific expression is as follows:

[0057]

[0058] Among them, u is the object distance (this distance is the length between the lens and the plant leaves, which is about 10 meters. At this time, the impact of the high-altitude drone on the photographed object is relatively small), v is the image distance, and f is the focal length.

[0059] To further verify the availability of the selected camera and lens, we used the camera internal and external parameter calculation formula to verify and prove that the camera and lens we matched are effective. The formula is as follows:

[0060]

[0061] Among them, the focal length f = 70mm ~ 200mm, the object distance u = 10m, the length and width of the SONY Alpha 7R V full-frame micro single digital camera are x = 35.7mm and y = 23.8mm respectively, and the length and width of the photographed crop are x c =1.8m and y c =1.2m.

[0062] Among them, the DJI RS 3 Pro gimbal, Ronin 4D handle, 61-megapixel SONY Alpha 7R V full-frame micro-single digital camera, and SONY FE 24-70mm F2.8 GM OSS II lens with a focal length of 70mm to 200mm are selected, which can support the acquisition of high-definition texture images of objects with a size of 1.8m×1.2m; the configured drone's front HD camera provides image information in front of the flight path, and the flight remote control can be used to accurately control the drone's flight. It is also equipped with two high-capacity batteries to support flight; the bottom interface of the small host is connected to the DJI RS 3 Pro gimbal, and the Bluetooth function is used to realize data transmission between devices. It also deploys a low-parameter plant disease and insect pest recognition model; DJI RS The 3Pro gimbal features a quick-release design and is a professional-grade three-axis stabilized gimbal specifically designed for HD cameras. This makes use, storage, installation, and leveling more flexible and convenient, and supports a 4.5 kg payload. The Ronin4D handle and flight controller wirelessly connect to the high-altitude texture camera via Bluetooth, enabling autofocus, remote monitoring, and collaborative shooting. A light sensor captures light intensity data and assists the HD camera in adjusting image brightness, contrast, and saturation. During the high-altitude texture image acquisition process, all data transmission and operation commands are handled by the small host computer and its Bluetooth-enabled device. To find the perfect shooting angle, the flight controller displays the image feed from the drone's front-mounted HD camera, allowing for timely flight adjustments using the left and right joysticks (9 and 11). The DJI RS 3 Pro gimbal controls the shooting angle of the HD camera using the shooting angle selector button 7 on the Ronin 4D's handle. The zoom button 10 automatically adjusts the lens focus. Finally, the photo button 6 is used to take photos, and high-quality images with high-altitude textures are transmitted to the remote controller for display. The flight mode button 8 controls the drone's flight state. During takeoff, the mode must be switched from "Disabled" to "Unlocked" to allow takeoff. While collecting photos along a pre-planned flight route, adjustments can be made on the fly using the two control instruments, significantly improving data collection quality and efficiency. The DJI RS 3 Pro gimbal then adjusts the multispectral camera's shooting angle, but focusing is not necessary. By keeping the drone steady and waiting for the instrument to complete image acquisition, multispectral images with different band information can be viewed on the remote controller. This eliminates the hassle of constantly adjusting the equipment when collecting photos from the ground, significantly improving image acquisition speed.

[0063] A plant disease and insect pest detection method based on drone high-altitude texture imaging includes the following steps:

[0064] Step 1: Train the plant disease and insect pest recognition model; the specific process is as follows:

[0065] Step 101: Adjust the brightness, contrast, and saturation of the image according to the real-time light intensity, collect plant disease and insect pest image data, and perform data preprocessing. The specific expression of image enhancement is as follows:

[0066] I aug =AL(I,LI);

[0067] Among them, I is the plant disease image, I aug is the enhanced image, LI is the real-time illumination intensity, and AL(·) is the adaptive function that adjusts the image brightness according to the illumination intensity.

[0068] Step 102: Input the preprocessed data obtained in step 101 into the ResNet-101 model, and train the ResNet-101 model to learn and extract features of plant pests and diseases texture, shape, and structure from the plant pest and disease images, and convert the extracted features into a high-dimensional feature vector f0; wherein, the specific expression of the feature extraction process is as follows:

[0069] f0=R(I aug ;φ);

[0070] Among them, φ is the parameter of the ResNet-101 model, R represents the ResNet-101 model, and f0 is the output feature vector.

[0071] Step 103: Input the high-dimensional feature vector f0 extracted in step 102 into the 3×3 convolution layer and the normalization layer respectively to obtain the attention map A focusing on the adaptive nearest neighbor K. opt ; Among them, the specific expression for obtaining the attention map of the adaptive nearest neighbor K is as follows:

[0072] A opt =Conv(Norm(Conv(f0)));

[0073] Among them, Conv(·) represents the convolution operation on image I, and Norm(·) is the normalization operation, which is used to standardize the feature map after convolution to ensure its numerical stability during training.

[0074] Step 104: Input the high-dimensional feature vector f0 extracted in step 102 into the graph convolution layer and the global average pooling layer with the nearest neighbor set {K1=3, K2=5, K3=7} to obtain the multi-scale feature set {f1, f2, f3, f a}

[0075] Step 105: Through the multi-scale feature set {f1,f2,f3,f a} and the attention map A of the adaptive nearest neighbor K opt Fusion is performed to obtain a multi-scale fusion feature map f that retains more multi-scale information m ; Among them, obtain the multi-scale fusion feature map f m The specific expression is as follows:

[0076] f m =(f1+f2+f3+f a )×A opt .

[0077] Step 106: Multi-scale fusion feature map f m After 3×3 convolution layer, fully connected layer and activation layer and f m Add it to itself to obtain the spatial-channel feature enhancement map F; the specific expression for obtaining the spatial-channel feature enhancement map F is as follows:

[0078] F=σ(Conv(f m ))*σ(FC(f m ))+f m ;

[0079] Among them, Conv(·) is the convolution operation, FC(·) is the fully connected layer, and σ is the activation function.

[0080] Step 107: Map the spatial-channel feature enhancement map F to the category space through the fully connected layer, calculate the probability distribution of each category using the softmax classifier, and output the final classification result. The specific expression for mapping the spatial-channel feature enhancement map F to the category space through the fully connected layer is as follows:

[0081]

[0082] Among them, P(a i |b) indicates that the given input image b belongs to category a i The probability of is the i-th element of the j-th category output by the fully connected layer, and n is the total number of categories

[0083] Step 2: Deploy the trained plant disease and insect pest identification model to a small host. By embedding the plant disease and insect pest identification model directly into the device, we can achieve real-time image processing and recognition without relying on external network connections or cloud servers, greatly improving the reliability and practicality of the system.

[0084] Step 3: Use the high-altitude texture camera to capture images and transmit the captured image data to the small host computer deployed with the plant pest and disease recognition model in Step 2 for detection. Specifically, once the high-altitude texture camera captures the image, the data is transmitted to the small host computer for processing. To improve recognition accuracy, these high-altitude texture images are typically cut into 54 small images of uniform size. This operation helps reduce the burden on the model, allowing for more precise detection of signs of plant pests and diseases, thereby improving recognition efficiency. In addition, this also enables the model to better process different parts of the image, thus better adapting to various complex environments and scenarios. This allows us to promptly identify potential pest and disease problems throughout the image and take appropriate measures to address them, reducing reliance on chemical pesticides and protecting crop growth and yield.

[0085] pass Figure 7 We can see that our proposed method can focus on the characteristics of plant diseases and can still identify them regardless of whether the disease area is large or small. This shows that the proposed method can achieve good recognition results for diseases at both long and short distances.

[0086] Therefore, the present invention adopts the above-mentioned plant disease and pest detection equipment and method based on drone high-altitude texture imaging, which can obtain more high-altitude texture features of plant diseases and pests, thereby realizing accurate identification of plant diseases and pests, and providing more accurate management and decision-making support for agricultural production.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A plant disease and insect pest detection method based on high-altitude texture imaging by unmanned aerial vehicles, characterized by: The drone includes a main body, a gimbal, a high-altitude texture camera, and a small host. The high-altitude texture camera is arranged below the gimbal, the gimbal is arranged below the small host, and the small host is arranged below the main body of the drone. The main body of the drone is equipped with a front-facing high-definition camera, a battery, and a light sensor. The high-altitude texture camera is wirelessly equipped with a handle and a flight remote controller via Bluetooth. The high-altitude texture camera is composed of a 61-megapixel camera and a lens with a focal length of 70mm to 200mm. The 61-megapixel camera and the lens with a focal length of 70mm to 200mm are determined by the Gaussian imaging formula. The specific expression is as follows: ； in, is the object distance, v is the image distance, and f is the focal length; The following steps are involved: Step 1: Train the plant disease and insect pest recognition model; Step 2: Deploy the trained plant disease and insect pest recognition model to the small host; Step 3: Capture images using a high-altitude texture camera, and transmit the captured image data to the small host computer deployed with the plant disease and insect pest identification model in step 2 for detection; The specific process of training the plant disease and insect pest recognition model in step 1 is as follows: Step 101: adjusting the brightness, contrast, and saturation of the image according to the real-time light intensity, collecting plant disease and insect pest image data, and performing data preprocessing; Step 102: Input the preprocessed data obtained in step 101 into the ResNet-101 model and train the ResNet-101 model to learn and extract features of plant pests and diseases texture, shape and structure from the plant pests and diseases images, and convert the extracted features into high-dimensional feature vectors. ; Step 103: The high-dimensional feature vector extracted in step 102 Input into 3×3 convolution layer and normalization layer respectively to obtain the adaptive nearest neighbor Attention map ; Step 104: The high-dimensional feature vector extracted in step 102 Input to the nearest neighbor set is { }Graph convolution layer and global average pooling layer to obtain multi-scale feature sets ; Step 105: Multi-scale feature set and adaptive nearest neighbor Attention map Fusion is performed to obtain a multi-scale fusion feature map ; Step 106: The multi-scale fusion feature map obtained in step 105 is After 3×3 convolution layer, fully connected layer and activation layer and Add itself to obtain the spatial-channel feature enhancement map ; Step 107: Enhance the spatial-channel feature map obtained in step 106 After being mapped into the category space through the fully connected layer, the softmax classifier is used to calculate the probability distribution of each category and output the final classification result.

2. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 1 is characterized in that: The specific expression of the illumination perception enhancement in step 101 is as follows: ； in, For plant disease images, For the enhanced image, is the real-time light intensity, is an adaptive function that adjusts the image brightness according to the light intensity.

3. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 2 is characterized in that: The specific expression of the feature extraction process in step 102 is as follows: ； in, are the parameters of the ResNet-101 model, represents the ResNet-101 model, is the output feature vector.

4. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 3 is characterized in that: Adaptive nearest neighbor in step 103 The specific expression of the attention map is as follows: ； in, Represents the image The convolution operation, It is a normalization operation.

5. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 4 is characterized in that: In step 105, a multi-scale fusion feature map is obtained. The specific expression is as follows: 。 6. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 5 is characterized in that: The spatial-channel feature enhancement map in step 106 The specific expression is as follows: ； in, is the fully connected layer, is the activation function.

7. The plant disease and insect pest detection method based on drone high-altitude texture imaging according to claim 6 is characterized in that: The spatial-channel feature enhancement map in step 107 The specific expression mapped to the category space through the fully connected layer is as follows: ； in, Represents a given input image Belong to category The probability of is the jth category of the fully connected layer output elements, is the total number of categories.

Images