Pest and Disease Monitoring System Based on Plant Protection UAVs

Abstract

This application relates to the field of pest and disease monitoring. Specifically, it discloses a pest and disease monitoring system based on a plant protection UAV. The system uses the plant protection UAV to collect crop images, and introduces image processing technology based on deep learning in the backend server. Through multi-scale analysis of local features and global features of the collected crop images, it can simultaneously capture local lesions and overall growth conditions of the crops. Furthermore, through interactive fusion analysis of the multi-scale state features of the crop images, it can achieve accurate understanding and intelligent identification of the pest and disease states of the crops. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thus providing strong technical support for agricultural production.

Description

Technical Field

[0001] The present application relates to the field of pest and disease monitoring, and more specifically, to a pest and disease monitoring system based on a plant protection drone. Background Art

[0002] In agricultural production, pests and diseases are one of the key factors affecting the healthy growth and yield of crops. With the advancement of agricultural modernization, pest and disease control in agricultural production has become increasingly important. Traditional pest and disease monitoring methods mainly rely on manual inspections and sampling analysis, which is not only inefficient and consumes a lot of manpower and material resources, but also difficult to achieve real-time monitoring of large areas of farmland. In addition, due to the different growth cycles of crops and the randomness of the time and place of pest and disease occurrence, traditional methods often cannot detect the occurrence of pests and diseases in a timely and accurate manner, resulting in missing the best prevention and control period and causing a decrease in crop yield and quality.

[0003] In recent years, the development of unmanned aerial vehicle (UAV) technology has provided new solutions for agricultural monitoring. Plant protection UAVs have been widely used in the agricultural field due to their advantages such as flexibility, wide operating range and relatively low cost. In particular, plant protection UAVs equipped with high-definition cameras and other sensors can quickly obtain high-resolution images of large areas of farmland, providing a rich source of data for monitoring crop health.

[0004] However, the manifestations of crop diseases and pests in images are complex and diverse, with both local lesions (such as spots and wormholes on leaves) and overall growth conditions (such as plant height and color changes). Traditional image analysis methods are usually based on manually designed feature extraction algorithms, such as color histograms, texture features (such as grayscale co-occurrence matrices), shape features, etc., to describe target objects or abnormal conditions in images. These methods work well when dealing with simple and regular targets, but when faced with the complex task of crop disease and pest monitoring, it is usually difficult to fully capture the complex manifestations of diseases and pests in images. This results in low monitoring accuracy.

[0005] Therefore, an optimized pest and disease monitoring system based on plant protection drones is expected. Summary of the invention

[0006] To solve the above technical problems, the present application is proposed. Embodiments of the present application provide a pest and disease monitoring system based on a plant protection UAV, which uses the plant protection UAV to collect crop images, and introduces image processing technology based on deep learning in the back-end server. By performing multi-scale analysis of local features and global features on the collected crop images, local lesions and overall growth conditions of the crops can be captured simultaneously. Furthermore, by performing interactive fusion analysis on the multi-scale state features of the crop images, accurate understanding and intelligent identification of the pest and disease states of the crops can be achieved. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0007] Correspondingly, according to one aspect of the present application, there is provided a pest and disease monitoring system based on a plant protection UAV, which includes:

[0008] A crop image acquisition module, configured to control a plant protection UAV equipped with a camera to fly along a predetermined route and use the camera to collect crop images;

[0009] An image import module, configured to import the crop images into a back-end pest and disease monitoring server;

[0010] A crop image multi-scale feature detection module, configured to perform multi-scale feature detection on the crop images in the back-end pest and disease monitoring server to obtain a local feature coding vector of the crop image and a global semantic coding vector of the crop image;

[0011] A multi-scale feature interactive fusion module, configured to perform attention interactive fusion based on latent collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of the crop image features;

[0012] A monitoring result generation module, configured to determine a pest and disease monitoring result based on the multi-scale fusion coding vector of the crop image features.

[0013] Compared with the prior art, the pest and disease monitoring system based on a plant protection UAV provided by the present application uses the plant protection UAV to collect crop images, and introduces image processing technology based on deep learning in the back-end server. By performing multi-scale analysis of local features and global features on the collected crop images, local lesions and overall growth conditions of the crops can be captured simultaneously. Furthermore, by performing interactive fusion analysis on the multi-scale state features of the crop images, accurate understanding and intelligent identification of the pest and disease states of the crops can be achieved. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production. Description of the Drawings

[0014] The above and other objects, features, and advantages of the present application will become more apparent by describing the embodiments of the present application in more detail with reference to the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 FIG. is a block diagram of a pest and disease monitoring system based on a plant protection unmanned aerial vehicle according to an embodiment of the present application.

[0016] Figure 2 FIG. is a schematic diagram of data flow of a pest and disease monitoring system based on a plant protection unmanned aerial vehicle according to an embodiment of the present application.

[0017] Figure 3 FIG. is a block diagram of a multi-scale feature detection module for crop images in a pest and disease monitoring system based on a plant protection unmanned aerial vehicle according to an embodiment of the present application.

[0018] Figure 4 FIG. is a block diagram of a multi-scale feature interaction and fusion module in a pest and disease monitoring system based on a plant protection unmanned aerial vehicle according to an embodiment of the present application. Detailed Embodiments

[0019] As shown in the present application and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0021] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. On the contrary, as needed, various steps can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0022] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0023] It should be noted that in the present application, all actions of obtaining data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining authorization from the owner of the corresponding device.

[0024] In view of the technical problems described in the above background art, the present application proposes an optimized pest and disease monitoring system based on a plant protection unmanned aerial vehicle (UAV). It uses the plant protection UAV to collect crop images, and introduces image processing technology based on deep learning in the back-end server. By performing multi-scale analysis of local features and global features on the collected crop images, it can simultaneously capture local lesions and overall growth conditions of the crops. Furthermore, through interactive fusion analysis of the multi-scale state features of the crop images, it can achieve accurate understanding and intelligent identification of the pest and disease states of the crops. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0025] Figure 1 FIG. is a block diagram of a pest and disease monitoring system based on a plant protection UAV according to an embodiment of the present application. Figure 2 FIG. is a schematic diagram of data flow of a pest and disease monitoring system based on a plant protection UAV according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the pest and disease monitoring system 100 based on a plant protection UAV includes: a crop image acquisition module 110, configured to control a plant protection UAV equipped with a camera to fly along a predetermined route and collect crop images using the camera; an image import module 120, configured to import the crop images into a back-end pest and disease monitoring server; a crop image multi-scale feature detection module 130, configured to perform multi-scale feature detection on the crop images in the back-end pest and disease monitoring server to obtain a local feature coding vector of the crop image and a global semantic coding vector of the crop image; a multi-scale feature interactive fusion module 140, configured to perform attention interactive fusion based on latent collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of the crop image features; and a monitoring result generation module 150, configured to determine a monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image features.

[0026] In the above-mentioned pest and disease monitoring system based on plant protection drones, the crop image acquisition module 110 is used to control a plant protection drone equipped with a camera to fly along a predetermined route and collect crop images using the camera. It should be understood that collecting crop images by drones can cover a larger area of farmland and is not restricted by terrain, greatly improving the efficiency and quality of data collection.

[0027] In modern agricultural production, in order to ensure that drones can efficiently and accurately complete the predetermined tasks, that is, fly along the predetermined route and collect crop images using the camera, multiple factors and technical details need to be comprehensively considered. The specific implementation methods of this process will be introduced in detail below:

[0028] First of all, in order to achieve effective coverage of the farmland and data collection, it is necessary to carefully plan the flight path of the drone. This involves the application of the farmland Geographic Information System (GIS), obtaining the boundaries and internal structures of the farmland through satellite maps or high-resolution aerial images, including the locations of ridges, irrigation channels and other natural obstacles. Based on this information, technicians can use professional path planning software, such as Pix4D or AgriDronePlanner, to generate the optimal flight path according to the crop planting pattern, terrain characteristics and drone performance parameters. The path design not only considers the coverage density and efficiency, but also fully evaluates potential safety risks to ensure that the drone can operate stably in a complex agricultural environment. In addition, meteorological conditions such as wind direction and wind speed, as well as the maximum effective payload and battery life of the drone, need to be considered to determine the optimal flight height and speed.

[0029] Next is to set the task parameters of the drone. For different types of crops and their growth stages, the shooting parameters of the camera carried by the drone need to be finely adjusted. For example, for short crops such as wheat, a wider-angle lens can be selected to increase the coverage area of a single shot; while for fruit trees, a narrow-angle lens and a higher flight height are required to obtain detailed canopy structure information. In addition, the influence of lighting conditions needs to be considered, and the ISO sensitivity, shutter speed and aperture value are dynamically adjusted at different times to ensure the consistency and stability of image quality. At the same time, to adapt to diverse monitoring needs, multiple sensors, such as thermal imagers, Light Detection and Ranging (LiDAR), etc., can be integrated to obtain richer data dimensions. At the same time, an appropriate shooting interval also needs to be set, which should not be too frequent to generate redundant data, nor too sparse to miss key information.

[0030] After completing the preliminary preparations, the drone can be launched to perform the flight mission. Modern plant protection drones are usually equipped with a GPS navigation system and an autopilot, which can accurately move along the preset trajectory and automatically trigger the camera shutter when reaching each designated position. To further improve the operation accuracy, some advanced models also adopt RTK (Real-Time Kinematic), which reduces the positioning error to the centimeter level. During the flight, the ground station continuously receives the status reports from the drone, including important parameters such as position coordinates, remaining battery power, environmental temperature and humidity, etc., allowing the operator to monitor the progress at any time and respond promptly to possible problems.

[0031] It should be noted that due to the differences in light intensity at different times, especially during the period after sunrise and before sunset, the shadow effect will affect the image quality. Therefore, it is recommended to choose the daytime with sufficient sunlight for shooting, or use a drone with a supplementary light device to ensure that all photos can meet the consistent standards. In addition, for those areas located in hilly regions or with large terrain undulations, the flight altitude should be appropriately increased to prevent partial areas from being unable to image due to terrain occlusion.

[0032] Finally, when the drone returns to the base after completing a round of inspections, all the collected image data will be transmitted to the back-end pest and disease monitoring server for subsequent in-depth data analysis. In this way, valuable information can be extracted from a large number of seemingly chaotic photos, potential pest and disease signs can be identified, and scientific and reasonable prevention and control suggestions can be provided for farmers.

[0033] In the above-mentioned pest and disease monitoring system based on plant protection drones, the image import module 120 is used to import the crop images into the back-end pest and disease monitoring server. In the technical solution of this application, the back-end pest and disease monitoring server is deployed with an image processing model based on deep learning, which is used to perform multi-scale image feature analysis on the crop images to achieve intelligent identification of pests and diseases.

[0034] Specifically, after the drone collects the crop images, they need to be imported into the back-end pest and disease monitoring server. Considering that the drone usually operates in a vast farmland environment, the selection of the data transmission scheme is crucial. This process not only involves how the data collected by the drone is transmitted to the server, but also includes data preprocessing, storage, and preparation for subsequent analysis. The specific implementation methods of this process will be introduced in detail below:

[0035] First, considering that the working environment of drones is usually open farmland, directly uploading images in real time through a wireless network (such as 4G / 5G mobile network) is a feasible method. This approach allows the drone to synchronously send image data to a remote server during flight, reducing the time lag of data. However, this method also has some challenges, such as signal coverage issues, network bandwidth limitations, and cost considerations. Therefore, in practical applications, data transmission may be chosen to occur after the drone returns to the ground station. At this time, a faster local Wi-Fi connection or directly copying the image files from the memory card to a computer on the ground through a physical interface such as USB can be used, and then this computer is responsible for uploading to a cloud server or an on-premises pest monitoring server.

[0036] For larger amounts of data, a batch transmission strategy can also be adopted. That is, the drone can temporarily store images in its built-in high-speed storage device during flight, and then upload a part of the images to the server according to network conditions or a preset time interval. This not only ensures the speed of data transmission but also prevents network congestion caused by uploading too much data at once. In addition, to improve efficiency, a preliminary screening algorithm can be installed on the drone to only upload suspicious images that may show signs of pests and diseases, thereby reducing unnecessary data traffic.

[0037] For easy management and retrieval, the uploaded images can be tagged with timestamps and geographical location labels and saved according to a specific directory structure. For example, folders can be created based on factors such as date, plot number, and crop type, and each picture is named according to its corresponding drone shooting parameters. Such an organization method helps to quickly locate images in a specific area or time period.

[0038] Finally, considering that pest and disease monitoring is a long-term and dynamic process, the server architecture should have good scalability and fault tolerance. This means not only supporting the long-term storage of large-scale image data but also being able to handle sudden data access demands. For this purpose, the server can be designed as a distributed system, using cloud computing resources to dynamically allocate computing power and storage space. At the same time, a redundancy mechanism is set up to ensure that even if a certain node fails, it will not affect the normal operation of the entire system. Through the above measures, a stable and reliable data import platform can be built to provide strong technical support for efficient pest and disease monitoring.

[0039] In the above-mentioned pest and disease monitoring system based on a plant protection drone, the crop image multi-scale feature detection module 130 is used to perform multi-scale feature detection on the crop image on the backend pest and disease monitoring server to obtain a local feature encoding vector of the crop image and a global semantic encoding vector of the crop image. Among them, Figure 3The block diagram of the multi-scale feature detection module for crop images in the pest and disease monitoring system based on a plant protection UAV according to an embodiment of the present application. As Figure 3 shown, the multi-scale feature detection module 130 for crop images includes: an image enhancement unit 131, configured to perform grayscale conversion and noise reduction processing on the crop image to obtain an enhanced crop image; a local feature extraction unit 132, configured to use a local feature detection module to perform local feature detection on the enhanced crop image to obtain a local feature coding vector of the crop image; and a global feature extraction unit 133, configured to use a global feature detection module to perform global feature detection on the enhanced crop image to obtain a global semantic coding vector of the crop image.

[0040] Specifically, the image enhancement unit 131 is configured to perform grayscale conversion and noise reduction processing on the crop image to obtain an enhanced crop image. It should be understood that in the present application, considering that the actually collected crop images are usually affected by noise factors such as sensor noise and environmental light interference, which will have an adverse impact on the accuracy of pest and disease analysis. And for pest and disease monitoring, many lesion features are more obvious in terms of brightness change, and show lower sensitivity to color change. Therefore, in order to improve the robustness of pest and disease analysis, in the backend pest and disease monitoring server, first, the crop image is subjected to grayscale conversion processing to reduce the interference of color information and highlight the lesion features based on brightness, making subsequent feature detection easier and more accurate. In addition, since the original color image generally consists of three feature channels of red, green, and blue, while the grayscale image only contains a single channel. Therefore, in the present application, by converting the original crop image into a grayscale image, the data volume can be effectively reduced, the computational complexity can be reduced, and the processing efficiency can be improved. Then, through image noise reduction technology, the noise in the image is further eliminated, and the influence of image noise on the analysis result is reduced to enhance the image quality. In a specific embodiment of the present application, a noise reduction method based on Gaussian filtering is used to remove the random noise in the image to obtain an enhanced crop image.

[0041] Specifically, the local feature extraction unit 132 is configured to use a local feature detection module to perform local feature detection on the enhanced crop image to obtain a local feature coding vector of the crop image. It should be understood that in order to effectively capture the detailed information in the enhanced crop image, such as local lesion features such as disease spots and insect holes, in the present application, a local feature detection module is further used to perform local feature detection on the enhanced crop image to obtain a local feature coding vector of the crop image. In an embodiment of the present application, the local feature detection module uses the SIFT algorithm or the HOG algorithm as the local feature detection algorithm to process the enhanced crop image.

[0042] Specifically, the SIFT (Scale-Invariant Feature Transform) algorithm is a powerful technique for detecting and describing local features in the fields of image processing and computer vision. Its main features are invariance to scale and rotation, and it also has a certain adaptability to illumination changes. In pest and disease detection, SIFT can determine the location of diseases by capturing the unique visual features (such as abnormal colors, shape changes, etc.) of the disease areas. Since SIFT features have good robustness to changes in illumination, perspective, etc., it is very suitable for the analysis of crop images in the field environment.

[0043] The main steps of the SIFT algorithm include: Scale-space extreme value detection: Detect potential interest points in the enhanced crop image by constructing a Difference of Gaussian (DoG) pyramid, and these points appear as local maxima or minima at different scales. Key point localization: More precisely determine the position and scale of each candidate key point by fitting a three-dimensional quadratic function, and at the same time exclude key points with low contrast and edge responses. Orientation assignment: Calculate the orientation histogram of the area around the key point, and select the peak of the histogram as the main orientation of the key point, so that the feature descriptor has rotational invariance. Key point descriptor: Calculate the orientation gradient histogram within the neighborhood around each key point to form a fixed-length vector as the descriptor of the key point. Finally, splice and combine the descriptors of each key point to form a crop image local feature encoding vector describing the enhanced crop image.

[0044] Among them, the HOG (Histogram of Oriented Gradients) algorithm is a feature descriptor for object detection. Its basic idea is to calculate the histogram of oriented gradients within small windows of the image to describe the local shape information of the image. The main steps of the HOG algorithm include: Image segmentation: Divide the enhanced crop image into multiple small connected regions (called cell units). Gradient calculation: Calculate the gradient magnitude and direction of the pixels within each cell unit. Histogram statistics: Construct an oriented gradient histogram within each cell unit, usually using 9 orientation bins. Block normalization: Combine several adjacent cell units into a larger block and normalize the histogram within the block to reduce the influence of illumination changes. Feature vector generation: Connect the histograms of all blocks to form a feature vector to achieve a complete description of the local region features of the image. In pest and disease detection, HOG can be used to capture the texture information of the disease areas. For example, certain diseases will cause specific texture changes on the leaf surface, and these changes can be effectively described by HOG, thereby assisting in the identification of diseases. In practical applications, the appropriate local feature detection algorithm can be selected according to actual needs.

[0045] Specifically, the global feature extraction unit 133 is configured to use a global feature detection module to perform global feature detection on the enhanced crop image to obtain the global semantic encoding vector of the crop image. It should be understood that in this application, it is considered that the disease characteristics of crops may not only be reflected in local areas, but also involve the overall growth conditions of the plants, such as global features like the height of the plants, leaf density, and leaf distribution. Therefore, in addition to the local feature detection module, this application also introduces a global feature detection module to perform global feature detection on the enhanced crop image to obtain the global semantic encoding vector of the crop image. In the embodiment of this application, the global feature detection module uses a ViT model (Vision Transformer) to perform global feature detection on the enhanced crop image. Those of ordinary skill in the art should be aware that the ViT model is an image recognition technology based on the Transformer architecture. It divides the image into multiple small patches, and then takes these patches as a sequence and inputs them into the Transformer model. Through the self-attention mechanism, it can capture the long-range dependencies between different regions in the image, thereby effectively extracting the global semantic information of the enhanced crop image and capturing the overall growth conditions of the plants, such as the health status and growth cycle of the plants.

[0046] In the above-mentioned pest and disease monitoring system based on a plant protection UAV, the multi-scale feature interaction and fusion module 140 is configured to perform attention interaction and fusion based on implicit collaborative feature guidance on the global semantic encoding vector of the crop image and the local feature encoding vector of the crop image to obtain the multi-scale fusion encoding vector of the crop image features. It should be understood that since the local feature encoding vector of the crop image mainly reflects the specific lesion detail features of the crops, while the global semantic encoding vector of the crop image more reflects the overall growth conditions and health levels of the crops, in order to make full use of the complementary information of the two to achieve more accurate pest and disease detection, this application further performs a fusion process on the global semantic encoding vector of the crop image and the local feature encoding vector of the crop image. In particular, to optimize the feature fusion effect, this application proposes an attention interaction and fusion method to achieve deeper feature fusion by learning the implicit association between the local detail features and global semantic features of the crop image.

[0047] Figure 4 The block diagram of the multi-scale feature interaction and fusion module in the pest and disease monitoring system based on a plant protection UAV according to an embodiment of the present application. As Figure 4As shown in the figure, the multi-scale feature interaction and fusion module 140 includes: a collaborative feature extraction unit 141, which is used to input the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image into a collaborative feature extraction network to obtain a latent collaborative encoding vector between multi-scale features of the crop image; a feature modulation and optimization unit 142, which is used to perform feature attention modulation on the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image respectively based on the latent collaborative encoding vector between multi-scale features of the crop image to obtain an optimized local feature encoding vector of the crop image and an optimized global semantic encoding vector of the crop image; a multi-scale feature fusion unit 143, which is used to fuse the optimized local feature encoding vector of the crop image and the optimized global semantic encoding vector of the crop image to obtain a multi-scale fusion encoding vector of the crop image features.

[0048] Specifically, the collaborative feature extraction unit 141 is used to input the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image into a collaborative feature extraction network to obtain a latent collaborative encoding vector between multi-scale features of the crop image. In a specific example of the present application, the collaborative feature extraction network includes three parallel feature interaction layers, a feature concatenation layer, a point convolution layer, and an activation layer based on the Leaky ReLU function, which is expressed by the formula:

[0049]

[0050] Among them, represents the local feature encoding vector of the crop image, represents the global semantic encoding vector of the crop image, represents point addition by position, represents point multiplication by position, represents point subtraction by position, represents feature concatenation, represents a 1×1 convolution operation, represents the Leaky ReLU activation function, represents the latent collaborative encoding vector between multi-scale features of the crop image.

[0051] That is, the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image are input into the collaborative feature extraction network. Through operations such as multi-level interaction, point convolution, and non-linear activation, deep associations between features are captured, and the implicit collaborative feature representation between the two is mined. Specifically, the feature modulation optimization unit 142 is used to: write the implicit collaborative encoding vector between multi-scale features of the crop image into the dynamic memory unit to obtain a dynamic key vector; extract the dynamic key vector from the dynamic memory unit, and input the local feature encoding vector of the crop image and the dynamic key vector into the feature attention modulation module based on the first transformer structure to obtain the optimized local feature encoding vector of the crop image; extract the dynamic key vector from the dynamic memory unit, and input the global semantic encoding vector of the crop image and the dynamic key vector into the feature attention modulation module based on the second transformer structure to obtain the optimized global semantic encoding vector of the crop image, which is expressed by the formula:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059] Among them, represents a 1×1 convolution operation, represents the dynamic key vector, and represent the first query embedding matrix and the first value embedding matrix respectively, and represent the first query vector and the first value vector respectively, and represent the second query embedding matrix and the second value embedding matrix respectively, and represent the second query vector and the second value vector respectively, 、 、 and represent different bias terms respectively, represents the normalized exponential function, represents the feature scale value of the dynamic key vector, and respectively represent the local feature encoding vector of the optimized crop image and the global semantic encoding vector of the optimized crop image.

[0060] Here, the dynamic memory unit is a special type of memory structure, whose working principle is similar to the memory unit in a recurrent neural network. It can dynamically adjust its own memory content and internal state according to the current input features to better adapt to the context requirements of the current scenario. Then, a query vector and a value vector for the local feature encoding vector of the crop image, and a query vector and a value vector for the global semantic encoding vector of the crop image are constructed respectively. Using the self-attention mechanism in the transformer structure, based on the dynamic key vector, feature modulation based on query attention is performed on the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image. This is equivalent to using the dynamic key vector (i.e., the implicit association feature between the local details features and the global semantic features of the crop image) as a common attention space to enhance the expression of key information in the original features, reduce the influence of noise and irrelevant information, and at the same time make the optimized local feature encoding vector of the crop image and the global semantic encoding vector of the crop image in an aligned semantic space, so as to facilitate subsequent feature interaction and fusion processing between the two.

[0061] Specifically, the multi-scale feature fusion unit 143 is used to: input the optimized local feature encoding vector of the crop image and the optimized global semantic encoding vector of the crop image into the feature linear interaction network for weighted fusion to obtain the multi-scale fusion encoding vector of the crop image features, which is expressed by the formula:

[0062]

[0063] where is the fusion weight parameter, represents the multi-scale fusion encoding vector of the crop image features.

[0064] That is, the feature linear interaction network is used to fuse the optimized local feature encoding vector of the crop image and the optimized global semantic encoding vector of the crop image, that is, through a linear transformation method, weighted fusion between the local detail features and the global semantic features is performed to generate the multi-scale fusion encoding vector of the crop image features.

[0065] In the above-mentioned pest and disease monitoring system based on a plant protection UAV, the monitoring result generation module 150 is used to determine the monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image features. In a specific example of the present application, the monitoring result generation module 150 is used to: input the multi-scale fusion coding vector of the crop image features into the pest and disease monitoring module based on a classifier to obtain the monitoring result, and the monitoring result is used to indicate whether there are pests and diseases. It should be understood that after the above processing, the multi-scale fusion coding vector of the crop image features contains the deep fusion information of the texture features and structural features of the crop image, and can effectively reflect whether there are pests and diseases in the crops. After receiving the multi-scale fusion coding vector of the crop image features, the classifier first further processes it using the internal neural network structure to extract higher-level abstract features, and calculates the probability distribution through the softmax function of the output layer to obtain the probability value that the multi-scale fusion coding vector of the crop image features belongs to whether there are pests and diseases. Furthermore, the most likely monitoring result can be determined according to the magnitude of the probability value.

[0066] In a preferred example of the present application, inputting the multi-scale fusion coding vector of the crop image features into the pest and disease monitoring module based on a classifier to obtain the monitoring result includes:

[0067] First, calculate the sum of the absolute values of all the eigenvalues of the multi-scale fusion coding vector of the crop image features to obtain the first multi-scale fusion coding space structure value of the crop image features, and calculate the square root of the sum of its squares to obtain the second multi-scale fusion coding space structure value of the crop image features, which is expressed by the formula:

[0068]

[0069]

[0070] Wherein, represents the eigenvalue at the th position in the multi-scale fusion coding vector of the crop image features, represents the first multi-scale fusion coding space structure value corresponding to the , and represents the second multi-scale fusion coding space structure value corresponding to the ;

[0071] Secondly, for each eigenvalue of the multi-scale fusion coding vector of the crop image features, calculate the first multi-scale fusion coding space structure value of the crop image features minus the product of the eigenvalue and the total number of eigenvalues of the multi-scale fusion coding vector of the crop image features to obtain the first multi-scale fusion coding long-range dependence value of the crop image features, which is expressed by the formula:

[0072]

[0073] Among them, represents the long-range dependence value of the multi-scale fusion coding of the first crop image feature corresponding to the above-mentioned ; represents the total number of eigenvalues of the multi-scale fusion coding vector of the crop image feature;

[0074] Then, calculate the square root of the total number of eigenvalues multiplied by the product of the eigenvalues minus the multi-scale fusion coding spatial structure value of the second crop image feature to obtain the long-range dependence value of the multi-scale fusion coding of the second crop image feature, which is expressed by the formula:

[0075]

[0076] Among them, represents the long-range dependence value of the multi-scale fusion coding of the second crop image feature corresponding to the above-mentioned ;

[0077] Next, perform weighted summation on the exponential value obtained by taking the first crop image feature multi-scale fusion coding long-range dependence value as the exponent of the natural constant and the reciprocal of the second crop image feature multi-scale fusion coding long-range dependence value to obtain the optimized eigenvalue corresponding to each eigenvalue, which is expressed by the formula:

[0078]

[0079] Among them, represents the optimized eigenvalue corresponding to the above-mentioned ; , respectively represent different weight parameters, represents calculating the reciprocal, represents calculating the exponential value with the natural constant as the base;

[0080] Finally, pass the optimized crop image feature multi-scale fusion coding vector composed of the optimized eigenvalues through the pest and disease monitoring module based on the classifier to obtain the monitoring result.

[0081] Here, when the global semantic encoding vector of the crop image and the local feature encoding vector of the crop image respectively represent the local image features and the global semantic features of the enhanced crop image, during the inter-feature attention interaction, the inconsistent image feature scales between the global semantic encoding vector of the crop image and the local feature encoding vector of the crop image will cause attention interaction mismatch, resulting in differences in the spatial structure of the multi-scale fusion distribution of image semantic features in the multi-scale fusion encoding vector of the crop image features, affecting the convergence consistency of classification regression, and thus affecting the accuracy of the monitoring results obtained by the pest and disease monitoring module based on the classifier.

[0082] Based on this, aiming at the inconsistent convergence of the implicit inference of the spatial structure information based on features in image semantic classification regression due to the possible lack of spatial structure in the feature set of the multi-scale fusion encoding vector of the crop image features in the high-dimensional space, by establishing a long-distance feature dependence relationship based on the overall feature scale of the multi-scale fusion encoding vector of the crop image features with respect to the spatial structure representation of the multi-scale fusion encoding vector of the crop image features, to establish the local connectivity of the features of the multi-scale fusion encoding vector of the crop image features, and by predicting the spatial ambiguity information of the object feature values through the unstructured feature value points of the multi-scale fusion encoding vector of the crop image features, thereby enhancing the spatial inductive bias perception ability of the feature set of the multi-scale fusion encoding vector of the crop image features, improving the convergence consistency of classification regression, and enhancing the accuracy of the monitoring results obtained by the multi-scale fusion encoding vector of the crop image features through the pest and disease monitoring module based on the classifier.

[0083] After determining the monitoring results of pests and diseases, taking effective measures is crucial for ensuring the healthy growth of crops and increasing yields. Based on the monitoring results, personalized prevention and control strategies can be formulated, taking into account environmental protection and the safety of agricultural products, minimizing the use of chemical pesticides as much as possible, and promoting green prevention and control technologies.

[0084] According to the specific situation of pests and diseases, one or a combination of physical control, biological control, or chemical control can be selected. Physical control does not use chemical agents but controls pests and diseases by changing environmental conditions, such as using insect-catching lights to lure and kill pests; setting up insect-proof nets to prevent pests from entering the farmland; or using high-temperature treatment of seeds and other methods to prevent the occurrence of diseases. For some specific types of pests and diseases, physical control can be used as a quick and effective emergency measure. Biological control is a method of using natural enemies existing in nature (such as parasitic wasps, insect-eating birds) or other beneficial microorganisms (such as bacteria, fungi) to control the number of pests. This method is not only environmentally friendly but also helps to maintain ecological balance. If the pests and diseases have reached the level of emergency intervention, chemical control may be a necessary option. At this time, highly effective and low-toxic pesticides should be selected and used strictly according to the recommended dosage. Spraying pesticides with drones can not only improve the operation efficiency but also ensure the uniformity of pesticide application and reduce environmental pollution. In addition, agricultural management measures, such as adjusting the planting structure, crop rotation, and fallow, can be combined to enhance the resistance of crops themselves.

[0085] After the implementation of the control measures, it is necessary to continue to use the plant protection drone to regularly collect farmland images, monitor the changes in pests and diseases, compare and analyze the data of the two times before and after, evaluate the control effect, and adjust the subsequent management plan accordingly. This is not only a process of verifying the effectiveness of the control measures but also an important link in continuously optimizing and improving the entire pest and disease management system. Collect and organize the experience and lessons accumulated at each stage to form a complete set of solution systems, and continuously explore new technologies and methods to meet the increasingly complex requirements of agricultural production and environmental protection.

[0086] In summary, the pest and disease monitoring system based on a plant protection drone according to the embodiments of the present application is elucidated. It uses the plant protection drone to collect crop images, and introduces image processing technology based on deep learning in the back-end server. By performing multi-scale analysis of local features and global features on the collected crop images, it can simultaneously capture local lesions and overall growth conditions of the crops. Furthermore, through interactive fusion analysis of the multi-scale state features of the crop images, it can achieve an accurate understanding and intelligent recognition of the pest and disease states of the crops. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0087] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are only examples and not limitations. It cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, rather than limitations. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0088] In the above embodiments, the descriptions of the various embodiments each have their own emphasis. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only illustrative. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

[0090] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0091] Finally, it should be noted that the above description has been given for purposes of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A pest and disease monitoring system based on plant protection drones, characterized in that: include: A crop image acquisition module, used to control a plant protection drone equipped with a camera to fly along a predetermined route and to use the camera to acquire crop images; An image import module, used to import the crop image into a back-end pest and disease monitoring server; A crop image multi-scale feature detection module is used to perform multi-scale feature detection on the crop image in the back-end pest monitoring server to obtain a crop image local feature coding vector and a crop image global semantic coding vector; A multi-scale feature interactive fusion module, used for performing attention interactive fusion based on implicit collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of crop image features; A monitoring result generation module, used to determine the monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image features; The multi-scale feature interactive fusion module includes: A collaborative feature extraction unit, used for inputting the crop image local feature coding vector and the crop image global semantic coding vector into a collaborative feature extraction network to obtain an implicit collaborative coding vector between multi-scale features of the crop image; A feature modulation optimization unit is used to perform feature attention modulation on the local feature coding vector of the crop image and the global semantic coding vector of the crop image based on the implicit collaborative coding vector between the multi-scale features of the crop image to obtain an optimized local feature coding vector of the crop image and an optimized global semantic coding vector of the crop image; A multi-scale feature fusion unit, used for fusing the optimized crop image local feature coding vector and the optimized crop image global semantic coding vector to obtain the crop image feature multi-scale fusion coding vector; The collaborative feature extraction network includes three parallel feature interaction layers, a feature cascade layer, a point convolution layer and an activation layer based on the Leaky ReLU function.

2. The pest monitoring system based on plant protection drone according to claim 1 is characterized in that: The crop image multi-scale feature detection module comprises: An image enhancement unit, used for performing grayscale and noise reduction processing on the crop image to obtain an enhanced crop image; A local feature extraction unit, configured to perform local feature detection on the enhanced crop image using a local feature detection module to obtain a local feature coding vector of the crop image; The global feature extraction unit is used to perform global feature detection on the enhanced crop image using a global feature detection module to obtain a global semantic coding vector of the crop image.

3. The pest monitoring system based on plant protection drone according to claim 2 is characterized in that: The local feature detection module uses the SIFT algorithm or the HOG algorithm as the local feature detection algorithm.

4. The pest monitoring system based on plant protection drone according to claim 3 is characterized in that: The global feature detection module performs global feature detection on the enhanced crop image using the ViT model.

5. The pest monitoring system based on plant protection drone according to claim 4 is characterized in that: The feature modulation optimization unit is used for: Writing the implicit collaborative coding vector between the multi-scale features of the crop image into a dynamic memory unit to obtain a dynamic key vector; Extracting the dynamic key vector from the dynamic memory unit, and inputting the crop image local feature encoding vector and the dynamic key vector into a feature attention modulation module based on a first converter structure to obtain the optimized crop image local feature encoding vector; The dynamic key vector is extracted from the dynamic memory unit, and the crop image global semantic encoding vector and the dynamic key vector are input into a feature attention modulation module based on a second converter structure to obtain the optimized crop image global semantic encoding vector.

6. The pest monitoring system based on plant protection drone according to claim 5, characterized in that: The multi-scale feature fusion unit is used to: The optimized crop image local feature coding vector and the optimized crop image global semantic coding vector are input into a feature linear interaction network for weighted fusion to obtain the crop image feature multi-scale fusion coding vector.

7. The pest monitoring system based on plant protection drone according to claim 6, characterized in that: The monitoring result generating module is used for: The crop image feature multi-scale fusion coding vector is input into a classifier-based pest and disease monitoring module to obtain the monitoring result, and the monitoring result is used to indicate whether there is a pest and disease.

Images