Pest and disease monitoring system based on plant protection unmanned aerial vehicle

Abstract

The invention relates to the field of disease and pest monitoring, and particularly discloses a disease and pest monitoring system based on a plant protection unmanned aerial vehicle, which utilizes the plant protection unmanned aerial vehicle to collect crop images and introduces an image processing technology based on deep learning into a rear-end server. Multi-scale analysis of local features and global features is carried out on collected crop images to capture local lesions and overall growth conditions of crops at the same time, and further, interactive fusion analysis is carried out on multi-scale state features of the crop images to realize accurate understanding and intelligent identification of crop disease and insect pest states. In this way, the accuracy and efficiency of disease and pest monitoring can be effectively improved, and therefore powerful technical support is provided for agricultural production.

Description

Technical Field

[0001] The present application relates to the field of pest and disease monitoring, and more specifically, to a pest and disease monitoring system based on a plant protection drone. Background Art

[0002] In agricultural production, pests and diseases are one of the key factors affecting the healthy growth and yield of crops. With the advancement of agricultural modernization, pest and disease control in agricultural production has become increasingly important. Traditional pest and disease monitoring methods mainly rely on manual inspections and sampling analysis, which is not only inefficient and consumes a lot of manpower and material resources, but also difficult to achieve real-time monitoring of large areas of farmland. In addition, due to the different growth cycles of crops and the randomness of the time and place of pest and disease occurrence, traditional methods often cannot detect the occurrence of pests and diseases in a timely and accurate manner, resulting in missing the best prevention and control period and causing a decrease in crop yield and quality.

[0003] In recent years, the development of unmanned aerial vehicle (UAV) technology has provided new solutions for agricultural monitoring. Plant protection UAVs have been widely used in the agricultural field due to their advantages such as flexibility, wide operating range and relatively low cost. In particular, plant protection UAVs equipped with high-definition cameras and other sensors can quickly obtain high-resolution images of large areas of farmland, providing a rich source of data for monitoring crop health.

[0004] However, the manifestations of crop diseases and pests in images are complex and diverse, with both local lesions (such as spots and wormholes on leaves) and overall growth conditions (such as plant height and color changes). Traditional image analysis methods are usually based on manually designed feature extraction algorithms, such as color histograms, texture features (such as grayscale co-occurrence matrices), shape features, etc., to describe target objects or abnormal conditions in images. These methods work well when dealing with simple and regular targets, but when faced with the complex task of crop disease and pest monitoring, it is usually difficult to fully capture the complex manifestations of diseases and pests in images. This results in low monitoring accuracy.

[0005] Therefore, an optimized pest and disease monitoring system based on plant protection drones is expected. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a pest and disease monitoring system based on a plant protection drone, which uses a plant protection drone to collect crop images, and introduces deep learning-based image processing technology in the back-end server, and performs multi-scale analysis of local and global features on the collected crop images to simultaneously capture local lesions and overall growth conditions of crops, and then, through interactive fusion analysis of the multi-scale state features of the crop images, accurate understanding and intelligent identification of the state of crop diseases and pests can be achieved. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0007] Accordingly, according to one aspect of the present application, a pest monitoring system based on a plant protection drone is provided, comprising: A crop image acquisition module, used to control a plant protection UAV equipped with a camera to fly along a predetermined route and to use the camera to acquire crop images; An image import module, used to import the crop image into a back-end pest and disease monitoring server; A crop image multi-scale feature detection module is used to perform multi-scale feature detection on the crop image in the back-end pest monitoring server to obtain a crop image local feature coding vector and a crop image global semantic coding vector; A multi-scale feature interactive fusion module, used for performing attention interactive fusion based on implicit collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of crop image features; The monitoring result generation module is used to determine the monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image features.

[0008] Compared with the prior art, the pest and disease monitoring system based on plant protection drones provided by this application uses plant protection drones to collect crop images, and introduces deep learning-based image processing technology in the back-end server. It performs multi-scale analysis of local and global features on the collected crop images to simultaneously capture local lesions and overall growth conditions of crops, and then interactively integrates and analyzes the multi-scale state features of crop images to achieve accurate understanding and intelligent identification of the state of crop pests and diseases. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 The block diagram is a pest monitoring system based on a plant protection drone according to an embodiment of the present application.

[0011] Figure 2 The figure is a data flow diagram of a pest and disease monitoring system based on a plant protection drone according to an embodiment of the present application.

[0012] Figure 3 The present invention is a block diagram of a multi-scale feature detection module for crop images in a pest and disease monitoring system based on a plant protection drone according to an embodiment of the present application.

[0013] Figure 4 The block diagram is a multi-scale feature interactive fusion module in the pest and disease monitoring system based on the plant protection drone according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0015] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0016] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0017] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0018] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization given by the owner of the corresponding device.

[0019] In response to the technical problems described in the above background technology, this application proposes an optimized pest and disease monitoring system based on plant protection drones, which uses plant protection drones to collect crop images and introduces deep learning-based image processing technology in the back-end server. It performs multi-scale analysis of local and global features of the collected crop images to simultaneously capture local lesions and overall growth conditions of crops, and then interactively integrates and analyzes the multi-scale state features of crop images to achieve accurate understanding and intelligent identification of the state of crop pests and diseases. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0020] Figure 1 The block diagram is a pest monitoring system based on a plant protection drone according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a pest monitoring system based on a plant protection drone according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the pest and disease monitoring system 100 based on the plant protection UAV includes: a crop image acquisition module 110, which is used to control the plant protection UAV equipped with a camera to fly according to a predetermined route and use the camera to collect crop images; an image import module 120, which is used to import the crop image into a back-end pest and disease monitoring server; a crop image multi-scale feature detection module 130, which is used to perform multi-scale feature detection on the crop image in the back-end pest and disease monitoring server to obtain a crop image local feature coding vector and a crop image global semantic coding vector; a multi-scale feature interactive fusion module 140, which is used to perform attention interactive fusion based on implicit collaborative feature guidance on the crop image global semantic coding vector and the crop image local feature coding vector to obtain a crop image feature multi-scale fusion coding vector; a monitoring result generation module 150, which is used to determine the pest and disease monitoring result based on the crop image feature multi-scale fusion coding vector.

[0021] In the above-mentioned pest monitoring system based on plant protection drones, the crop image acquisition module 110 is used to control the plant protection drone equipped with a camera to fly along a predetermined route and use the camera to acquire crop images. It should be understood that by acquiring crop images through drones, a larger range of farmland can be covered without being restricted by terrain, which greatly improves the efficiency and quality of data acquisition.

[0022] In modern agricultural production, in order to ensure that drones can complete their scheduled tasks efficiently and accurately, that is, fly along the scheduled route and use cameras to collect crop images, it is necessary to comprehensively consider multiple factors and technical details. The specific implementation methods of this process will be described in detail below: First of all, in order to achieve effective coverage and data collection of farmland, the flight path of the drone needs to be carefully planned. This involves the application of the farmland geographic information system (GIS), which uses satellite maps or high-resolution aerial images to obtain the boundaries and internal structures of the farmland, including the location of ridges, irrigation channels and other natural obstacles. Based on this information, technicians can use professional path planning software such as Pix4D or AgriDronePlanner to generate the optimal flight path based on crop planting patterns, terrain characteristics and drone performance parameters. The path design not only takes into account coverage density and efficiency, but also fully evaluates potential safety risks to ensure that the drone can operate stably in a complex agricultural environment. In addition, meteorological conditions such as wind direction and wind speed, as well as the maximum payload and battery life of the drone, must also be considered to determine the optimal flight altitude and speed.

[0023] The next step is to set the mission parameters of the drone. For different types of crops and their growth stages, the shooting parameters of the camera carried by the drone need to be finely adjusted. For example, for dwarf crops such as wheat, a lens with a wider viewing angle can be used to increase the coverage area of ​​a single shot; for fruit trees, a narrow-angle lens is required with a higher flight altitude to obtain detailed canopy structure information. In addition, the influence of lighting conditions must be considered, and the ISO sensitivity, shutter speed and aperture value must be dynamically adjusted in different time periods to ensure the consistency and stability of image quality. At the same time, in order to meet diverse monitoring needs, a variety of sensors can be integrated, such as thermal imagers, laser radars (LiDAR), etc., to obtain richer data dimensions. At the same time, it is also necessary to set a suitable shooting interval, which should not be too frequent to generate redundant data, nor too sparse to miss key information.

[0024] After completing the preliminary preparations, you can start the drone to perform the flight mission. Modern agricultural drones are usually equipped with GPS navigation systems and autopilots, which can accurately move along the preset trajectory and automatically trigger the camera shutter when reaching each designated location. In order to further improve the accuracy of the operation, some advanced models also use RTK (real-time dynamic differential positioning technology) to reduce the positioning error to the centimeter level. During the flight, the ground station will continue to receive status reports from the drone, including important parameters such as location coordinates, remaining power, ambient temperature and humidity, allowing the operator to monitor progress at any time and respond promptly to possible problems.

[0025] It is worth noting that due to the difference in light intensity at different times, especially after sunrise and before sunset, shadow effects will affect image quality. Therefore, it is recommended to shoot during the day when there is plenty of sunlight, or use a drone with a fill light device to ensure that all photos can meet the same standard. In addition, for those located in hilly areas or areas with large terrain undulations, the flight altitude should be appropriately increased to prevent some areas from being unable to be imaged due to terrain obstruction.

[0026] Finally, when the drone completes a round of inspection and returns to the base, all the collected image data will be transmitted to the back-end pest and disease monitoring server for subsequent in-depth data analysis. In this way, valuable information can be extracted from a large number of seemingly chaotic photos, potential signs of pests and diseases can be identified, and scientific and reasonable prevention and control suggestions can be provided to farmers.

[0027] In the above-mentioned pest and disease monitoring system based on plant protection drones, the image import module 120 is used to import the crop image into the back-end pest and disease monitoring server. In the technical solution of the present application, the back-end pest and disease monitoring server is deployed with an image processing model based on deep learning, which is used to perform multi-scale image feature analysis on crop images to realize intelligent identification of pests and diseases.

[0028] Specifically, after the drone collects crop images, they need to be imported into the back-end pest and disease monitoring server. Considering that drones usually work in vast farmland environments, the choice of data transmission scheme is crucial. This process not only involves how to transmit the data collected by the drone to the server, but also includes data preprocessing, storage, and preparation for subsequent analysis. The specific implementation method of this process will be described in detail below: First of all, considering that the working environment of drones is usually open farmland, uploading images directly in real time through wireless networks (such as 4G / 5G mobile networks) is a feasible method. This method allows the drone to send image data to the remote server synchronously during flight, reducing the data lag time. However, this method also has some challenges, such as signal coverage issues, network bandwidth limitations, and cost considerations. Therefore, in practical applications, it may be chosen to transmit data after the drone returns to the ground station. At this time, you can use a faster local Wi-Fi connection or directly copy the image file in the memory card to the computer on the ground through a physical interface such as USB, and then the computer is responsible for uploading it to the cloud server or the internally deployed pest and disease monitoring server.

[0029] For situations with large amounts of data, a batch transmission strategy can also be adopted. That is, the drone can temporarily store images on its built-in high-speed storage device during flight, and then upload a portion of the images to the server first according to network conditions or preset time intervals. This ensures the speed of data transmission without causing network congestion due to uploading too much data at one time. In addition, in order to improve efficiency, a preliminary screening algorithm can be installed on the drone to only upload those suspicious images that may have signs of pests and diseases, thereby reducing unnecessary data traffic.

[0030] For easy management and retrieval, uploaded images can be timestamped and geo-tagged, and saved in a specific directory structure. For example, folders can be created based on date, plot number, crop type, etc., and each image can be named according to its corresponding drone shooting parameters. This organization method helps to quickly locate images in a specific area or time period.

[0031] Finally, considering that pest and disease monitoring is a long-term and dynamic process, the server architecture should have good scalability and fault tolerance. This means not only supporting the long-term storage of large-scale image data, but also being able to cope with sudden data access needs. To this end, the server can be designed as a distributed system, using cloud computing resources to dynamically allocate computing power and storage space. At the same time, a redundancy mechanism is set up to ensure that even if a node fails, it will not affect the normal operation of the entire system. Through the above measures, a stable and reliable data import platform can be built to provide solid technical support for efficient pest and disease monitoring.

[0032] In the above-mentioned pest and disease monitoring system based on plant protection drones, the crop image multi-scale feature detection module 130 is used to perform multi-scale feature detection on the crop image in the back-end pest and disease monitoring server to obtain a crop image local feature coding vector and a crop image global semantic coding vector. Figure 3FIG. 4 is a block diagram of a multi-scale feature detection module for crop images in a pest and disease monitoring system based on a plant protection drone according to an embodiment of the present application. Figure 3 As shown, the crop image multi-scale feature detection module 130 includes: an image enhancement unit 131, used to perform grayscale and noise reduction processing on the crop image to obtain an enhanced crop image; a local feature extraction unit 132, used to perform local feature detection on the enhanced crop image using a local feature detection module to obtain a local feature coding vector of the crop image; and a global feature extraction unit 133, used to perform global feature detection on the enhanced crop image using a global feature detection module to obtain a global semantic coding vector of the crop image.

[0033] Specifically, the image enhancement unit 131 is used to grayscale and reduce noise on the crop image to obtain an enhanced crop image. It should be understood that the present application takes into account that the crop images actually collected are usually affected by noise factors such as sensor noise and ambient light interference, which in turn has an adverse effect on the accuracy of pest analysis. In addition, for pest monitoring, many lesion features are more obvious in brightness changes, while in contrast, they show lower sensitivity to color changes. Therefore, in order to improve the robustness of pest analysis, in the back-end pest monitoring server, first, the crop image is grayscaled to reduce the interference of color information, highlight the lesion features based on brightness, and make subsequent feature detection easier and more accurate. In addition, since the original color image is generally composed of three feature channels of red, green, and blue, while the grayscale image contains only a single channel. Therefore, by converting the original crop image into a grayscale image, the present application can effectively reduce the amount of data, reduce the computational complexity, and thus improve the processing efficiency. Then, the image noise reduction technology is used to further eliminate the noise in the image, reduce the impact of image noise on the analysis results, and enhance the image quality. In a specific embodiment of the present application, a noise reduction method based on Gaussian filtering is used to remove random noise in an image to obtain an enhanced crop image.

[0034] Specifically, the local feature extraction unit 132 is used to use a local feature detection module to perform local feature detection on the enhanced crop image to obtain a local feature coding vector of the crop image. It should be understood that in order to effectively capture the detailed information in the enhanced crop image, such as local pathological features such as disease spots and wormholes, the present application further uses a local feature detection module to perform local feature detection on the enhanced crop image to obtain a local feature coding vector of the crop image. In an embodiment of the present application, the local feature detection module uses a SIFT algorithm or a HOG algorithm as a local feature detection algorithm to process the enhanced crop image.

[0035] Specifically, the SIFT (Scale Invariant Feature Transform) algorithm is a powerful technology for detecting and describing local features in the field of image processing and computer vision. Its main features are invariance to scale and rotation, and a certain adaptability to changes in illumination. In pest and disease detection, SIFT can determine the location of the disease by capturing the unique visual features of the diseased area (such as color anomalies, shape changes, etc.). Since SIFT features are robust to changes in illumination, viewing angle, etc., they are very suitable for analyzing crop images in field environments.

[0036] The main steps of the SIFT algorithm include: scale space extreme value detection: by constructing a Gaussian difference pyramid to detect potential points of interest in the enhanced crop image, these points appear as local maxima or minima at different scales. Key point positioning: by fitting a three-dimensional quadratic function to more accurately determine the position and scale of each candidate key point, while excluding low-contrast key points and edge responses. Direction assignment: calculate the direction histogram of the area around the key point, and select the histogram peak as the main direction of the key point, so that the feature descriptor is rotationally invariant. Key point descriptor: calculate the directional gradient histogram in the neighborhood around each key point to form a vector of fixed length as the descriptor of the key point. Finally, the descriptors of each key point are spliced ​​and combined to form a crop image local feature encoding vector that describes the enhanced crop image.

[0037] Among them, the HOG (Histogram of Oriented Gradients) algorithm is a feature descriptor for object detection. Its basic idea is to calculate the directional gradient histogram in a small window of the image to describe the local shape information of the image. The main steps of the HOG algorithm include: Image segmentation: Divide the enhanced crop image into multiple small connected areas (called cell units). Gradient calculation: Calculate the gradient amplitude and direction of the pixel in each cell unit. Histogram statistics: Construct a directional gradient histogram in each cell unit, usually using 9 direction intervals. Block normalization: Combine several adjacent cell units into a larger block and normalize the histogram in the block to reduce the impact of illumination changes. Feature vector generation: Connect the histograms of all blocks to form a feature vector to achieve a complete description of the features of the local area of ​​the image. In pest and disease detection, HOG can be used to capture the texture information of the diseased area. For example, some diseases can cause specific texture changes on the surface of leaves. HOG can effectively describe these changes and assist in the identification of diseases. In practical applications, the appropriate local feature detection algorithm can be selected according to actual needs.

[0038] Specifically, the global feature extraction unit 133 is used to use a global feature detection module to perform global feature detection on the enhanced crop image to obtain a global semantic coding vector for the crop image. It should be understood that the present application takes into account that the disease characteristics of crops may not only be reflected in local areas, but also involve the overall growth status of the plant, such as global features such as plant height, leaf density, and leaf distribution. Therefore, in addition to the local feature detection module, the present application also introduces a global feature detection module to perform global feature detection on the enhanced crop image to obtain a global semantic coding vector for the crop image. In an embodiment of the present application, the global feature detection module uses the ViT model (Vision Transformer) to perform global feature detection on the enhanced crop image. Those skilled in the art should know that the ViT model is an image recognition technology based on the Transformer architecture. It divides the image into multiple small patches (patches) and then inputs these small patches into the Transformer model as a sequence. It uses the self-attention mechanism to capture the long-distance dependencies between different regions in the image, thereby effectively extracting the global semantic information of the enhanced crop image and capturing the overall growth status of the plant, such as the health of the plant, the growth cycle, etc.

[0039] In the above-mentioned pest and disease monitoring system based on plant protection drones, the multi-scale feature interactive fusion module 140 is used to perform attention interactive fusion based on implicit collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of crop image features. It should be understood that since the local feature coding vector of the crop image mainly reflects the specific pathological detail features of the crops, and the global semantic coding vector of the crop image more reflects the overall growth status and health level of the crops, in order to make full use of the complementary information of the two to achieve more accurate pest and disease detection, the present application further fuses the global semantic coding vector of the crop image and the local feature coding vector of the crop image. In particular, in order to optimize the feature fusion effect, the present application proposes an attention interactive fusion method, which achieves deeper feature fusion by learning the implicit association between the local detail features of the crop image and the global semantic features.

[0040] Figure 4 FIG. 1 is a block diagram of a multi-scale feature interactive fusion module in a pest monitoring system based on a plant protection drone according to an embodiment of the present application. Figure 4As shown, the multi-scale feature interactive fusion module 140 includes: a collaborative feature extraction unit 141, which is used to input the crop image local feature coding vector and the crop image global semantic coding vector into a collaborative feature extraction network to obtain an implicit collaborative coding vector between multi-scale features of the crop image; a feature modulation optimization unit 142, which is used to perform feature attention modulation on the crop image local feature coding vector and the crop image global semantic coding vector respectively based on the implicit collaborative coding vector between multi-scale features of the crop image to obtain an optimized crop image local feature coding vector and an optimized crop image global semantic coding vector; a multi-scale feature fusion unit 143, which is used to fuse the optimized crop image local feature coding vector and the optimized crop image global semantic coding vector to obtain the crop image feature multi-scale fusion coding vector.

[0041] Specifically, the collaborative feature extraction unit 141 is used to input the crop image local feature coding vector and the crop image global semantic coding vector into a collaborative feature extraction network to obtain an implicit collaborative coding vector between multi-scale features of the crop image. In a specific example of the present application, the collaborative feature extraction network includes three parallel feature interaction layers, a feature cascade layer, a point convolution layer, and an activation layer based on a Leaky ReLU function, which is expressed by the formula:

[0042] in, represents the local feature encoding vector of the crop image, represents the global semantic encoding vector of the crop image, Indicates adding by position point. It means point multiplication by position. It means to subtract by position point. represents feature cascade, represents a 1×1 convolution operation, represents the Leaky ReLU activation function, Represents the implicit co-coding vector between multi-scale features of crop images.

[0043] That is, the crop image local feature coding vector and the crop image global semantic coding vector are input into the collaborative feature extraction network, and through multi-level interaction, point convolution and nonlinear activation and other operations, the deep-level association between the features is captured, and the implicit collaborative feature representation between the two is mined. Specifically, the feature modulation optimization unit 142 is used to: write the implicit collaborative coding vector between the multi-scale features of the crop image into the dynamic memory unit to obtain a dynamic key vector; extract the dynamic key vector from the dynamic memory unit, and input the crop image local feature coding vector and the dynamic key vector into the feature attention modulation module based on the first converter structure to obtain the optimized crop image local feature coding vector; extract the dynamic key vector from the dynamic memory unit, and input the crop image global semantic coding vector and the dynamic key vector into the feature attention modulation module based on the second converter structure to obtain the optimized crop image global semantic coding vector, which is expressed by the formula:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] in, represents a 1×1 convolution operation, represents the dynamic key vector, and denote the first query embedding matrix and the first value embedding matrix respectively, and denote the first query vector and the first value vector respectively, and denote the second query embedding matrix and the second value embedding matrix respectively, and denote the second query vector and the second value vector respectively, , , and Represent different bias terms, represents the normalized exponential function, represents the characteristic scale value of the dynamic key vector, and They respectively represent the optimized local feature encoding vector of the crop image and the optimized global semantic encoding vector of the crop image.

[0051] Here, the dynamic memory unit is a special type of memory structure, and its working principle is similar to that of the memory unit in the recursive neural network, which can dynamically adjust its own memory content and internal state according to the current input features to better adapt to the context requirements of the current scene. Then, the query vector and value vector for the local feature encoding vector of the crop image, and the query vector and value vector for the global semantic encoding vector of the crop image are constructed respectively, and the self-attention mechanism in the converter structure is used to perform feature modulation based on the query attention on the local feature encoding vector of the crop image and the global semantic encoding vector of the crop image based on the dynamic key vector, which is equivalent to using the dynamic key vector (i.e., the implicit association feature between the local detail feature of the crop image and the global semantic feature) as a common attention space to enhance the expression of key information in the original feature, reduce the influence of noise and irrelevant information, and also make the optimized local feature encoding vector of the crop image and the global semantic encoding vector of the crop image in an aligned semantic space, so as to facilitate the subsequent feature interaction and fusion processing between the two.

[0052] Specifically, the multi-scale feature fusion unit 143 is used to: input the optimized crop image local feature coding vector and the optimized crop image global semantic coding vector into the feature linear interaction network for weighted fusion to obtain the crop image feature multi-scale fusion coding vector, which is expressed as:

[0053] in, is the fusion weight parameter, represents the multi-scale fusion coding vector of the crop image features.

[0054] That is, the optimized crop image local feature coding vector and the optimized crop image global semantic coding vector are fused using a feature linear interaction network, that is, weighted fusion of local detail features and global semantic features is performed through linear transformation to generate a multi-scale fused coding vector of crop image features.

[0055] In the above-mentioned pest and disease monitoring system based on plant protection drone, the monitoring result generation module 150 is used to determine the monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image feature. In a specific example of the present application, the monitoring result generation module 150 is used to: input the multi-scale fusion coding vector of the crop image feature into the pest and disease monitoring module based on the classifier to obtain the monitoring result, and the monitoring result is used to indicate whether there are pests and diseases. It should be understood that after the above processing, the multi-scale fusion coding vector of the crop image feature contains the deep fusion information of the texture features and structural features of the crop image, which can effectively reflect whether there are pests and diseases in the crops. After the classifier receives the multi-scale fusion coding vector of the crop image feature, it first uses the internal neural network structure to further process it to extract higher-level abstract features, and performs probability distribution calculation through the softmax function of the output layer to obtain the probability value of whether the multi-scale fusion coding vector of the crop image feature belongs to pests and diseases. Then, the most likely monitoring result can be determined according to the size of the probability value.

[0056] In a preferred example of the present application, the multi-scale fusion coding vector of the crop image features is passed through a classifier-based pest monitoring module to obtain a monitoring result, which includes: First, the absolute value sum of all eigenvalues ​​of the crop image feature multi-scale fusion coding vector is calculated to obtain the first crop image feature multi-scale fusion coding spatial structure value, and the square root of the square sum is calculated to obtain the second crop image feature multi-scale fusion coding spatial structure value, which is expressed as follows:

[0057]

[0058] in, Represents the first The eigenvalues ​​at the positions, Indicates the The corresponding first crop image feature multi-scale fusion encoding spatial structure value, Indicates the The corresponding second crop image feature multi-scale fusion encoding spatial structure value; Secondly, for each eigenvalue of the crop image feature multi-scale fusion coding vector, the product of the eigenvalue and the total number of eigenvalues ​​of the crop image feature multi-scale fusion coding vector is calculated to obtain the first crop image feature multi-scale fusion coding long-range dependency value, which is expressed as follows:

[0059] in, Indicates the The corresponding first crop image feature multi-scale fusion encoding long-range dependency value, Represents the total number of eigenvalues ​​of the multi-scale fusion coding vector of crop image features; Then, the product of the square root of the total number of eigenvalues ​​multiplied by the eigenvalues ​​minus the second crop image feature multi-scale fusion coding spatial structure value is calculated to obtain the second crop image feature multi-scale fusion coding long-range dependency value, which is expressed as:

[0060] in, Indicates the The corresponding second crop image feature multi-scale fusion encoding long-range dependency value; Next, the exponential value calculated by taking the first crop image feature multi-scale fusion coding long-range dependency value as the exponent of the natural constant and the inverse of the second crop image feature multi-scale fusion coding long-range dependency value are weightedly summed to obtain the optimized eigenvalue corresponding to each eigenvalue, which is expressed as follows:

[0061] in, Indicates the The corresponding optimized eigenvalue is , Represent different weight parameters respectively. Indicates the calculation of the inverse. Indicates the calculation of the exponential value with the natural constant as the base; Finally, the optimized crop image feature multi-scale fusion coding vector composed of the optimized feature values ​​is passed through a classifier-based pest monitoring module to obtain a monitoring result.

[0062] Here, in the case where the crop image global semantic coding vector and the crop image local feature coding vector respectively represent the image local features and the image global semantic features of the enhanced crop image, when performing attention interaction between the features, the inconsistency in the image feature scales between the crop image global semantic coding vector and the crop image local feature coding vector will cause attention interaction mismatch, resulting in the crop image feature multi-scale fusion coding vector having a difference in the spatial structure of the multi-scale fusion distribution of the image semantic features, affecting the classification and regression convergence consistency, thereby affecting the accuracy of the monitoring results obtained by the pest and disease monitoring module based on the classifier.

[0063] Based on this, in view of the possible spatial structure loss of the feature set of the crop image feature multiscale fusion coding vector in the high-dimensional space, which leads to the convergence inconsistency of the image semantic classification regression based on the feature-based implicit inference of spatial structure information, a long-distance feature dependency relationship is established based on the overall feature scale of the crop image feature multiscale fusion coding vector relative to the spatial structure representation of the crop image feature multiscale fusion coding vector, so as to establish the feature local connectivity of the crop image feature multiscale fusion coding vector, and capture the spatial ambiguous information of the object feature value through the unstructured feature value point prediction of the crop image feature multiscale fusion coding vector, thereby improving the spatial inductive bias perception ability of the feature set of the crop image feature multiscale fusion coding vector, improving the classification regression convergence consistency, and improving the accuracy of the monitoring results obtained by the crop image feature multiscale fusion coding vector through the classifier-based pest and disease monitoring module.

[0064] After determining the monitoring results of pests and diseases, taking effective measures is crucial to ensure the healthy growth of crops and increase yields. Based on the monitoring results, personalized prevention and control strategies can be formulated, taking into account environmental protection and agricultural product safety issues, minimizing the use of chemical pesticides, and promoting green prevention and control technologies.

[0065] Depending on the specific situation of pests and diseases, one or more combinations of physical control, biological control or chemical control can be selected. Physical control does not use chemical drugs, but controls pests and diseases by changing environmental conditions, such as using insect traps to kill pests; setting up insect nets to prevent pests from entering farmland; or using high temperature to treat seeds to prevent the occurrence of diseases. For some specific types of pests and diseases, physical control can be used as a quick and effective emergency measure. Biological control is a method of using natural enemies (such as parasitic wasps, insectivorous birds) or other beneficial microorganisms (such as bacteria and fungi) to control the number of pests. This method is not only environmentally friendly, but also helps to maintain ecological balance. If pests and diseases have reached the level of emergency intervention, chemical control may be a necessary choice. At this time, high-efficiency and low-toxic pesticides should be selected and used strictly according to the recommended dosage. Drone spraying pesticides can not only improve work efficiency, but also ensure the uniformity of pesticide application and reduce environmental pollution. In addition, agricultural management measures can be combined, such as adjusting the planting structure, crop rotation and fallow, to enhance the resistance of crops themselves.

[0066] After the prevention and control measures are implemented, it is necessary to continue to use plant protection drones to regularly collect farmland images, monitor the changes in pests and diseases, compare and analyze the data before and after, evaluate the prevention and control effects, and adjust the subsequent management plan accordingly. This is not only a process to verify the effectiveness of prevention and control measures, but also an important part of continuously optimizing and improving the entire pest and disease management system. Collect and organize the lessons learned from each stage to form a complete solution system, and continuously explore new technologies and methods to meet the increasingly complex needs of agricultural production and environmental protection.

[0067] In summary, the pest and disease monitoring system based on the plant protection drone of the embodiment of the present application is explained, which uses the plant protection drone to collect crop images, and introduces deep learning-based image processing technology in the back-end server, and performs multi-scale analysis of local and global features of the collected crop images to simultaneously capture local lesions and overall growth conditions of crops, and then, through interactive fusion analysis of the multi-scale state features of the crop images, to achieve accurate understanding and intelligent identification of the state of crop pests and diseases. In this way, the accuracy and efficiency of pest and disease monitoring can be effectively improved, thereby providing strong technical support for agricultural production.

[0068] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0069] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0070] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0071] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0072] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not limited thereto. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A pest and disease monitoring system based on plant protection drones, characterized in that: include: A crop image acquisition module, used to control a plant protection drone equipped with a camera to fly along a predetermined route and to use the camera to acquire crop images; An image import module, used to import the crop image into a back-end pest and disease monitoring server; A crop image multi-scale feature detection module is used to perform multi-scale feature detection on the crop image in the back-end pest monitoring server to obtain a crop image local feature coding vector and a crop image global semantic coding vector; A multi-scale feature interactive fusion module, used for performing attention interactive fusion based on implicit collaborative feature guidance on the global semantic coding vector of the crop image and the local feature coding vector of the crop image to obtain a multi-scale fusion coding vector of crop image features; The monitoring result generation module is used to determine the monitoring result of pests and diseases based on the multi-scale fusion coding vector of the crop image features.

2. The pest monitoring system based on plant protection drone according to claim 1 is characterized in that: The crop image multi-scale feature detection module comprises: An image enhancement unit, used for performing grayscale and noise reduction processing on the crop image to obtain an enhanced crop image; A local feature extraction unit, configured to perform local feature detection on the enhanced crop image using a local feature detection module to obtain a local feature coding vector of the crop image; The global feature extraction unit is used to perform global feature detection on the enhanced crop image using a global feature detection module to obtain a global semantic coding vector of the crop image.

3. The pest monitoring system based on plant protection drone according to claim 2 is characterized in that: The local feature detection module uses the SIFT algorithm or the HOG algorithm as the local feature detection algorithm.

4. The pest monitoring system based on plant protection drone according to claim 3 is characterized in that: The global feature detection module performs global feature detection on the enhanced crop image using the ViT model.

5. The pest monitoring system based on plant protection drone according to claim 4 is characterized in that: The multi-scale feature interactive fusion module includes: A collaborative feature extraction unit, used for inputting the crop image local feature coding vector and the crop image global semantic coding vector into a collaborative feature extraction network to obtain an implicit collaborative coding vector between multi-scale features of the crop image; A feature modulation optimization unit is used to perform feature attention modulation on the local feature coding vector of the crop image and the global semantic coding vector of the crop image based on the implicit collaborative coding vector between the multi-scale features of the crop image to obtain an optimized local feature coding vector of the crop image and an optimized global semantic coding vector of the crop image; The multi-scale feature fusion unit is used to fuse the optimized crop image local feature coding vector and the optimized crop image global semantic coding vector to obtain the crop image feature multi-scale fusion coding vector.

6. The pest monitoring system based on plant protection drone according to claim 5, characterized in that: The collaborative feature extraction network includes three parallel feature interaction layers, a feature cascade layer, a point convolution layer and an activation layer based on the Leaky ReLU function.

7. The pest monitoring system based on plant protection drone according to claim 6, characterized in that: The feature modulation optimization unit is used for: Writing the implicit collaborative coding vector between the multi-scale features of the crop image into a dynamic memory unit to obtain a dynamic key vector; Extracting the dynamic key vector from the dynamic memory unit, and inputting the crop image local feature encoding vector and the dynamic key vector into a feature attention modulation module based on a first converter structure to obtain the optimized crop image local feature encoding vector; The dynamic key vector is extracted from the dynamic memory unit, and the crop image global semantic encoding vector and the dynamic key vector are input into a feature attention modulation module based on a second converter structure to obtain the optimized crop image global semantic encoding vector.

8. The pest monitoring system based on plant protection drone according to claim 7, characterized in that: The multi-scale feature fusion unit is used to: The optimized crop image local feature coding vector and the optimized crop image global semantic coding vector are input into a feature linear interaction network for weighted fusion to obtain the crop image feature multi-scale fusion coding vector.

9. The pest monitoring system based on plant protection drone according to claim 8, characterized in that: The monitoring result generating module is used for: The crop image feature multi-scale fusion coding vector is input into a classifier-based pest and disease monitoring module to obtain the monitoring result, and the monitoring result is used to indicate whether there is a pest and disease.

Images