Forest pest automatic identification method based on multispectral image and deep learning

Abstract

The invention relates to the technical field of image recognition, and discloses a multispectral image and deep learning-based forest disease and insect pest automatic recognition method, which comprises the following steps of 1, carrying a multispectral camera containing a red edge wave band through an unmanned aerial vehicle to obtain a forest region image; 2, calculating a red edge normalized vegetation index of the image; 3, performing time sequence modeling on the red edge normalized vegetation index data of more than five consecutive periods, and inputting a time sequence convolutional network to generate an early lesion probability graph; 4, detecting a pest and disease damage target by adopting a multi-scale adaptive feature pyramid network; 5, outputting a disease and pest distribution thermodynamic diagram; and 6, driving the unmanned aerial vehicle cluster to execute precise pesticide spraying. According to the method, through the high sensitivity of the red-edge wave band to chlorophyll degradation and in combination with sequential convolutional network dynamic modeling, an initial lesion area can be recognized 7-10 days before disease development, the early disease recognition capability is remarkably improved, the disease discovery period is shortened, and large-scale disease diffusion is effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to a forest pest automatic identification method based on multi-spectral images and deep learning. BACKGROUND

[0002] With the development of remote sensing technology and image processing technology, multi-spectral images combined with unmanned aerial vehicle platforms have been widely applied in the field of forest pest monitoring. The existing technology usually uses multi-spectral cameras to obtain forest area images, evaluates the health status of vegetation through normalized difference vegetation index (NDVI) and other indicators, and combines traditional machine learning methods to classify and identify pests and diseases. To some extent, this method realizes the preliminary detection of pests and diseases, provides data support for forestry management, and exhibits high efficiency in wide-area monitoring.

[0003] However, the existing technology has significant limitations. First, the monitoring method based on traditional NDVI has insufficient sensitivity to early lesion characteristics, making it difficult to capture weak signals of chlorophyll degradation, resulting in low accuracy of early pest and disease identification. SUMMARY

[0004] To overcome the above shortcomings, the present application provides a forest pest automatic identification method based on multi-spectral images and deep learning, aiming to improve the sensitivity of the monitoring method based on traditional NDVI to early lesion characteristics, making it difficult to capture weak signals of chlorophyll degradation, resulting in low accuracy of early pest and disease identification.

[0005] In a first aspect, the present application provides a forest pest automatic identification method based on multi-spectral images and deep learning, comprising the following steps:

[0006] Step one, obtain forest area images by unmanned aerial vehicle carrying multi-spectral cameras containing red edge band, the wavelength range of the red edge band is 705 nanometers to 750 nanometers;

[0007] Step two, calculate the red edge normalized difference vegetation index of the image, the formula is the red edge band reflectance minus the red light band reflectance, divided by the red edge band reflectance plus the red light band reflectance;

[0008] Step three, time series modeling of more than five consecutive periods of red edge normalized difference vegetation index data, input time series convolution network to generate early lesion probability map;

[0009] Step four, use multi-scale adaptive feature pyramid network to detect pest and disease targets, the network embeds Transformer encoding layer and background suppression module;

[0010] Step five, execute steps three to four in real time through edge computing devices, and output pest and disease distribution heat map;

[0011] Step six, generate a variable-granularity pesticide application grid according to the hazard level of the distribution heat map, and drive the UAV cluster to perform precise pesticide spraying.

[0012] Through the above technical solution, the UAV carries a red edge band multispectral camera to obtain forest area images; the red edge normalized vegetation index is calculated; the time series modeling is performed on more than five consecutive RENDVI data, the early lesion probability map is generated by inputting the time series convolution network, the multi-scale adaptive feature pyramid network with embedded Transformer encoding layer and background suppression module is used to detect the disease and pest targets, the edge computing device generates the disease and pest distribution heat map in real time, and the variable-granularity pesticide application grid is generated according to the hazard level of the heat map to drive the UAV cluster to perform precise pesticide spraying. The method significantly improves the sensitivity and accuracy of early lesion detection through the red edge normalized vegetation index and the time series convolution network.

[0013] Preferably, the time series data of the red edge normalized vegetation index in step two needs to be processed by wavelet denoising, and the decay slope is used as the input feature of the time series convolution network.

[0014] Preferably, the background suppression module execution process includes:

[0015] The near-infrared reflectance and dynamic light threshold are used to generate a forest canopy shadow mask;

[0016] The mask and the feature map of the multi-scale adaptive feature pyramid network are subjected to element-by-element multiplication operation to suppress background interference.

[0017] Preferably, the multi-scale adaptive feature pyramid network includes a backbone network, a feature fusion layer, and a prediction head:

[0018] The backbone network is an improved GhostNet structure, and the channel number of the convolution layer is compressed to one fourth of the standard convolution;

[0019] The feature fusion layer calculates the spatial attention weight of the multi-scale feature through the Transformer encoding layer;

[0020] The prediction head adopts a dynamic anchor box mechanism, and the anchor box size is adaptively adjusted according to the tree crown height.

[0021] Preferably, the reference size of the dynamic anchor box mechanism is determined by the product of the UAV flight height and the tree crown width coefficient, wherein the pine crown width coefficient is 0.2, and the broadleaf tree crown width coefficient is 0.3.

[0022] Preferably, the edge computing device deploys a two-stage distillation model:

[0023] The first stage distillation uses the lesion area bounding box output of the teacher model as the soft label;

[0024] The second stage distillation calculates a loss function with the feature map difference between the teacher model and the student model.

[0025] Preferably, the division rule of the variable granularity pesticide application grid is:

[0026] The first hazard level corresponds to a five-meter-by-five-meter grid granularity, and a one-hundred-milliliter spray amount per plant is executed.

[0027] The second hazard level corresponds to a ten-meter-by-ten-meter grid granularity, and a fifty-milliliter spray amount per plant is executed.

[0028] The third hazard level corresponds to a twenty-meter-by-twenty-meter grid granularity, and is marked as an artificial review area.

[0029] In a second aspect, the present application provides the following technical solution: a forest pest automatic identification system based on multispectral images and deep learning, comprising:

[0030] A UAV platform carries a multispectral camera and an embedded processor;

[0031] An edge computing module is used to execute a forest pest automatic identification method based on multispectral images and deep learning;

[0032] A decision control terminal receives a pest distribution heat map and generates a pesticide application path instruction;

[0033] A UAV cluster controller dispatches a pesticide application UAV according to the pesticide application path instruction.

[0034] In a third aspect, the present application provides the following technical solution: a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned forest pest automatic identification method based on multispectral images and deep learning.

[0035] In a fourth aspect, the present application provides the following technical solution: a readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the above-mentioned forest pest automatic identification method based on multispectral images and deep learning.

[0036] The present application has the following beneficial effects:

[0037] 1、In the present application, by the high sensitivity of the red edge band to chlorophyll degradation and the dynamic modeling of the time series convolution network, the initial disease area can be identified 7-10 days before the disease is symptomatic, the early disease identification capability is significantly improved, the disease discovery cycle is shortened, and large-scale disease spread is effectively avoided.

[0038] 2、The multi-scale adaptive feature pyramid network in the application embeds a background suppression module, effectively filters out interference such as clouds, fog and canopy shadow, has a high single tree disease detection rate and a low missed detection rate, and solves the problem of small target missed detection caused by background interference in traditional methods.

[0039] 3、In the application, the red edge time decay slope, the super-resolution reconstructed image and the spatial distribution thermal map are fused, the disease evolution trend is quantified through the Gaussian mixture model, data basis is provided for prevention and control priority division and pesticide dosage allocation, and the blindness of traditional experience decision is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The method flowchart of the automatic forest pest identification method based on multispectral images and deep learning provided by the application is shown in the figure.

[0041] Figure 2 The system framework diagram of the automatic forest pest identification system based on multispectral images and deep learning provided by the application is shown in the figure. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0043] Embodiment one

[0044] Reference Figure 1 A national pine forest protection area is located in a temperate monsoon climate zone, and the protection area needs to implement early warning and accurate prevention and control of pine wood nematode disease. The terrain of the protection area is complex, and traditional manual patrol cannot cover the whole area, so an automatic forest pest identification method based on multispectral images and deep learning is needed to realize real-time identification and rapid response to diseases, as shown in the figure. Figure 1 As shown in the figure, it includes:

[0045] Step one, acquire forest area images by unmanned aerial vehicle carrying multispectral camera containing red edge band, the wavelength range of red edge band is 705 nanometers to 750 nanometers.

[0046] Specifically, data acquisition: unmanned aerial vehicle carries multispectral camera, contains 705-750nm red edge band, and acquires image with spatial resolution of 0.1 meter / pixel.

[0047] Reflectivity conversion: extract red band reflectivity matrix ρ Red and red edge band reflectivity matrix ρ RedEdge . Wherein:

[0048] ρRed represents 650-680 nm band reflectance, healthy pine needles show low values 0.08-0.12 due to chlorophyll absorption;

[0049] p RedEdge represents 705-750 nm band reflectance, sensitive to chlorophyll degradation, significantly reduced in diseased leaves.

[0050] The spatial resolution of the multispectral image is 0.1 meters per pixel, and the reflectance data of each band is stored as a high-precision floating-point matrix. The reflectance data of the red band and the red edge band are extracted from the multispectral image. The red band reflectance reflects the chlorophyll absorption characteristics of the leaves, and the red edge band reflectance is highly sensitive to chlorophyll degradation. By calculating the red edge normalized vegetation index, a vegetation health quantization matrix is generated, with a value range of -1 to 1, and the typical value of healthy pine forest is higher than 0.6.

[0051] Further, step two, calculate the red edge normalized vegetation index of the image, the formula is red edge band reflectance minus red band reflectance, divided by red edge band reflectance plus red band reflectance; the time series data of the red edge normalized vegetation index in step two needs to be processed by wavelet denoising, and the decay slope is taken as the input feature of the time series convolution network.

[0052] Specifically, the red edge vegetation index is calculated: The output RENDVI matrix has a value range of [-1, 1], and the typical value of healthy pine forest is > 0.6;

[0053] Time series decay slope modeling: linear regression is performed on the RENDVI data of the last 5 periods with a 7-day interval: where t i is the time index, RENDVI i is the mean value of the ii-th period RENDVI, n=5 is the number of time series, and k is the decay slope, a negative value indicating the chlorophyll degradation rate;

[0054] The red edge normalized vegetation index data collected in the last five periods are constructed into a time series with a 7-day interval, and the environmental noise is filtered out by wavelet denoising algorithm. Then, linear regression analysis is performed on the time series data of each pixel point to fit the decay slope feature. The slope quantifies the chlorophyll degradation rate of the leaves, and a negative value indicates a continuous decrease in health. The decay slope matrix is input into the time series convolution network, which uses a four-layer causal convolution structure with a kernel size of three and an expansion coefficient that increases layer by layer. The expansion convolution operation captures long-term temporal dependencies, and the activation function is used to process the output of the disease probability map. The probability value range is zero to one, indicating the risk level of leaf disease.

[0055] Further, step three, time series modeling is performed on the red edge normalized vegetation index data of more than five consecutive periods, and the early disease probability map is generated by inputting the time series convolution network.

[0056] Specifically, the TCN operation: input the decay slope matrix k into a 4-layer causal convolution network: P = σ (W4 * ReLU (W3 * …ReLU (W1 * k))), where W i is the weight of the second convolution kernel, the kernel size is 3, the dilation coefficient is [1, 2, 4, 8]; * is the dilated convolution operation, and σ is the Sigmoid activation function. The output is the early lesion probability map P ∈ [0, 1]. The decay slope matrix is input into the TCN, which contains 4 layers of causal convolution layers with dilation coefficients of 1, 2, 4, and 8 in turn, and the activation function is ReLU. The output is the early lesion probability map, and the probability value ranges from 0 to 1.

[0057] Further, step four, adopt a multi-scale adaptive feature pyramid network to detect pest and disease targets. The network embeds a Transformer encoding layer and a background suppression module. The background suppression module performs the following process: generate a canopy shadow mask using near-infrared reflectance and a dynamic light threshold; perform element-wise multiplication between the mask and the feature map of the multi-scale adaptive feature pyramid network to suppress background interference; the multi-scale adaptive feature pyramid network includes a backbone network, a feature fusion layer, and a prediction head: the backbone network is an improved GhostNet structure, and the number of channels of the convolution layer is compressed to one-fourth of the standard convolution; the feature fusion layer calculates the spatial attention weight of the multi-scale feature through the Transformer encoding layer; the prediction head adopts a dynamic anchor box mechanism, and the anchor box size is adaptively adjusted according to the tree crown height; the reference size of the dynamic anchor box mechanism is determined by the product of the unmanned aerial vehicle flight height and the tree crown width coefficient, wherein the pine crown width coefficient is 0.2, and the broad-leaved tree crown width coefficient is 0.3.

[0058] Specifically, super-resolution reconstruction: input the original image I low , and generate a high-resolution image I high through a lightweight Pest-SRGAN:

[0059] Generator structure: 4 residual blocks + sub-pixel convolution layer,

[0060] Loss function:

[0061] Background suppression module (BSM): Where ρ NIR is the near-infrared reflectance, τ shadow is the dynamic shadow threshold, which is adaptively calibrated according to the light intensity, and the output is multiplied element-wise with the feature map to suppress cloud and fog interference,

[0062] Dynamic anchor box mechanism: anchor box side length = k species ·H, where H is the unmanned aerial vehicle flight height, and kspecies C is the crown area coefficient, for pine trees = 0.2, for broadleaf trees = 0.3,

[0063] Transformer-CNN feature fusion: Where Q, K, V are query / key / value matrices generated by linear transformation of feature maps, d k is a scaling factor, taking the square root of the number of feature map channels;

[0064] Further, step five, real-time execution of steps three to four through the edge computing device, outputting a pest and disease distribution heat map; step six, generating a variable granularity pesticide application grid according to the hazard level of the distribution heat map, driving the drone swarm to perform precision pesticide spraying; the edge computing device deploys a two-stage distillation model: the first stage distillation takes the output of the teacher model as the soft label of the disease area bounding box; the second stage distillation calculates the loss function with the feature map difference between the teacher model and the student model; the variable granularity pesticide application grid division rule is: the first hazard level corresponds to a five-meter by five-meter grid granularity, and a single plant sprays one hundred milliliters of pesticide; the second hazard level corresponds to a ten-meter by ten-meter grid granularity, and a single plant sprays fifty milliliters of pesticide; the third hazard level corresponds to a twenty-meter by twenty-meter grid granularity, and is marked as an artificial review area;

[0065] Specifically, two-stage knowledge distillation: The teacher model is ResNet50, and the student model is an improved GhostNet, and the dynamic switching logic is:

[0066]

[0067] Table I: Variable granularity pesticide application decision: input disease density

[0068]

[0069]

[0070] Drone swarm path planning: minimize total length of pesticide spraying path Where (x i , y i ) is the center coordinate of the ii-th high-risk grid, based on the GIS coordinate system.

[0071] Embodiment two:

[0072] Referring Figure 2 In the second embodiment of the present application, the present application provides an automatic forest pest and disease identification system based on multispectral images and deep learning, comprising:

[0073] A drone platform carrying a multispectral camera and an embedded processor;

[0074] An edge computing module is configured to execute the automatic forest pest identification method based on multi-spectrum images and deep learning;

[0075] A decision control terminal is configured to receive the pest distribution heat map and generate pesticide application path instructions;

[0076] A UAV cluster controller is configured to dispatch pesticide spraying UAVs according to the pesticide application path instructions.

[0077] Specifically, the UAV platform is equipped with a multi-spectrum camera and an embedded processor for acquiring multi-spectrum image data in the forest area; the edge computing module executes the pest identification algorithm in real time, including red edge feature extraction, time series convolution network prediction and multi-scale target detection; the decision control terminal generates a variable-granularity pesticide application grid based on the pest distribution heat map and divides the pesticide spraying amount according to the damage level; the UAV cluster controller dispatches the pesticide spraying UAVs to perform precise prevention and control and optimize the pesticide application path; the system realizes real-time processing through onboard edge computing, combines dynamic grid decision-making with UAV cooperative operation, and completes the closed-loop management from data acquisition to prevention and control execution.

[0078] Embodiment three

[0079] The third embodiment of the present application is based on the same inventive concept, and the present application provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the automatic forest pest identification method based on multi-spectrum images and deep learning of the above-mentioned embodiments.

[0080] Embodiment four

[0081] The fourth embodiment of the present application is based on the same inventive concept, and the present application provides a computer device, which comprises a processor and a memory; the processor and the memory communicate with each other; the memory is used to store instructions; and the processor is used to execute the instructions in the memory to implement the automatic forest pest identification method based on multi-spectrum images and deep learning of the above-mentioned embodiments.

[0082] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0083] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A forest pest automatic identification method based on multispectral image and deep learning, characterized in that, The method comprises the following steps: Step 1: Obtain forest area images by a UAV carrying a multispectral camera containing a red edge band, the wavelength range of the red edge band being 705-750 nm; Step 2: Calculate the red edge normalized vegetation index of the images, the formula being red edge band reflectance minus red band reflectance, divided by red edge band reflectance plus red band reflectance; Step 3: Perform time series modeling on the red edge normalized vegetation index data of more than five consecutive periods, input a time series convolution network to generate an early lesion probability map; Step 4: Detect the pest and disease targets by using a multi-scale adaptive feature pyramid network, the network embedding a Transformer encoding layer and a background suppression module; Step 5: Real-time execute steps 3-4 by an edge computing device, and output a pest and disease distribution heat map; Step 6: Generate a variable-granularity pesticide application grid according to the hazard level of the distribution heat map, and drive a UAV cluster to perform precise pesticide spraying. 2.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 1, characterized in that, The time series data of the red edge normalized vegetation index in step 2 need to be processed by wavelet denoising, and the decay slope is used as the input feature of the time series convolution network. 3.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 1, characterized in that, The background suppression module performs the following process: Generate a forest canopy shadow mask using near-infrared band reflectance and a dynamic light threshold; Perform element-wise multiplication operation on the mask and the feature map of the multi-scale adaptive feature pyramid network to suppress background interference. 4.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 1, characterized in that, The multi-scale adaptive feature pyramid network comprises a backbone network, a feature fusion layer and a prediction head: The backbone network is an improved GhostNet structure, and the channel number of the convolution layer is compressed to one fourth of that of the standard convolution; The feature fusion layer calculates the spatial attention weight of the multi-scale feature by a Transformer encoding layer; The prediction head adopts a dynamic anchor box mechanism, and the anchor box size is adaptively adjusted according to the tree crown height. 5.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 4, characterized in that, The reference size of the dynamic anchor box mechanism is determined by the product of the UAV flight height and the tree crown width coefficient, wherein the pine crown width coefficient is 0.2, and the broad-leaved tree crown width coefficient is 0.

3. 6.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 1, characterized in that, The edge computing device deploys a two-stage distillation model: The first stage distillation uses the lesion area bounding box output of the teacher model as the soft label; The second stage distillation calculates the loss function by the difference between the feature maps of the teacher model and the student model. 7.The forest pest automatic identification method based on multi-spectral image and deep learning according to claim 1, characterized in that, The division rule of the variable-granularity pesticide application grid is as follows: The first hazard level corresponds to a five-meter-by-five-meter grid granularity, and a single plant is sprayed with 100 milliliters of pesticide; The second hazard level corresponds to a ten-meter-by-ten-meter grid granularity, and a single plant is sprayed with 50 milliliters of pesticide; The third hazard level corresponds to a twenty-meter-by-twenty-meter grid granularity, and is marked as an artificial review area.

8. An automatic forest pest identification system based on multispectral images and deep learning, characterized in that, The method for automatically identifying forest pests and diseases based on multispectral images and deep learning according to any one of claims 1-7 comprises: A UAV platform carrying a multispectral camera and an embedded processor; An edge computing module for executing the method for automatically identifying forest pests and diseases based on multispectral images and deep learning; A decision control terminal receiving the pest and disease distribution heat map and generating pesticide application path instructions; A UAV cluster controller scheduling pesticide spraying UAVs according to the pesticide application path instructions.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the forest pest automatic identification method based on multispectral images and deep learning according to any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, characterized by, The readable storage medium stores the computer program, and the computer program is executed by the processor to implement the forest pest automatic identification method based on multispectral images and deep learning according to any one of claims 1 to 7.

Images