Early warning method and device for intelligently identifying forest plant diseases and insect pests based on visible light and multispectrum

Abstract

The invention provides an early warning method and device for intelligently identifying forest diseases and insect pests based on visible light and multispectrums. The method comprises the following steps: acquiring a visible light image and a multispectral image of a forest region to be early warned; inputting the visible light image into a first preset prediction network model, and outputting a first pest recognition result; determining a target point in each predicted pest area and coordinate information of the target point; determining the image area of the target detection area based on the forest tree type of the forest tree area to be pre-warned, and determining a local multi-spectral image corresponding to the target detection area in the multi-spectral image based on the coordinate information of the target point and the image area of the target detection area; and inputting the local multispectral image into a second preset prediction network model, outputting a second pest and disease identification result, and performing pest and disease early warning on the to-be-early-warned forest region according to the second pest and disease identification results of the plurality of target detection regions. According to the scheme, the algorithm complexity of the scheme is reduced, and the computing power is saved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural technologies, and particularly to a warning method and device for intelligently identifying forest tree diseases and pests based on visible light and multispectral. Background Art

[0002] Forest tree diseases and pests pose a serious threat to forest health and ecological balance. Traditional monitoring and warning methods often rely on manual inspections, which are inefficient and have limited coverage. With the development of remote sensing technology, visible light images and multispectral images provide new solutions for forest tree disease and pest warning. These image technologies can capture the spectral characteristic changes of forest trees under the influence of diseases and pests, providing a data basis for accurate warning.

[0003] Currently, when using visible light and multispectral for forest tree disease and pest warning, generally, image fusion technologies and algorithms are first adopted to fuse visible light images and multispectral images, and then the fused images obtained after fusion are used to carry out forest tree disease and pest warning by using this information. Image fusion is generally divided into three different processing levels: pixel level, feature level, and decision level. However, no matter which processing level of image fusion it is, its complexity and the required computing power are relatively high. When warning for a large area of forest trees is needed, this image fusion process often requires complex processing of images from different sources, consuming a large amount of computing power, resulting in too long warning time and thus untimely warning. Summary of the Invention

[0004] The present invention provides a warning method and device for intelligently identifying forest tree diseases and pests based on visible light and multispectral, so as to solve the problem in the prior art that a large amount of computing power is consumed, resulting in too long warning time and thus untimely warning.

[0005] On the one hand, the present invention provides a warning method for intelligently identifying forest tree diseases and pests based on visible light and multispectral, including:

[0006] Obtain a visible light image and a multispectral image of a forest tree area to be warned, and obtain the coordinate information of multiple ground control points in the forest tree area to be warned;

[0007] Input the visible light image into a first preset prediction network model, and output a first disease and pest identification result, where the first disease and pest identification result is a visible light image marked with a predicted disease and pest area; the first preset prediction network model is trained through forest tree visible light image samples, and the forest tree visible light image samples are marked with first disease and pest identification result labels;

[0008] For each predicted disease and pest area, determine the target points in the predicted disease and pest area, and determine the coordinate information of the target points based on the coordinate information of multiple ground control points;

[0009] Determine the image area of the target detection area based on the tree species in the forest area to be warned, and determine the corresponding local multi-spectral image of the target detection area in the multi-spectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area;

[0010] Input the local multi-spectral image into the second preset prediction network model, output the second pest and disease identification result, and perform pest and disease warning on the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; The second prediction network model is trained through forest multi-spectral image samples, and the forest multi-spectral image samples are marked with the second pest and disease identification results.

[0011] In an alternative embodiment of the present invention, the method further includes:

[0012] Obtain the warning response duration required for the forest area to be warned, and determine the number of reference spectral feature parameters;

[0013] Input the local multi-spectral image samples of the detection areas corresponding to multiple different image areas into the second preset prediction network model, process the spectral feature parameters of the number of reference spectral feature parameters of the local multi-spectral image samples through the second preset network model, output the corresponding second pest and disease identification results, and record the duration required for the output of the corresponding second pest and disease identification results;

[0014] Determine the image area of the detection area corresponding to the local multi-spectral image sample whose output required duration matches the warning response duration as the reference image area;

[0015] Determining the image area of the target detection area based on the tree species in the forest area to be warned includes:

[0016] Determine the required number of target spectral feature parameters based on the current required warning accuracy level of the forest area to be warned;

[0017] Determine the image area of the target detection area based on the number of target spectral feature parameters, the number of reference spectral feature parameters, and the reference image area.

[0018] In an alternative embodiment of the present invention, determining the image area of the target detection area based on the number of target spectral feature parameters, the number of reference spectral features, and the reference image area includes:

[0019] If the number of target spectral feature parameters is equal to the number of reference spectral feature parameters, then determine the reference image area as the image area of the target detection area;

[0020] If the number of target spectral feature parameters is greater than the number of reference spectral feature parameters, the image area of the target detection region is obtained by reducing the area of the reference image by a specified step size;

[0021] If the number of target spectral feature parameters is less than the number of reference spectral feature parameters, the image area of the target detection region is obtained by increasing the area of the reference image by a specified step size.

[0022] In an alternative embodiment of the present invention, the image area of the target detection region is determined based on the tree species in the forest area to be warned, and the local multi-spectral image corresponding to the target detection region in the multi-spectral image is determined based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection region, including:

[0023] Determine the first position of the target point in the multi-spectral image based on the target point and the coordinate information of the target point;

[0024] Based on the preset shape of the target detection region, the image area of the target detection region, and the first position, determine multiple second positions of the edge of the target detection region in the multi-spectral image;

[0025] Based on the first position and each second position, obtain the local multi-spectral image corresponding to the target detection region in the multi-spectral image.

[0026] In an alternative embodiment of the present invention, the shape of the target detection region is a square or a circle.

[0027] In an alternative embodiment of the present invention, for each predicted pest and disease area, determining the target point in the predicted pest and disease area includes:

[0028] Obtain the tree density of the predicted pest and disease area, and assign a normalized tree density value to each pixel point in the predicted pest and disease area;

[0029] Based on a preset density clustering algorithm, obtain multiple pixel clustering clusters in the predicted pest and disease area, and determine the clustering center of the clustering cluster with the highest average density value as the target point.

[0030] In an alternative embodiment of the present invention, according to the second pest and disease identification results of the target detection regions corresponding to multiple predicted pest and disease areas, warning of pests and diseases in the forest area to be warned includes:

[0031] For each predicted pest and disease area, if both the first pest and disease identification result and the second pest and disease identification result indicate the presence of pests and diseases, issue a first-level warning and feedback the position information corresponding to the target detection region;

[0032] If the first pest and disease identification result indicates the existence of a pest and disease risk, and the second pest and disease identification result indicates the non-existence of a pest and disease risk, a secondary warning is issued, and the position information corresponding to the predicted pest and disease area is fed back.

[0033] In a second aspect, the present invention provides an early warning device for intelligently identifying forest pests and diseases based on visible light and multispectral, including:

[0034] An image acquisition module, configured to acquire a visible light image and a multispectral image of the forest area to be warned, and acquire the coordinate information of multiple ground control points in the forest area to be warned;

[0035] A first pest and disease identification result acquisition module, configured to input the visible light image into a first preset prediction network model, and output a first pest and disease identification result, where the first pest and disease identification result is a visible light image marked with a predicted pest and disease area; the first preset prediction network model is trained by a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label;

[0036] A target point acquisition module, configured to, for each predicted pest and disease area, determine the target point in the predicted pest and disease area, and determine the coordinate information of the target point based on the coordinate information of multiple ground control points;

[0037] A local multispectral image acquisition module, configured to determine the image area of the target detection area based on the tree species of the forest area to be warned, and determine the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area;

[0038] A second pest and disease identification result acquisition and early warning module, configured to input the local multispectral image into a second preset prediction network model, output a second pest and disease identification result, and perform pest and disease early warning on the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is trained by a forest multispectral image sample, and the forest multispectral image sample is marked with a second pest and disease identification result.

[0039] In an optional embodiment of the present invention, the device further includes a reference image area determination module, configured to:

[0040] Acquire the warning response duration required for the forest area to be warned, and determine the number of reference spectral feature parameters;

[0041] Input local multi-spectral image samples of multiple detection regions corresponding to different image areas into the second preset prediction network model. Process the spectral feature parameters of the number of reference spectral feature parameters of the local multi-spectral image samples through the second preset network model, output the corresponding second pest and disease identification results, and record the time required for outputting the corresponding second pest and disease identification results.

[0042] Determine the image area of the detection region corresponding to the local multi-spectral image sample whose output required time matches the early warning response time as the reference image area.

[0043] Determine the image area of the target detection region based on the tree species of the forest area to be warned, including:

[0044] The local multi-spectral image acquisition module is specifically used for:

[0045] Determine the image area of the target detection region based on the number of target spectral feature parameters, the number of reference spectral feature parameters, and the reference image area.

[0046] In an alternative embodiment of the present invention, the local multi-spectral image acquisition module is further used for:

[0047] If the number of target spectral feature parameters is equal to the number of reference spectral feature parameters, determine the reference image area as the image area of the target detection region;

[0048] If the number of target spectral feature parameters is greater than the number of reference spectral feature parameters, obtain the image area of the target detection region after reducing the reference image area by a specified step size;

[0049] If the number of target spectral feature parameters is less than the number of reference spectral feature parameters, obtain the image area of the target detection region after increasing the reference image area by a specified step size.

[0050] In an alternative embodiment of the present invention, the local multi-spectral image acquisition module is further used for:

[0051] Determine the first position of the target point in the multi-spectral image based on the target point and the coordinate information of the target point;

[0052] Based on the preset shape of the target detection region, the image area of the target detection region, and the first position, determine multiple second positions of the edge of the target detection region in the multi-spectral image;

[0053] Based on the first position and each second position, obtain the local multi-spectral image corresponding to the target detection region in the multi-spectral image.

[0054] In an alternative embodiment of the present invention, the shape of the target detection region is a square or a circle.

[0055] In an alternative embodiment of the present invention, the target point acquisition module is specifically configured to:

[0056] Obtain the forest tree density of the predicted pest and disease area, and assign a normalized forest tree density value to each pixel point in the predicted pest and disease area;

[0057] Based on a preset density clustering algorithm, obtain multiple pixel clustering clusters in the predicted pest and disease area, and determine the clustering center of the clustering cluster with the highest average density value as the target point.

[0058] In an alternative embodiment of the present invention, the second pest and disease identification result acquisition and early warning module is specifically configured to:

[0059] For each predicted pest and disease area, if both the first pest and disease identification result and the second pest and disease identification result indicate the existence of pests and diseases, issue a first-level early warning and feedback the corresponding position information of the target detection area;

[0060] If the first pest and disease identification result indicates the existence of a pest and disease risk, and the second pest and disease identification result indicates the non-existence of a pest and disease risk, issue a second-level early warning and feedback the corresponding position information of the predicted pest and disease area.

[0061] In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements any one of the above-mentioned early warning methods for intelligently identifying forest tree pests and diseases based on visible light and multispectral.

[0062] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above-mentioned early warning methods for intelligently identifying forest tree pests and diseases based on visible light and multispectral.

[0063] In a fifth aspect, the present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements any one of the above-mentioned early warning methods for intelligently identifying forest tree pests and diseases based on visible light and multispectral.

[0064] An early warning method and device for intelligent identification of forest tree diseases and pests based on visible light and multispectral images provided by the present invention obtain a visible light image and a multispectral image of a forest tree area to be warned, and obtain coordinate information of multiple ground control points in the forest tree area to be warned; input the visible light image into a first preset prediction network model to output a first disease and pest identification result, where the first disease and pest identification result is a visible light image marked with a predicted disease and pest area; for each predicted disease and pest area, determine a target point in the predicted disease and pest area, and determine the coordinate information of the target point based on the coordinate information of multiple ground control points; determine the image area of the target detection area based on the tree species of the forest tree area to be warned, and determine the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area; input the local multispectral image into a second preset prediction network model to output a second disease and pest identification result, and perform early warning of diseases and pests on the forest tree area to be warned according to the second disease and pest identification results of the target detection areas corresponding to multiple predicted disease and pest areas. This solution first uses the visible light image for preliminary detection, and then uses the local hyperspectral image for accurate detection. On the one hand, there is no need to fuse the overall visible light image and hyperspectral image of the forest tree area to be warned, which reduces the complexity of the solution algorithm, greatly saves computing power, and ensures the timeliness of early warning. On the other hand, through preliminary detection, subsequent accurate detection does not need to detect the overall hyperspectral image of the forest tree area to be warned, and at the same time, the image area of the local hyperspectral image for accurate detection is also controlled, further reducing the complexity of the solution algorithm, greatly saving computing power, and ensuring the timeliness of early warning. Description of the Drawings

[0065] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0066] Figure 1 It is a schematic flow chart of an early warning method for intelligent identification of forest tree diseases and pests based on visible light and multispectral images provided by the present invention;

[0067] Figure 2 It is a structural block diagram of an early warning device for intelligent identification of forest tree diseases and pests based on visible light and multispectral images provided by the present invention;

[0068] Figure 3 It is a schematic structural diagram of an electronic device provided by the present invention. Detailed Embodiments

[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts fall within the protection scope of the present invention.

[0070] The embodiment of the present invention provides a system architecture on which an early warning method for intelligent identification of forest pests and diseases based on visible light and multispectral depends. The system may include an early warning background server, an early warning front end, and an image acquisition device. Among them, the image acquisition device may be a remote sensing satellite capable of acquiring visible light images and multispectral images, or an operation device such as a drone equipped with a visible light image acquisition sensor and a multispectral image acquisition sensor. The image acquisition device acquires images of the forest area to be warned and sends the acquired visible light images and multispectral images to the early warning background server. In the early warning background server, the relevant network models and algorithm programs mentioned in the embodiment of the present invention are deployed, and the early warning result is obtained through the processing and operation of the early warning background server. The early warning background server then sends the early warning result to the early warning front end, which may be various devices with early warning functions that users can use or perceive, such as mobile phones, personal computers, or relevant early warning devices. The following will illustrate the specific implementation process of the early warning method for intelligent identification of forest pests and diseases based on visible light and multispectral in the embodiment of the present invention.

[0071] Figure 1 It is a schematic flowchart of an early warning method for intelligent identification of forest pests and diseases based on visible light and multispectral provided by an embodiment of the present invention. As Figure 1 shown, the method may include:

[0072] Step S101, obtain the visible light image and multispectral image of the forest area to be warned, and obtain the coordinate information of multiple ground control points in the forest area to be warned.

[0073] Among them, the visible light image can intuitively show the growth status, color change, and morphological characteristics of the forest. Under the attack of pests and diseases, the characteristics such as the leaf color, morphology, and texture of the forest will change significantly, and these changes can be clearly captured in the visible light image.

[0074] The multispectral image can obtain spectral information in different bands, including invisible light bands such as infrared and ultraviolet. These bands are more sensitive to the subtle changes caused by pests and diseases and can provide richer information. Through multispectral image analysis, the types, degrees, and spread ranges of pests and diseases can be identified.

[0075] Specifically, visible light images and multi-spectral images of the forest area to be warned can be collected by remote sensing satellites, or visible light images and multi-spectral images of the forest area to be warned can be collected by drones carrying relevant collection equipment.

[0076] Among them, a Ground Control Point (GCP) is a known point on the ground used to calibrate the geographical information on satellite or aerial images in photogrammetry and remote sensing. These points have accurate known position coordinates, which can be obvious ground feature points (such as bridge corners, road intersections, etc.) or artificially set targets (such as survey markers). The GCP provides reference information on the geographical location for the collected visible light images and hyperspectral images, thus ensuring that each pixel on the image can be accurately mapped to the actual geographical coordinates.

[0077] Step S102: Input the visible light image into the first preset prediction network model to output the first pest and disease identification result. The first pest and disease identification result is a visible light image marked with the predicted pest and disease area; the first preset prediction network model is trained through forest visible light image samples, and the forest visible light image samples are marked with the first pest and disease identification result labels.

[0078] Specifically, in the embodiments of the present invention, the visible light image and the hyperspectral image are not fused. First, the prediction of the forest pest and disease area is based on the visible light image. Specifically, the first preset prediction network model can adopt a Convolutional Neural Networks (CNN) model, and the YOLOv5 (You Only Look Once version 5) model is adopted in the embodiments of the present invention. This network model includes parts such as an input layer, a feature extraction layer, a feature fusion layer, a detection head layer, and an output layer. First, preprocess the visible light image, such as denoising and enhancing it. Then, receive the preprocessed visible light image as input through the input layer, adjust the visible light image to the size and resolution required by the model, and perform normalization processing. The feature extraction layer extracts the feature information in the image through convolution operations, including color, texture, shape, etc., and uses multiple convolutional layers, activation functions (such as ReLU), and pooling layers to extract and compress the image features layer by layer. The feature fusion layer fuses the feature information of different scales to improve the detection accuracy and robustness of the model, and fuses the feature maps of different layers through methods such as upsampling, downsampling, or skip connections. The detection head layer predicts the position and category of the pest and disease area according to the extracted feature information, and uses structures such as convolutional layers and fully connected layers to process the feature map to generate a prediction result containing information such as the position, size, and category of the pest and disease area. The output layer outputs the final pest and disease area prediction result, decodes and visualizes the prediction result generated by the detection head to obtain an easy-to-understand pest and disease area distribution map, that is, the first pest and disease identification result.

[0079] Step S103: For each predicted pest and disease area, determine the target point in the predicted pest and disease area, and determine the coordinate information of the target point based on the coordinate information of multiple ground control points.

[0080] Specifically, after obtaining the first pest and disease identification result through the above steps, that is, obtaining multiple predicted pest and disease areas through visible light images, the solution of the present invention will use the method of hyperspectral image detection to further determine whether there is a pest and disease risk in the predicted pest and disease area. Before that, a target point needs to be determined for each predicted pest and disease area, and then when using the method of hyperspectral image detection to further determine whether there is a pest and disease risk in the predicted pest and disease area, the corresponding processing area, that is, the local hyperspectral image, can be determined.

[0081] It can be understood that in the foregoing steps, based on the prediction of the visible light image, multiple predicted pest and disease areas can be determined. These predicted pest and disease areas are the areas for subsequent hyperspectral image detection, that is, the hyperspectral image detection is not performed on the entire forest area to be warned. In other words, first use the visible light image for preliminary screening, and then use the corresponding local hyperspectral image for fine prediction on the basis of the preliminary screening.

[0082] Step S104: Determine the image area of the target detection area based on the tree species in the forest area to be warned. Determine the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area.

[0083] Among them, in the embodiment of the present invention, the predicted pest and disease area is not directly used as the target detection area for subsequent hyperspectral detection. Instead, the target point of the predicted pest and disease area is first determined from the predicted pest and disease area, and then the local multispectral image during multispectral detection is determined based on the target point and the image area of the target detection range. This method not only retains the regional position information during the preliminary screening in the foregoing steps during subsequent multispectral monitoring, but also further ensures that the multispectral detection time will not be too long by controlling the image size of the target detection area.

[0084] Step S105: Input the local multispectral image into the second preset prediction network model, output the second pest and disease identification result, and perform pest and disease warning on the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is trained through forest multispectral image samples, and the forest multispectral image samples are marked with the second pest and disease identification results.

[0085] Specifically, the second preset prediction network model in the embodiments of the present invention also uses CNN, but this second prediction network model is a three-dimensional CNN. Since multi-spectral images usually contain multiple bands, and each band represents a specific spectral information (i.e., spectral feature parameters), the three-dimensional convolution kernel can effectively capture the relationships between different bands.

[0086] The second preset prediction network model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. After preprocessing each local multi-spectral image, the input layer receives the multi-local spectral images as inputs. Since multi-spectral images are usually three-dimensional (width, height, and number of bands), the input layer needs to be able to process this three-dimensional data, adjust the local multi-spectral images to the size and resolution required by the network model, and perform necessary preprocessing (such as normalization, denoising, etc.). The convolutional layer extracts features in the image through convolutional operations. In multi-spectral images, the convolutional layer can learn the relationships between different bands and extract useful features from them. It performs convolutional operations on the input image using three-dimensional convolutional kernels and adds non-linearity through activation functions (such as ReLU). As the network depth increases, the convolutional layer can gradually extract higher-level features. The pooling layer downsamples the feature maps, reduces the number of parameters, improves the computational efficiency, and increases the robustness of the model. Common pooling operations include max pooling and average pooling. In a three-dimensional convolutional neural network, three-dimensional pooling kernels can be used for downsampling. The fully connected layer (or global average pooling layer) flattens the feature maps extracted by the convolutional layer into a one-dimensional vector and outputs the final prediction result. The fully connected layer can connect all the features and map them to the output layer. In some cases, the global average pooling layer can also be used instead of the fully connected layer to reduce the number of parameters and improve the computational efficiency. The output layer outputs the prediction result of the presence of pests and diseases, that is, the second pests and diseases identification result.

[0087] After obtaining the second pests and diseases identification results corresponding to multiple local hyperspectral images, that is, obtaining the first pests and diseases identification results and the second pests and diseases identification results corresponding to multiple predicted pests and diseases regions, these two results are combined to perform pests and diseases early warning on the forest area with warning.

[0088] Specifically, for each predicted pests and diseases region, if both the first pests and diseases identification result and the second pests and diseases identification result indicate the presence of pests and diseases, a first-level warning is issued, and the location information corresponding to the target detection region is fed back; if the first pests and diseases identification result indicates the presence of pests and diseases risk, and the second pests and diseases identification result indicates the absence of pests and diseases risk, a second-level warning is issued, and the location information corresponding to the predicted pests and diseases region is fed back.

[0089] The solution provided by the present invention obtains the visible light image and the hyperspectral image of the forest area to be warned, and obtains the coordinate information of multiple ground control points in the forest area to be warned; inputs the visible light image into the first preset prediction network model to output the first pest and disease identification result, and the first pest and disease identification result is the visible light image marked with the predicted pest and disease area; for each predicted pest and disease area, determines the target point in the predicted pest and disease area, and determines the coordinate information of the target point based on the coordinate information of multiple ground control points; determines the image area of the target detection area based on the tree species in the forest area to be warned, and determines the corresponding local hyperspectral image of the target detection area in the hyperspectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points and the image area of the target detection area; inputs the local hyperspectral image into the second preset prediction network model to output the second pest and disease identification result, and warns of the pest and disease in the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas. This solution first uses the visible light image for preliminary detection, and then uses the local hyperspectral image for precise detection. On the one hand, there is no need to fuse the overall visible light image and hyperspectral image of the forest area to be warned, which reduces the complexity of the solution algorithm, greatly saves computing power, and ensures the timeliness of the warning. On the other hand, through the preliminary detection, the subsequent precise detection does not need to detect the overall hyperspectral image of the forest area to be warned, and at the same time, the image area of the local hyperspectral image for precise detection is also controlled, further reducing the complexity of the solution algorithm, greatly saving computing power, and ensuring the timeliness of the warning.

[0090] In an alternative embodiment of the present invention, the method further includes:

[0091] Obtaining the warning response duration required for the forest area to be warned, and determining the number of reference spectral feature parameters;

[0092] Inputting the local hyperspectral image samples of multiple detection areas corresponding to different image areas into the second preset prediction network model, processing the spectral feature parameters of the number of reference spectral feature parameters of the local hyperspectral image samples through the second preset network model, outputting the corresponding second pest and disease identification results, and recording the duration required for outputting the corresponding second pest and disease identification results;

[0093] Determining the image area of the detection area corresponding to the local hyperspectral image sample whose output required duration matches the warning response duration as the reference image area.

[0094] Among them, the early warning reaction duration can be set according to requirements. For example, it can be set according to different tree species. The multi-spectral image contains spectral information corresponding to multiple different bands (such as infrared band, ultraviolet band, etc.). In the embodiments of the present invention, the number of spectral feature parameters refers to the number of selected bands. Therefore, the number of reference spectral feature parameters can also refer to the number of spectral bands. The number of reference spectral feature parameters can be determined according to experience, providing a reference benchmark for the determination of the image area under the number of actually selected spectral feature parameters later.

[0095] Specifically, after determining the early warning reaction duration and the number of reference spectral feature parameters, the local multi-spectral image samples of multiple detection regions corresponding to different image areas are processed. In the embodiments of the present invention, the model will select the spectral features of the number of reference spectral feature parameters as the subsequent processing information through preprocessing, then record the duration required for the model output result, and then determine the reference image area through the matching of this duration and the early warning reaction duration. The image area of the local multi-spectral image sample corresponding to the duration within a preset range of the difference between this duration and the early warning reaction duration can be determined as the reference image area.

[0096] Determining the image area of the target detection region based on the tree species of the forest area to be warned includes:

[0097] Based on the current required early warning accuracy level of the forest area to be warned, determine the required number of target spectral feature parameters;

[0098] Based on the number of target spectral feature parameters, the number of reference spectral feature parameters, and the reference image area, determine the image area of the target detection region.

[0099] Specifically, when performing local hyperspectral image processing through the first preset prediction model, its prediction accuracy level (or the final early warning accuracy level) is positively correlated with the number of spectral feature parameters. Therefore, after determining the required early warning accuracy level, the required number of target spectral features, that is, the number of target bands, can be determined according to practical experience or other means. Then, the increase or decrease in the number of bands will affect the processing speed of the model. Therefore, in order to ensure that the early warning duration does not exceed the previously determined required early warning duration and at the same time ensure the required early warning accuracy level, the image area of the local multi-spectral image can be changed when the number of selected spectral features changes.

[0100] Further, determining the image area of the target detection region based on the number of target spectral feature parameters, the number of reference spectral features, and the reference image area includes:

[0101] If the number of target spectral feature parameters is equal to the number of reference spectral feature parameters, then determine the reference image area as the image area of the target detection region;

[0102] If the number of target spectral feature parameters is greater than the number of reference spectral feature parameters, the image area of the target detection region is obtained by reducing the reference image area by a specified step size.

[0103] If the number of target spectral feature parameters is less than the number of reference spectral feature parameters, the image area of the target detection region is obtained by increasing the reference image area by a specified step size.

[0104] Specifically, the reference image area and the number of reference spectral feature parameters are determined in the foregoing. Therefore, after the number of target spectral feature parameters is actually determined, the number of target spectral feature parameters can be compared with the number of reference spectral feature parameters. If the number of target spectral feature parameters is greater than the number of reference spectral feature parameters, using the reference image area will increase the early warning duration. Therefore, the image area of the target detection region can be obtained by reducing the reference image area by a specified step size, reducing the image area to be processed and the processing content, thereby ensuring that the early warning duration is not increased. Similarly, if the number of target spectral feature parameters is less than the number of reference spectral feature parameters, using the reference area will reduce the early warning duration. Therefore, the image area can be appropriately increased, that is, the image area of the target detection region can be obtained by increasing the reference image area by a specified step size, thereby ensuring that the early warning duration is not increased and the early warning accuracy can be further improved.

[0105] In an alternative embodiment of the present invention, the image area of the target detection region is determined based on the tree species in the forest area to be warned, and the local hyperspectral image corresponding to the target detection region in the hyperspectral image is determined based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection region, including:

[0106] Determine the first position of the target point in the hyperspectral image based on the target point and the coordinate information of the target point;

[0107] Based on the preset shape of the target detection region, the image area of the target detection region, and the first position, determine multiple second positions of the edge of the target detection region in the hyperspectral image;

[0108] Based on the first position and each second position, obtain the local hyperspectral image corresponding to the target detection region in the hyperspectral image.

[0109] Wherein, the shape of the target detection region is a square or a circle.

[0110] Specifically, when the target point, the coordinate information of the target point, the image area, and the shape of the target detection region are determined, the local hyperspectral image in the hyperspectral image can be determined according to the coordinate information of the ground control points.

[0111] In an alternative embodiment of the present invention, for each predicted pest and disease area, determining a target point in the predicted pest and disease area includes:

[0112] Obtain the tree density of the predicted pest and disease area, and assign a normalized tree density value to each pixel point in the predicted pest and disease area;

[0113] Based on a preset density clustering algorithm, obtain multiple pixel clustering clusters in the predicted pest and disease area, and determine the clustering center of the clustering cluster with the highest average density value as the target point.

[0114] Among them, the preset density clustering algorithm is the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm.

[0115] Figure 2 This is a structural block diagram of an early warning device for intelligently identifying forest pests and diseases based on visible light and multispectral provided by an embodiment of the present invention. The device may include:

[0116] The image acquisition module 201 is used to acquire a visible light image and a multispectral image of the forest area to be warned, and acquire the coordinate information of multiple ground control points in the forest area to be warned;

[0117] The first pest and disease identification result acquisition module 202 is used to input the visible light image into a first preset prediction network model and output a first pest and disease identification result. The first pest and disease identification result is a visible light image marked with the predicted pest and disease area; the first preset prediction network model is trained by a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label;

[0118] The target point acquisition module 203 is used to, for each predicted pest and disease area, determine a target point in the predicted pest and disease area, and determine the coordinate information of the target point based on the coordinate information of multiple ground control points;

[0119] The local multispectral image acquisition module 204 is used to determine the image area of the target detection area based on the tree species in the forest area to be warned, and determine the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area;

[0120] The second pest and disease identification result acquisition and early warning module 205 is used to input the local multispectral image into the second preset prediction network model, output the second pest and disease identification result, and perform pest and disease early warning on the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is obtained by training with forest multispectral image samples, and the forest multispectral image samples are marked with the second pest and disease identification results.

[0121] The solution provided by the present invention obtains the visible light image and multispectral image of the forest area to be warned, and obtains the coordinate information of multiple ground control points in the forest area to be warned; inputs the visible light image into the first preset prediction network model, and outputs the first pest and disease identification result, and the first pest and disease identification result is a visible light image marked with the predicted pest and disease area; for each predicted pest and disease area, determine the target point in the predicted pest and disease area, and determine the coordinate information of the target point based on the coordinate information of multiple ground control points; determine the image area of the target detection area based on the tree species of the forest area to be warned, and determine the local multispectral image corresponding to the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area; input the local multispectral image into the second preset prediction network model, output the second pest and disease identification result, and perform pest and disease early warning on the forest area to be warned according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas. This solution first uses the visible light image for preliminary detection, and then uses the local hyperspectral image for precise detection. On the one hand, there is no need to fuse the overall visible light image and hyperspectral image of the forest area to be warned, which reduces the complexity of the solution algorithm, greatly saves computing power, and ensures the timeliness of early warning. On the other hand, through preliminary detection, subsequent precise detection does not need to detect the overall hyperspectral image of the forest area to be warned, and at the same time controls the image area of the local hyperspectral image for precise detection, further reducing the complexity of the solution algorithm, greatly saving computing power, and ensuring the timeliness of early warning.

[0122] In an alternative embodiment of the present invention, the device further includes a reference image area determination module, which is used for:

[0123] Obtain the early warning response duration required for the forest area to be warned, and determine the number of reference spectral feature parameters;

[0124] Input the local multispectral image samples of multiple detection areas corresponding to different image areas into the second preset prediction network model, process the spectral feature parameters of the number of reference spectral feature parameters of the local multispectral image samples through the second preset network model, output the corresponding second pest and disease identification results, and record the duration required for outputting the corresponding second pest and disease identification results;

[0125] Determine the image area of the corresponding detection region of the local multi-spectral image sample that matches the required duration of the output and the early warning response duration as the reference image area;

[0126] Determine the image area of the target detection region based on the tree species in the forest area to be warned, including:

[0127] The local multi-spectral image acquisition module is specifically used for:

[0128] Determine the image area of the target detection region based on the number of target spectral feature parameters, the number of reference spectral feature parameters, and the reference image area.

[0129] In an alternative embodiment of the present invention, the local multi-spectral image acquisition module is further used for:

[0130] If the number of target spectral feature parameters is equal to the number of reference spectral feature parameters, determine the reference image area as the image area of the target detection region;

[0131] If the number of target spectral feature parameters is greater than the number of reference spectral feature parameters, obtain the image area of the target detection region after reducing the reference image area by a specified step size;

[0132] If the number of target spectral feature parameters is less than the number of reference spectral feature parameters, obtain the image area of the target detection region after increasing the reference image area by a specified step size.

[0133] In an alternative embodiment of the present invention, the local multi-spectral image acquisition module is further used for:

[0134] Determine the first position of the target point in the multi-spectral image based on the target point and the coordinate information of the target point;

[0135] Determine multiple second positions of the edge of the target detection region in the multi-spectral image based on the preset shape of the target detection region, the image area of the target detection region, and the first position;

[0136] Obtain the local multi-spectral image corresponding to the target detection region in the multi-spectral image based on the first position and each second position.

[0137] In an alternative embodiment of the present invention, the shape of the target detection region is a square or a circle.

[0138] In an alternative embodiment of the present invention, the target point acquisition module is specifically used for:

[0139] Obtain the tree density of the predicted pest and disease area, and assign a normalized tree density value to each pixel point in the predicted pest and disease area;

[0140] Based on a preset density clustering algorithm, multiple pixel clustering clusters of the predicted pest and disease area are obtained, and the clustering center of the clustering cluster with the highest average density value is determined as the target point.

[0141] In an alternative embodiment of the present invention, the second pest and disease identification result acquisition and warning module is specifically configured to:

[0142] For each predicted pest and disease area, if both the first pest and disease identification result and the second pest and disease identification result indicate the presence of pests and diseases, a first-level warning is issued, and the position information corresponding to the target detection area is fed back;

[0143] If the first pest and disease identification result indicates the presence of a pest and disease risk, and the second pest and disease identification result indicates the absence of a pest and disease risk, a second-level warning is issued, and the position information corresponding to the predicted pest and disease area is fed back.

[0144] Figure 3 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 3 shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340. Among them, the processor 310, the communication interface 320, and the memory 330 complete mutual communication through the communication bus 340. The processor 310 can call the logical instructions in the memory 330 to execute a warning method for intelligently identifying forest pests and diseases based on visible light and multispectral images. The method includes: obtaining a visible light image and a multispectral image of the forest area to be warned, and obtaining the coordinate information of multiple ground control points in the forest area to be warned; inputting the visible light image into a first preset prediction network model to output a first pest and disease identification result, and the first pest and disease identification result is a visible light image marked with the predicted pest and disease area; the first preset prediction network model is trained through a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label; for each predicted pest and disease area, determining the target point in the predicted pest and disease area, and determining the coordinate information of the target point based on the coordinate information of multiple ground control points; determining the image area of the target detection area based on the tree species of the forest area to be warned, and determining the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area; inputting the local multispectral image into a second preset prediction network model to output a second pest and disease identification result, and warning the forest area to be warned of pests and diseases according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is trained through a forest multispectral image sample, and the forest multispectral image sample is marked with a second pest and disease identification result.

[0145] In addition, when the logical instructions in the above-mentioned memory 330 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0146] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the warning method for forest tree pests and diseases based on visible light and multispectral intelligent identification provided by the above-mentioned various methods. The method includes: acquiring a visible light image and a multispectral image of a forest tree area to be warned, and acquiring coordinate information of multiple ground control points in the forest tree area to be warned; inputting the visible light image into a first preset prediction network model to output a first pest and disease identification result, where the first pest and disease identification result is a visible light image marked with a predicted pest and disease area; the first preset prediction network model is trained through forest tree visible light image samples, and the forest tree visible light image samples are marked with first pest and disease identification result labels; for each predicted pest and disease area, determining a target point in the predicted pest and disease area, and determining the coordinate information of the target point based on the coordinate information of multiple ground control points; determining the image area of the target detection area based on the tree species of the forest tree area to be warned, and determining the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area; inputting the local multispectral image into a second preset prediction network model to output a second pest and disease identification result, and warning the forest tree area to be warned of pests and diseases according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is trained through forest tree multispectral image samples, and the forest tree multispectral image samples are marked with second pest and disease identification results.

[0147] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a warning method for intelligent identification of forest pests and diseases based on visible light and multispectral, and the method includes: obtaining a visible light image and a multispectral image of a forest area to be warned, and obtaining coordinate information of multiple ground control points in the forest area to be warned; inputting the visible light image into a first preset prediction network model to output a first pest and disease identification result, and the first pest and disease identification result is a visible light image marked with a predicted pest and disease area; the first preset prediction network model is trained through a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label; for each predicted pest and disease area, determining a target point in the predicted pest and disease area, and determining the coordinate information of the target point based on the coordinate information of multiple ground control points; determining the image area of the target detection area based on the forest species of the forest area to be warned, and determining the corresponding local multispectral image of the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of multiple ground control points, and the image area of the target detection area; inputting the local multispectral image into a second preset prediction network model to output a second pest and disease identification result, and warning the forest area to be warned of pests and diseases according to the second pest and disease identification results of the target detection areas corresponding to multiple predicted pest and disease areas; the second prediction network model is trained through a forest multispectral image sample, and the forest multispectral image sample is marked with a second pest and disease identification result.

[0148] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0149] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An early warning method for intelligently identifying forest pests and diseases based on visible light and multi-spectrum, characterized in that: include: Obtaining visible light images and multispectral images of the forest area to be warned, and obtaining coordinate information of multiple ground control points in the forest area to be warned; Input the visible light image into a first preset prediction network model, and output a first pest and disease identification result, wherein the first pest and disease identification result is a visible light image marked with a predicted pest and disease area; the first preset prediction network model is obtained by training a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label; For each predicted pest and disease area, determining a target point in the predicted pest and disease area, and determining coordinate information of the target point based on the coordinate information of the plurality of ground control points; Determine the image area of ​​the target detection area based on the tree type of the to-be-warned forest area, and determine the local multispectral image corresponding to the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of the plurality of ground control points and the image area of ​​the target detection area; Input the local multispectral image into a second preset prediction network model, output a second pest and disease identification result, and issue a pest and disease warning to the forest area to be warned based on the second pest and disease identification results of the target detection area corresponding to the multiple predicted pest and disease areas; The second prediction network model is obtained by training with forest multispectral image samples, and the forest multispectral image samples are marked with second pest and disease identification results.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the warning response time required for the forest area to be warned, and determining the number of reference spectral characteristic parameters; Input a plurality of local multispectral image samples corresponding to detection areas of different image areas into the second preset prediction network model, process the spectral characteristic parameters of the number of reference spectral characteristic parameters of the local multispectral image samples through the second preset network model, output the corresponding second pest and disease identification result, and record the time required to output the corresponding second pest and disease identification result; Determine the image area of ​​the detection area corresponding to the local multispectral image sample whose output required time length matches the warning reaction time length as the reference image area; The determining of the image area of ​​the target detection area based on the tree types in the forest area to be warned includes: Based on the current warning accuracy level required for the forest area to be warned, determining the required number of target spectral characteristic parameters; The image area of ​​the target detection region is determined based on the target spectrum feature parameter quantity, the reference spectrum feature parameter quantity and the reference image area.

3. The method according to claim 2, characterized in that The determining the image area of ​​the target detection area based on the target spectral feature parameter quantity, the reference spectral feature quantity and the reference image area comprises: If the number of characteristic parameters of the target spectrum is equal to the number of characteristic parameters of the reference spectrum, determining the reference image area as the image area of ​​the target detection area; If the number of characteristic parameters of the target spectrum is greater than the number of characteristic parameters of the reference spectrum, the image area of ​​the target detection area is obtained by reducing the reference image area according to a specified step size; If the number of characteristic parameters of the target spectrum is less than the number of characteristic parameters of the reference spectrum, the image area of ​​the target detection region is obtained by increasing the reference image area according to a specified step size.

4. The method according to claim 1, characterized in that: The method of determining the image area of ​​the target detection area based on the tree type of the to-be-warned forest area, and determining the local multispectral image corresponding to the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of the plurality of ground control points, and the image area of ​​the target detection area, includes: Determine a first position of the target point in the multispectral image based on the target point and the coordinate information of the target point; Determine a plurality of second positions of the edge of the target detection area in the multispectral image based on a preset target detection area shape, an image area of ​​the target detection area, and the first position; Based on the first position and each of the second positions, a local multispectral image corresponding to the target detection area in the multispectral image is acquired.

5. The method according to claim 4, characterized in that The target detection area is in the shape of a square or a circle.

6. The method according to claim 1, characterized in that For each predicted pest and disease area, determining a target point in the predicted pest and disease area includes: Obtaining the tree density of the predicted pest and disease area, and assigning a normalized tree density value to each pixel point in the predicted pest and disease area; Based on a preset density clustering algorithm, multiple pixel clusters of the predicted pest and disease area are obtained, and the cluster center of the cluster with the highest average density value is determined as the target point.

7. The method according to claim 1, characterized in that The step of providing an early warning of pests and diseases in the forest area to be warned based on the second pest and disease identification results of the target detection area corresponding to the plurality of predicted pest and disease areas comprises: For each predicted pest and disease area, if both the first pest and disease identification result and the second pest and disease identification result indicate the presence of pests and diseases, a first-level warning is issued, and the location information corresponding to the target detection area is fed back; If the first pest and disease identification result indicates that there is a pest and disease risk, and the second pest and disease identification result indicates that there is no pest and disease risk, a secondary warning is issued, and the location information corresponding to the predicted pest and disease area is fed back.

8. An early warning device for intelligently identifying forest pests and diseases based on visible light and multi-spectrum, characterized in that: include: An image acquisition module is used to acquire visible light images and multispectral images of the forest area to be warned, and to acquire coordinate information of multiple ground control points in the forest area to be warned; A first pest and disease identification result acquisition module is used to input the visible light image into a first preset prediction network model, and output a first pest and disease identification result, wherein the first pest and disease identification result is a visible light image marked with a predicted pest and disease area; the first preset prediction network model is obtained by training a forest visible light image sample, and the forest visible light image sample is marked with a first pest and disease identification result label; A target point acquisition module, for determining, for each predicted pest and disease area, a target point in the predicted pest and disease area, and determining coordinate information of the target point based on coordinate information of the plurality of ground control points; A local multispectral image acquisition module is used to determine the image area of ​​the target detection area based on the tree species in the forest area to be warned, and determine the local multispectral image corresponding to the target detection area in the multispectral image based on the target point, the coordinate information of the target point, the coordinate information of the multiple ground control points and the image area of ​​the target detection area; A second pest and disease identification result acquisition and early warning module is used to input the local multispectral image into a second preset prediction network model, output a second pest and disease identification result, and issue an early warning of pests and diseases to the forest area to be warned based on the second pest and disease identification results of the target detection area corresponding to the multiple predicted pest and disease areas; The second prediction network model is obtained by training with forest multispectral image samples, and the forest multispectral image samples are marked with second pest and disease identification results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images