Intelligent monitoring method of forest pests based on UAV remote sensing

Abstract

The present invention relates to the field of pest monitoring technology, and specifically to an intelligent forest pest monitoring method based on unmanned aerial vehicle remote sensing. Hyperspectral data and multiple forest images of each measuring point in the forest area to be measured at different shooting positions are obtained. The difference in spectral curve changes and wavelength absorbance of the hyperspectral data are used to determine the pest characteristic value. Considering that the scope of the pest area is gradually expanding, adjacent measuring points should have similar characteristics, so the mutation of the pest characteristic value between the measuring points at the same shooting position is analyzed to obtain the pest mutation coefficient to characterize the jitter effect of the unmanned aerial vehicle. The number and position characteristics of edge lines and virtual shadow lines in the forest image are analyzed to determine the shooting jitter characteristic value. Combined with the pest mutation coefficient and the shooting jitter characteristic value, the reference value of the pest characteristic value of each measuring point is quantified, and the shooting positions that are less affected by the jitter of the unmanned aerial vehicle are screened. Finally, the pest characteristic value at the screened shooting position is monitored to improve accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of pest monitoring, and in particular to an intelligent forest pest monitoring method based on unmanned aerial vehicle remote sensing. Background Art

[0002] As one of the most important ecosystems on Earth, forests play a vital role in maintaining ecological balance, protecting biodiversity and promoting climate stability. However, the occurrence of forest pests will cause a series of abnormal changes in the morphology and organization of trees, leading to poor growth and development of trees, and even causing the death of trees and the deterioration of the ecological environment. Drones, with their high efficiency, flexibility and wide coverage, can obtain a large amount of hyperspectral data from forest areas in a short period of time. In recent years, with the rapid development of remote sensing technology, drone remote sensing technology has gradually shown great potential in the field of forest monitoring.

[0003] When using drones to monitor forest pests, existing technologies usually analyze the hyperspectral data of forest areas collected by drones in real time to quantify the pest situation in the forest area. However, during the shooting process, the drone may shake due to factors such as flight posture and wind speed, which may cause a deviation between the actual monitoring position and the expected monitoring position. Therefore, if pest monitoring is performed directly based on the hyperspectral data of the forest area collected by the drone, the accuracy of the final pest judgment result will be reduced. Summary of the invention

[0004] In order to solve the technical problem that the drone may jitter during shooting due to the influence of factors such as flight attitude and wind speed, which may cause the actual monitoring position to deviate from the expected monitoring position, if the insect pest monitoring is directly performed based on the hyperspectral data of the forest area collected by the drone, the accuracy of the final insect pest judgment result will be reduced, the purpose of the present invention is to provide an intelligent forest pest monitoring method based on drone remote sensing, and the technical scheme adopted is as follows:

[0005] In the forest area to be measured, hyperspectral data and multiple forest images are obtained at each measuring point at each shooting position;

[0006] At each shooting position, analyze the differences in the spectral curves between the hyperspectral data of the measuring points, as well as the absorbance corresponding to the wavelength in the spectral curve of the measuring point, to determine the pest characteristic value at each measuring point;

[0007] The differences in pest characteristic values ​​at different measuring points under the same shooting position are analyzed to obtain the pest mutation coefficient at each measuring point; in each forest image at each measuring point, the edge line and the virtual shadow line corresponding to each edge line are obtained; for any measuring point, based on the quantity characteristics of the edge lines and virtual shadow lines in all forest images of the measuring point under different shooting positions, as well as the position characteristics between the edge lines and virtual shadow lines, the shooting jitter characteristic value of the measuring point at each shooting position is determined;

[0008] The reference value of each measuring point at each shooting position is obtained by combining the pest mutation coefficient and shooting jitter characteristic value of each measuring point at each shooting position. The shooting positions are screened based on the reference values ​​of each measuring point at different shooting positions, and the pests in the forest area to be tested are monitored based on the pest characteristic values ​​of each measuring point at the screened shooting positions.

[0009] Furthermore, the method for obtaining the pest characteristic value includes:

[0010] Based on the differences between the spectral curves corresponding to the hyperspectral data of each measuring point at the same shooting position, the first pest factor at each measuring point is determined;

[0011] At the same shooting position, the maximum value in the spectrum curve at each measuring point is obtained, and the sum of all the maximum values ​​is taken as the second pest factor at each measuring point;

[0012] The sum of the first pest factor and the second pest factor of each measuring point at the same shooting position is normalized to obtain the pest characteristic value of each measuring point at each shooting position.

[0013] Furthermore, the method for obtaining the first pest factor includes:

[0014] At each shooting position, the area enclosed by the spectral curve corresponding to the hyperspectral data of each measuring point and the coordinate axis is taken as the integral eigenvalue;

[0015] The ratio of the integral eigenvalue corresponding to each measuring point to the maximum integral eigenvalue of all measuring points is taken as the first pest factor at each measuring point.

[0016] Furthermore, the method for obtaining the pest mutation coefficient includes:

[0017] In the preset neighborhood of any measuring point, at each shooting position, the absolute value of the difference between the pest characteristic value of the measuring point and each neighboring measuring point is calculated as the pest deviation factor, and the sum of the pest deviation factors between the measuring point and all neighboring measuring points is used as the mutation factor of the measuring point;

[0018] The normalized value of the ratio of the mutation factor of each measuring point to the maximum mutation factor of all measuring points is used as the pest mutation coefficient of each measuring point at each shooting position.

[0019] Furthermore, obtaining edge lines and virtual shadow lines corresponding to each edge line in each forest image at each measuring point includes:

[0020] In each forest image at each measuring point, all edge lines in the forest image are obtained based on the Sobel operator;

[0021] Taking each edge line as a template, template matching is performed in the forest image to which the edge line belongs, so as to obtain the virtual shadow line corresponding to each edge line and the matching point pairs between each edge line and each corresponding virtual shadow line.

[0022] Furthermore, the method for obtaining the shooting jitter characteristic value includes:

[0023] At each shooting position, based on the quantity characteristics of edge lines and virtual shadow lines in each forest image of each measuring point, the blur degree value of each forest image is determined;

[0024] Based on the positional features between the edge line and the virtual shadow line in each forest image at each measuring point, the shaking direction of each forest image is determined;

[0025] At each shooting position, the blur degree value of each forest image at each measuring point is used as the modulus, and the jitter direction corresponding to each forest image at each measuring point is used as the direction, so as to obtain the jitter vector corresponding to each forest image at each measuring point, and the sum value of the jitter vectors corresponding to all forest images at each measuring point is used as the sum vector;

[0026] The ratio of the modulus length of the sum vector corresponding to each measuring point at each shooting position to the maximum modulus length of the sum vector corresponding to each measuring point at all shooting positions is used as the first jitter factor of each measuring point at each shooting position;

[0027] At each shooting position, for any measuring point, calculate the angle between the jitter vectors corresponding to any two forest images at the measuring point;

[0028] Calculate the cosine similarity between the jitter vector of the measuring point in each forest image and the sum vector as a similarity value, and use the number of jitter vectors with similarity values ​​greater than a preset similarity threshold as a quantity factor;

[0029] Obtaining a second jitter factor of the measuring point at each shooting position according to the maximum angle value corresponding to the measuring point and the quantity factor, wherein the second jitter factor is positively correlated with the maximum first angle value, and the second jitter factor is negatively correlated with the quantity factor;

[0030] A value obtained by normalizing the sum of the first jitter factor and the second jitter factor is used as a shooting jitter characteristic value of the measuring point at each shooting position.

[0031] Furthermore, the method for obtaining the blur degree value includes:

[0032] At each shooting position, for any measuring point, the average value of the number of virtual shadow lines corresponding to all edge lines in each forest image of the measuring point is normalized and used as the blur degree value of each forest image of the measuring point.

[0033] Furthermore, the method for obtaining the shaking direction includes:

[0034] At each shooting position, in each forest image of each measuring point, for any edge line, the matching point on the edge line is taken as the starting point, and the matching point corresponding to each virtual shadow line corresponding to the edge line is taken as the end point, and all feature vectors are obtained, and the angle value between each feature vector and the preset direction is obtained, and the angle value that appears most frequently is taken as the jitter direction corresponding to each forest image of each measuring point.

[0035] Furthermore, the method for obtaining the reference value includes:

[0036] At each shooting position, for any measuring point, the value of the pest mutation coefficient at the measuring point after negative correlation mapping is used as the first reference factor;

[0037] The value after negative correlation mapping of the shooting jitter characteristic value corresponding to the measuring point is used as the second reference factor;

[0038] The value obtained by normalizing the product of the first reference factor and the second reference factor is used as the reference value of the measuring point at each shooting position.

[0039] Further, the shooting positions are screened based on the reference values ​​of each measuring point at different shooting positions, and the insect pests in the forest area to be tested are monitored based on the insect pest characteristic values ​​of each measuring point at the screened shooting positions, including:

[0040] For any measuring point, the shooting position corresponding to the maximum value of the reference value of the measuring point at all shooting positions is taken as the target position, and the pest characteristic value of the measuring point at the target position is taken as the final pest degree value of the measuring point;

[0041] In the forest area to be tested, when the average of the final pest severity values ​​of all measuring points is greater than the preset pest threshold, the pest severity is considered to be high and an early warning is required; when the average of the final pest severity values ​​of all measuring points is less than or equal to the preset pest threshold, the pest severity is considered to be low and no early warning is required.

[0042] The present invention has the following beneficial effects:

[0043] The present invention can comprehensively reflect the physiological and ecological conditions and pest conditions of the forest by acquiring hyperspectral data and multiple forest images at each shooting position of each measuring point in the forest area to be measured, thereby improving the richness of the data. When forest trees are infested with insects, elements in leaves or branches will be missing, and the pest conditions can be characterized by the change in the absorbance of the wavelength in the spectral curve of the hyperspectral data. Therefore, the difference in the spectral curves between the hyperspectral data of the measuring points and the absorbance corresponding to the wavelength are analyzed to determine the pest characteristic value at each measuring point. Because the scope of the pest area is gradually increasing, adjacent measuring points should have relatively similar performance characteristics. If the pest characteristic value at the measuring point has a relatively obvious mutation at the same shooting position, it can be regarded as that the UAV is affected by factors such as flight posture and wind speed during the shooting process, resulting in jitter and data deviation. Furthermore, if the UAV jitters, blur or phantom may appear in the acquired image data. Therefore, in each forest image of each measuring point, the edge line and the phantom line corresponding to each edge line are obtained. In this way, the shooting jitter characteristic value of each measuring point at each shooting position can be determined based on the quantity characteristics and position characteristics of the edge lines and phantom lines in all forest images of each measuring point at different shooting positions. Both the pest mutation coefficient and the shooting jitter characteristic value can measure the degree of jitter of the drone at each shooting position. Therefore, by combining these two indicators, the reference value of the pest characteristic value of each measuring point at each shooting position can be quantified, and the shooting positions can be further screened based on the reference value. At this time, the shooting positions screened out by each measuring point can be regarded as positions that are less affected by the jitter of the drone. Therefore, the pests in the forest area to be tested can be monitored based on the pest characteristic values ​​of the measuring points at the screened shooting positions, and more accurate monitoring results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A method flow chart of a method for intelligent monitoring of forest pests based on UAV remote sensing provided by one embodiment of the present invention;

[0046] Figure 2 A method flow chart of a method for obtaining pest characteristic values ​​provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of an edge line and a virtual shadow line provided by an embodiment of the present invention;

[0048] Figure 4 A method flow chart of a method for obtaining a shooting jitter characteristic value provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of an intelligent forest pest monitoring method based on drone remote sensing proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0051] The following is a detailed description of a specific scheme of an intelligent forest pest monitoring method based on UAV remote sensing provided by the present invention in conjunction with the accompanying drawings.

[0052] See also Figure 1 , which shows a method flow chart of a method for intelligent monitoring of forest pests based on drone remote sensing provided by an embodiment of the present invention, the method comprising the following steps:

[0053] Step S1: in the forest area to be measured, hyperspectral data and multiple forest images are obtained at each measuring point at each shooting position.

[0054] As one of the most important ecosystems on Earth, forests play a vital role in maintaining ecological balance, protecting biodiversity, and promoting climate stability. However, forest pests have long been one of the main factors threatening forest health. As drone remote sensing technology has gradually shown great potential in the field of forest monitoring, drones can be used to capture image data in the forest area to be tested for analysis of pest conditions.

[0055] During the operation of the drone, it is easily affected by factors such as flight attitude and wind speed, and may jitter, which may cause a deviation between the actual monitoring position and the expected monitoring position, thereby resulting in a problem of low accuracy of the final pest monitoring result. Therefore, in an embodiment of the present invention, multiple shooting positions are set, and the hyperspectral imaging device carried by the drone is used to obtain hyperspectral data at each measuring point in the forest area to be tested at multiple shooting positions. The hyperspectral data contains spatial information and spectral information. The spectral information can reflect the absorbance of leaves or branches at each measuring point at different wavelengths (the horizontal axis is wavelength, and the vertical axis is absorbance), indicating the element situation in the leaves or branches at each measuring point, and then characterizing the pest characteristics; at the same time, since the jitter of the drone will cause blur or phantom in the image, the camera function of the drone is used while obtaining the hyperspectral data to obtain multiple forest images of each measuring point at each shooting position, wherein the forest images are continuous in time sequence, and the forest images are processed grayscale images.

[0056] It should be noted that, in the embodiment of the present invention, the shooting position is set to: the height is 4m, 6m and 10m from the ground respectively, at each height, a circle is drawn with the center of the forest area to be measured as the center and a radius of 3m, and on the circle at each height, 5 hovering positions are evenly set, thereby obtaining 15 hovering positions as the shooting positions of the drone; the acquisition interval of the forest image can be set to 2ms; the setting of the specific shooting position and the acquisition interval of the forest image can be adjusted according to the implementation scenario, and are not limited here, but it is necessary to ensure that the acquisition interval of the forest image is small, so that the acquisition period of multiple forest images can be approximately overlapped with the acquisition period of the hyperspectral data, thereby improving the accuracy of the subsequent jitter analysis; the grayscale image acquisition method can adopt average grayscale, maximum value method or weighted grayscale and other methods, and they are all technical means well known to those skilled in the art, and are not limited or elaborated here.

[0057] Step S2: At each shooting position, analyze the difference in spectral curves between the hyperspectral data of the measuring points and the absorbance corresponding to the wavelength in the spectral curve of the measuring point to determine the pest characteristic value at each measuring point.

[0058] Hyperspectral data contains rich spectral information, which can reflect the absorption, reflection and transmission characteristics of substances at different wavelengths. For forest pest monitoring, this information can reveal subtle changes in the chemical composition of leaves or branches. When insect spots appear at a certain measuring point, compared with other normal measuring points, the content of elements such as Fe, Mg, and Si contained in it will be significantly reduced due to disease. In the hyperspectral data, the peak value of the wavelength corresponding to the elements such as Fe, Mg, and Si will be higher (the lower the element content, the less light is absorbed, and the more light is reflected, so the intensity of the reflected light absorbed by the hyperspectral imager will be higher, so the peak value of the corresponding wavelength will be higher). Therefore, in the hyperspectral data, the pest characteristic value of each measuring point at each shooting position can be quantified based on the difference in the spectral curves between the measuring points and the absorbance corresponding to the wavelength in the spectral curve of the measuring point.

[0059] Preferably, in one embodiment of the present invention, the method for obtaining the pest characteristic value includes:

[0060] See also Figure 2 , which shows a method flow chart of a method for obtaining pest characteristic values ​​in one embodiment of the present invention, the method comprising the following steps:

[0061] Step S201: determining a first insect pest factor at each measuring point based on differences between spectral curves corresponding to hyperspectral data of each measuring point at the same shooting position.

[0062] Based on the above analysis, it can be seen that the absorbance of the wavelength in the hyperspectral data at the measuring point can reflect the lack of elements in the leaves or branches of the forest at the measuring point, and the more elements are missing, the greater the absorbance.

[0063] Therefore, at each shooting position, the area enclosed by the spectral curve corresponding to the hyperspectral data of each measuring point and the coordinate axis is taken as the integral eigenvalue. The integral eigenvalue quantifies the overall shape of the spectral curve. At this time, the larger the integral eigenvalue, the greater the possibility that the forest at the measuring point is affected by insect pests.

[0064] Then the maximum value of the integrated eigenvalues ​​of all measuring points is used as the denominator, the integrated eigenvalue corresponding to each measuring point is used as the numerator, and the obtained ratio is used as the first pest factor. The first pest factor can measure the relative degree of forest pests at each measuring point, and the larger the value, the greater the degree of forest pests at the measuring point.

[0065] Step S202: Under the same shooting position, the second pest factor at each measuring point is determined based on the absorbance corresponding to the wavelength in the spectrum curve at each measuring point.

[0066] Insect pests will cause the absorbance of elements at corresponding wavelengths in the spectral curve to increase. Therefore, at the same shooting position, the maximum value in the spectral curve at each measuring point is obtained. Each maximum value represents the absorption peak corresponding to an element. Finally, the sum of all the maximum values ​​is taken as the second insect pest factor at each measuring point. The larger the second insect pest factor, the more serious the insect pest on the forest at the measuring point.

[0067] Step S203: The first pest factor and the second pest factor of each measuring point at each shooting position are integrated to obtain the pest characteristic value of each measuring point at each shooting position.

[0068] Based on the analysis in the above steps, it can be known that the larger the first pest factor at each measuring point, the higher the possibility that the forest at the measuring point is infested with pests; the larger the second pest factor at each measuring point, the more serious the pests suffered by the forest at the measuring point, so the first pest factor and the second pest factor at each measuring point are positively correlated with the pest characteristics at the measuring point. In this embodiment of the present invention, the sum of the first pest factor and the second pest factor of each measuring point at the same shooting position is normalized to obtain the pest characteristic value of each measuring point at each shooting position. At this time, the larger the pest characteristic value, the greater the degree of pests suffered by the forest at the measuring point. Normalization is a technical means well known to those skilled in the art, and the selection of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0069] Step S3: Analyze the differences in pest characteristic values ​​at different measuring points at the same shooting position to obtain the pest mutation coefficient at each measuring point; obtain the edge line and the virtual shadow line corresponding to each edge line in each forest image at each measuring point; for any measuring point, determine the shooting jitter characteristic value of the measuring point at each shooting position based on the quantity characteristics of the edge lines and virtual shadow lines in all forest images of the measuring point at different shooting positions, as well as the position characteristics between the edge lines and the virtual shadow lines.

[0070] Pest spots on leaves or branches in the forest usually have a tendency to expand gradually, and they themselves have a certain range. Therefore, if the pest characteristic value at a certain measuring point at a certain shooting position is greatly different from that at an adjacent position, then it can be regarded that the monitoring result of the pest characteristic value of the measuring point obtained at the shooting position does not conform to the generation law of pest spots, and thus the pest monitoring result at the measuring point at this position may be affected by the jitter of the drone; at the same time, when the drone jitters, the forest image taken by the drone will have a virtual image to a great extent, resulting in blurred images. Therefore, in an embodiment of the present invention, the forest image at each measuring point at each shooting position can be analyzed to obtain the edge lines and the corresponding virtual shadow lines therein, and then analyze the quantity characteristics and position characteristics of the edge lines and the virtual shadow lines, so that the shooting jitter characteristic value of each measuring point at each shooting position can be determined, and the value measures the influence of the jitter of the drone at each shooting position on the pest monitoring of the measuring point.

[0071] Firstly, the differences between the characteristic values ​​of pests at different measuring points under the same shooting position were analyzed to obtain the pest mutation coefficient at each measuring point as an indicator to characterize the jitter of the UAV.

[0072] Preferably, in one embodiment of the present invention, the method for obtaining the pest mutation coefficient includes:

[0073] A preset neighborhood of each measuring point is determined, and in the preset neighborhood of each measuring point, other measuring points except itself are taken as neighboring measuring points.

[0074] Then, within the preset neighborhood of any measuring point, at each shooting position, the absolute value of the difference between the pest characteristic value of the measuring point and each neighboring measuring point is calculated as the pest deviation factor. The larger the pest deviation factor, the greater the difference in the pest conditions of adjacent measuring points at the same shooting position, and the more abrupt the measuring point is. Thus, the pest deviation factor between each measuring point and each corresponding neighboring measuring point can be obtained. The sum of the pest deviation factors between the measuring point and all the neighboring measuring points can be used as the mutation factor of the measuring point. The larger the mutation factor, the more prominent the pest condition at the measuring point is, and the less it conforms to the normal pest characteristic law. Therefore, it can be regarded that the calculation of the pest characteristic value at the measuring point is affected by the jitter of the drone.

[0075] Finally, the maximum value of the mutation factors of all measuring points is used as the denominator, and the mutation factor corresponding to each measuring point is used as the numerator. The obtained ratio is normalized and the value is used as the pest mutation coefficient of each measuring point at each shooting position. The pest mutation coefficient can measure the relative degree of pest mutation at each measuring point, and the larger the value, the greater the degree of mutation of the pest situation in the forest at the measuring point, which can be regarded as the greater impact of the drone jitter. Normalization is a technical means well known to those skilled in the art, and the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0076] It should be noted that, in this embodiment of the present invention, the preset neighborhood is set to construct a 3×3 neighborhood block with each measuring point as the center, and the specific size can be adjusted according to the implementation scenario and is not limited here.

[0077] If the drone shakes during shooting, the resulting forest image will be blurred. The blur of the image can be characterized by the edge lines and virtual shadow lines in the forest image. Then, the shooting jitter characteristic value of the measuring point at each shooting position is quantified according to the positional characteristics and quantity characteristics between the edge lines and virtual shadow lines, which is used to characterize the jitter of the drone.

[0078] Preferably, in one embodiment of the present invention, in each forest image at each measuring point, obtaining edge lines and virtual shadow lines corresponding to each edge line includes:

[0079] The edge is where the grayscale value in the image changes significantly, usually corresponding to the outline of an object or the boundary of different areas. In a forest image, the edge line may represent the outline of a tree or leaf, etc. When the drone shakes, the outline of the tree or leaf will become blurred. Therefore, first, in each forest image at each measuring point, all edge lines in the forest image are obtained based on the Sobel operator.

[0080] Then, using each edge line as a template, template matching is performed in the forest image to which the edge line belongs, so as to obtain the virtual shadow line corresponding to each edge line, wherein one edge line may correspond to multiple virtual shadow lines; at the same time, when performing template matching, the matching point pairs between each edge line and each corresponding virtual shadow line can be obtained, and the matching point pairs can be prepared for the subsequent calculation of the shooting jitter feature value. Figure 3 , which shows a schematic diagram of edge lines and virtual shadow lines in one embodiment of the present invention.

[0081] It should be noted that obtaining edge lines using the Sobel operator, template matching, and obtaining matching point pairs are all well-known technologies, and the specific processes are not described in detail here.

[0082] At this point, edge lines and virtual shadow lines corresponding to the edge lines can be obtained in each forest image.

[0083] The quantitative characteristics of edge lines and virtual shadow lines can reflect the degree of blurring in an image due to jitter: when an image jitters, the originally clear edge lines may become blurred and produce virtual shadow lines. Therefore, by comparing the number of edge lines and virtual shadow lines, the degree of jitter can be indirectly evaluated. Under the influence of jitter, there may be an obvious position offset between the edge lines and the virtual shadow lines. This offset can be quantified as part of the jitter characteristic value, thereby more accurately describing the degree of jitter of the drone. Therefore, in an embodiment of the present invention, for any measuring point, the shooting jitter characteristic value of the measuring point at each shooting position is determined based on the quantitative characteristics and positional characteristics of the edge lines and virtual shadow lines in all forest images of the measuring point at different shooting positions.

[0084] Preferably, in one embodiment of the present invention, the method for obtaining the shooting jitter characteristic value includes:

[0085] See also Figure 4 , which shows a method flow chart of a method for obtaining a shooting jitter characteristic value in one embodiment of the present invention, the method comprising the following steps:

[0086] Step S301: at each shooting position, based on the quantity characteristics of edge lines and virtual shadow lines in each forest image at each measuring point, determine the blur level value of each forest image.

[0087] Each edge line may correspond to multiple virtual shadow lines, and the more virtual shadow lines there are, the more blurred the forest image is. Therefore, at each shooting position, for any measuring point, the average value of the number of virtual shadow lines corresponding to all edge lines in each forest image of the measuring point is normalized as the blur degree value of each forest image of the measuring point. The larger the blur degree value, the lower the clarity of the object in the forest image, that is, the higher the blur degree. Normalization is a technical means well known to those skilled in the art, and the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0088] Step S302: determining the shaking direction of each forest image based on the positional features between the edge line and the virtual shadow line in each forest image at each measuring point.

[0089] The virtual shadow lines in the forest image are the "trajectory" or "afterimage" of the edge lines during the jittering process, so by analyzing the positional relationship between the edge lines and the virtual shadow lines, the direction of the jittering can be determined.

[0090] At each shooting position, in each forest image of each measuring point, for any edge line, the matching point on the edge line is taken as the starting point, and the matching point corresponding to each virtual shadow line corresponding to the edge line is taken as the end point to obtain all the feature vectors (the number of feature vectors is equal to the number of matching point pairs). For any feature vector, the angle value from the preset direction counterclockwise to the feature vector is obtained. The angle value can represent the direction corresponding to the feature vector. Finally, the angle value that appears most frequently is taken as the jitter direction corresponding to each forest image of each measuring point.

[0091] It should be noted that, in this embodiment of the present invention, a coordinate system is constructed with the starting point of each eigenvector as the origin, and the preset direction can be the positive direction of the x-axis.

[0092] Step S303: at each shooting position, according to the blur level value and the shaking direction in each forest image at each measuring point, determine the shaking vector of each forest image at each measuring point.

[0093] Based on the above steps, the blur value and jitter direction of each forest image can be obtained. Here, the jitter direction and the blur value can be combined to comprehensively reflect the jitter characteristics.

[0094] At each shooting position, the blur degree value of each forest image at each measuring point is taken as the modulus, and the jitter direction corresponding to each forest image at each measuring point is taken as the direction, so as to construct the jitter vector corresponding to each forest image at each measuring point; the jitter vector provides an intuitive and quantitative way to describe the jitter characteristics of the forest image.

[0095] Step S304: comparing the jitter vectors of the forest image at each measuring point at all shooting positions, thereby determining a first jitter factor of each measuring point at each shooting position.

[0096] At each shooting position, the sum of the jitter vectors corresponding to all forest images of each measuring point is taken as the sum vector, and the sum vector is the superposition of the jitter characteristics when the drone shoots the forest image of the measuring point at each shooting position.

[0097] The maximum value of the modulus of the sum vector corresponding to each measuring point at all shooting positions is used as the denominator, and the modulus of the sum vector corresponding to each measuring point at each shooting position is used as the numerator to obtain the first jitter factor of each measuring point at each shooting position. The purpose of this is to make the jitter conditions of the same measuring point at different shooting positions comparable. The larger the first jitter factor of a measuring point at a certain shooting position, the more obvious the jitter degree at this shooting position is relative to the jitter degrees at other shooting positions.

[0098] Step S305: at each shooting position, analyzing the angle relationship and similarity between the jitter vectors of all forest images of each measuring point, and obtaining a second jitter factor of each measuring point at each shooting position.

[0099] At each shooting position, for any measuring point, the angle between the jitter vectors corresponding to any two forest images of the measuring point is calculated. The angle value can be used to quantify the change of the jitter vector. The larger the angle value, the more drastic the change in the jitter direction.

[0100] The sum vector is the superposition of the jitter characteristics of the drone when shooting the forest image of the measuring point at each shooting position, which can characterize the overall jitter trend. Therefore, the cosine similarity between the jitter vector of the measuring point in each forest image and the sum vector is calculated as the similarity value. The larger the similarity value, the more consistent the jitter trend in each forest image is with the overall jitter trend. The number of jitter vectors with similarity values ​​greater than the preset similarity threshold is used as the quantitative factor. The larger the quantitative factor, the more consistent the jitter trend in all forest images of each measuring point at each shooting position is, and thus the jitter degree of the drone can be regarded as small. Conversely, if the quantitative factor is smaller, it indicates that the jitter degree of the drone is large. It should be noted that the similarity threshold here is set to 0.65, and the specific value can be adjusted according to the implementation scenario, and is not limited here.

[0101] Finally, the second jitter factor of the measuring point at each shooting position is obtained according to the maximum angle value and the quantity factor corresponding to the measuring point. The formula model of the second jitter factor includes:

[0102]

[0103] in, Indicates the second jitter factor of each measuring point at each shooting position; It represents the maximum angle between the jitter vectors of each measuring point in all forest images at each shooting position; Indicates the quantity factor corresponding to each measuring point at each shooting position; Indicates the preset first parameter; Represents the normalization function.

[0104] In the formula model of the second jitter factor, based on the above analysis, it can be known that the larger the angle value, the more drastic the change in the jitter direction. If the quantitative factor is smaller, it indicates that the degree of jitter of the drone is greater. Therefore, the second jitter factor is positively correlated with the maximum angle value, and the second jitter factor is negatively correlated with the quantitative factor. In this embodiment of the present invention, the second jitter factor is obtained in the form of a ratio. At this time, the larger the second jitter factor of a certain measuring point at a certain shooting position, it means that when the drone obtains various data of the measuring point at the shooting position, a more violent jitter occurs, and the credibility of the data will be reduced.

[0105] It should be noted that the first parameter is preset The function of is to prevent the denominator from being 0. The value can be 0.001. The specific value can be adjusted according to the implementation scenario and is not limited here.

[0106] Step S306: The first jitter factor and the second jitter factor of each measuring point at each shooting position are integrated to determine the shooting jitter characteristic value of each measuring point at each shooting position.

[0107] Based on the analysis of the above steps, it can be known that the larger the first jitter factor of a certain measuring point at a certain shooting position, the more obvious the jitter degree at the shooting position is relative to the jitter degree at other shooting positions; the larger the second jitter factor of a certain measuring point at a certain shooting position, the more severe jitter occurs when the drone obtains various data of the measuring point at the shooting position; so the sum of the first jitter factor and the second jitter factor is normalized to the value as the shooting jitter characteristic value of the measuring point at each shooting position. At this time, the larger the shooting jitter characteristic value, the higher the jitter degree of the drone, and the monitoring position may have a deviation, thereby reducing the accuracy of the monitoring result. Normalization is a technical means well known to those skilled in the art, and the selection of the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0108] Step S4: The reference value of each measuring point at each shooting position is obtained by combining the pest mutation coefficient and shooting jitter characteristic value of each measuring point at each shooting position; the shooting positions are screened based on the reference values ​​of each measuring point at different shooting positions, and the pests in the forest area to be tested are monitored based on the pest characteristic values ​​of each measuring point at the screened shooting positions.

[0109] In step S3, two indicators for measuring the degree of drone jitter can be obtained, namely, the pest mutation coefficient and the shooting jitter characteristic value. The two can be combined to determine the credibility of the data of each measuring point at each shooting position, that is, to obtain the reference value. Then, the shooting positions can be screened based on the reference values ​​of each measuring point at different shooting positions, so that the pest characteristic values ​​of the measuring points at the finally screened shooting positions are least affected by the drone jitter, and the pest degree at each measuring point can be more accurately characterized. Finally, the pests in the forest area to be tested are monitored based on the pest characteristic values ​​of each measuring point at the screened shooting positions, and more accurate and more reliable monitoring results can be obtained.

[0110] Preferably, a method for obtaining a reference value in an embodiment of the present invention includes:

[0111] Based on the analysis in step S3, it can be known that at each shooting position, for any measuring point, the larger the pest mutation coefficient value at the measuring point, the greater the degree of mutation of the pest situation in the forest at the measuring point, which can be regarded as the greater the impact of the drone jitter, and the lower the reference value. Therefore, the pest mutation coefficient at the measuring point is negatively mapped to achieve logical relationship correction and obtain the first reference factor. At this time, the larger the first reference factor is, the smaller the impact of the jitter is, and the higher the reference value is.

[0112] Similarly, the larger the shooting jitter characteristic value of the measuring point, the higher the degree of jitter of the drone, and the monitored position may be biased, thereby reducing the accuracy of the monitoring result. Therefore, the shooting jitter characteristic value corresponding to the measuring point is also negatively correlated to achieve logical relationship correction and obtain the second reference factor. At this time, the larger the second reference factor, the higher the reference value.

[0113] Finally, the product of the first reference factor and the second reference factor is normalized to obtain the value as the reference value of the measurement point at each shooting position. The higher the reference value, the less affected by the jitter of the drone, and thus the more accurate the data. Normalization is a technical means well known to those skilled in the art, and the normalization function can be linear normalization or standard normalization, etc. The specific normalization method is not limited here.

[0114] It should be noted that, in this embodiment of the present invention, the negative correlation mapping can be performed using the formula , or ,in, represents an exponential function with the natural constant e as the base; x represents the independent variable; It indicates that the second parameter is preset to prevent the denominator from being 0. Here, the value can be 0.001. The specific value can be adjusted according to the implementation scenario and is not limited.

[0115] After calculating the reference value of each measuring point at different shooting positions, the shooting positions can be screened based on the reference value, so as to monitor the insect pests in the forest to be measured based on the insect pest characteristic values ​​of each measuring point at the screened shooting positions.

[0116] Preferably, in one embodiment of the present invention, the shooting positions are screened based on the reference values ​​of each measuring point at different shooting positions, and the insect pests in the forest area to be tested are monitored based on the insect pest characteristic values ​​of each measuring point at the screened shooting positions, including:

[0117] For any measuring point, the shooting position corresponding to the maximum value of the reference value of the measuring point in all shooting positions is taken as the target position, and the pest characteristic value of the measuring point at the target position is taken as the final pest degree value of the measuring point. The final pest characteristic value at this time can remove the interference of drone jitter to the greatest extent, so it can more realistically reflect the pest situation of the forest at the measuring point.

[0118] Then, in the forest area to be tested, when the mean of the final pest severity values ​​of all measuring points is greater than the preset pest threshold, the pest severity is considered to be high and an early warning is required; and when the mean of the final pest severity values ​​of all measuring points is less than or equal to the preset pest threshold, the pest severity is considered to be low and no early warning is required.

[0119] It should be noted that the preset pest threshold is set to 0.6, and the specific value can be adjusted according to the implementation scenario and is not limited here.

[0120] In other embodiments of the present invention, a three-dimensional model can also be built. At each measuring point, the forest image with the smallest blur is selected from the forest images taken at all shooting positions to construct a plan view of the forest area to be measured. Then, each measuring point in the plan view has a corresponding final pest degree value, thereby achieving visualization and facilitating the staff to monitor the forest area to be measured.

[0121] In summary, the present invention can comprehensively reflect the physiological and ecological conditions and pest conditions of the forest by acquiring hyperspectral data and multiple forest images of each measuring point at each shooting position in the forest area to be measured, thereby improving the richness of the data. When forest trees are infested with insects, elements in leaves or branches will be missing, and the pest situation can be characterized by the change in the absorbance of the wavelength in the spectral curve of the hyperspectral data. Therefore, the difference in the spectral curves between the hyperspectral data of the measuring points and the absorbance corresponding to the wavelength are analyzed to determine the pest characteristic value at each measuring point. Because the scope of the pest area is gradually increasing, adjacent measuring points should have relatively similar performance characteristics. If the pest characteristic value at the measuring point has a relatively obvious mutation at the same shooting position, it can be regarded as that the UAV is affected by factors such as flight posture and wind speed during the shooting process, resulting in jitter and data deviation. Furthermore, if the UAV jitters, blur or phantom may appear in the acquired image data. Therefore, in each forest image of each measuring point, the edge line and the phantom line corresponding to each edge line are obtained. In this way, the shooting jitter characteristic value of each measuring point at each shooting position can be determined based on the quantity characteristics and position characteristics of the edge lines and phantom lines in all forest images of each measuring point at different shooting positions. Both the pest mutation coefficient and the shooting jitter characteristic value can measure the degree of jitter of the drone at each shooting position. Therefore, by combining these two indicators, the reference value of the pest characteristic value of each measuring point at each shooting position can be quantified, and the shooting positions can be further screened based on the reference value. At this time, the shooting positions screened out by each measuring point can be regarded as positions that are less affected by the jitter of the drone. Therefore, the pests in the forest area to be tested can be monitored based on the pest characteristic values ​​of the measuring points at the screened shooting positions, and more accurate monitoring results can be obtained.

[0122] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An intelligent forest pest monitoring method based on UAV remote sensing, characterized in that: The method comprises: In the forest area to be measured, hyperspectral data and multiple forest images are obtained at each measuring point at each shooting position; At each shooting position, analyze the differences in the spectral curves between the hyperspectral data of the measuring points, as well as the absorbance corresponding to the wavelength in the spectral curve of the measuring point, to determine the pest characteristic value at each measuring point; The differences in pest characteristic values ​​at different measuring points under the same shooting position are analyzed to obtain the pest mutation coefficient at each measuring point; in each forest image at each measuring point, the edge line and the virtual shadow line corresponding to each edge line are obtained; for any measuring point, based on the quantity characteristics of the edge lines and virtual shadow lines in all forest images of the measuring point under different shooting positions, as well as the position characteristics between the edge lines and virtual shadow lines, the shooting jitter characteristic value of the measuring point at each shooting position is determined; The reference value of each measuring point at each shooting position is obtained by combining the pest mutation coefficient and shooting jitter characteristic value of each measuring point at each shooting position; the shooting positions are screened based on the reference values ​​of each measuring point at different shooting positions, and the pests in the forest area to be tested are monitored based on the pest characteristic values ​​of each measuring point at the screened shooting positions; The step of obtaining edge lines and virtual shadow lines corresponding to each edge line in each forest image at each measuring point includes: In each forest image at each measuring point, all edge lines in the forest image are obtained based on the Sobel operator; Taking each edge line as a template, template matching is performed in the forest image to which the edge line belongs, so as to obtain the virtual shadow line corresponding to each edge line, and the matching point pairs between each edge line and each corresponding virtual shadow line; The method for obtaining the shooting jitter characteristic value comprises: At each shooting position, based on the quantity characteristics of edge lines and virtual shadow lines in each forest image of each measuring point, the blur degree value of each forest image is determined; Based on the positional features between the edge line and the virtual shadow line in each forest image at each measuring point, the shaking direction of each forest image is determined; At each shooting position, the blur degree value of each forest image at each measuring point is used as the modulus, and the jitter direction corresponding to each forest image at each measuring point is used as the direction, so as to obtain the jitter vector corresponding to each forest image at each measuring point, and the sum value of the jitter vectors corresponding to all forest images at each measuring point is used as the sum vector; The ratio of the modulus length of the sum vector corresponding to each measuring point at each shooting position to the maximum modulus length of the sum vector corresponding to each measuring point at all shooting positions is used as the first jitter factor of each measuring point at each shooting position; At each shooting position, for any measuring point, calculate the angle between the jitter vectors corresponding to any two forest images at the measuring point; Calculate the cosine similarity between the jitter vector of the measuring point in each forest image and the sum vector as a similarity value, and use the number of jitter vectors with similarity values ​​greater than a preset similarity threshold as a quantity factor; Obtaining a second jitter factor of the measuring point at each shooting position according to the maximum angle value corresponding to the measuring point and the quantity factor, wherein the second jitter factor is positively correlated with the maximum first angle value, and the second jitter factor is negatively correlated with the quantity factor; A value obtained by normalizing the sum of the first jitter factor and the second jitter factor is used as a shooting jitter characteristic value of the measuring point at each shooting position.

2. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method for obtaining the pest characteristic value comprises: Based on the differences between the spectral curves corresponding to the hyperspectral data of each measuring point at the same shooting position, the first pest factor at each measuring point is determined; At the same shooting position, the maximum value in the spectrum curve at each measuring point is obtained, and the sum of all the maximum values ​​is taken as the second pest factor at each measuring point; The sum of the first pest factor and the second pest factor of each measuring point at the same shooting position is normalized to obtain the pest characteristic value of each measuring point at each shooting position.

3. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 2 is characterized in that: The method for obtaining the first pest factor comprises: At each shooting position, the area enclosed by the spectral curve corresponding to the hyperspectral data of each measuring point and the coordinate axis is taken as the integral eigenvalue; The ratio of the integral eigenvalue corresponding to each measuring point to the maximum integral eigenvalue of all measuring points is taken as the first pest factor at each measuring point.

4. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method for obtaining the pest mutation coefficient comprises: In the preset neighborhood of any measuring point, at each shooting position, the absolute value of the difference between the pest characteristic value of the measuring point and each neighboring measuring point is calculated as the pest deviation factor, and the sum of the pest deviation factors between the measuring point and all neighboring measuring points is used as the mutation factor of the measuring point; The normalized value of the ratio of the mutation factor of each measuring point to the maximum mutation factor of all measuring points is used as the pest mutation coefficient of each measuring point at each shooting position.

5. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method for obtaining the blur degree value includes: At each shooting position, for any measuring point, the average value of the number of virtual shadow lines corresponding to all edge lines in each forest image of the measuring point is normalized and used as the blur degree value of each forest image of the measuring point.

6. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method for obtaining the shaking direction includes: At each shooting position, in each forest image of each measuring point, for any edge line, the matching point on the edge line is taken as the starting point, and the matching point corresponding to each virtual shadow line corresponding to the edge line is taken as the end point, and all feature vectors are obtained, and the angle value between each feature vector and the preset direction is obtained, and the angle value that appears most frequently is taken as the jitter direction corresponding to each forest image of each measuring point.

7. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method for obtaining the reference value includes: At each shooting position, for any measuring point, the value of the pest mutation coefficient at the measuring point after negative correlation mapping is used as the first reference factor; The value after negative correlation mapping of the shooting jitter characteristic value corresponding to the measuring point is used as the second reference factor; The value obtained by normalizing the product of the first reference factor and the second reference factor is used as the reference value of the measuring point at each shooting position.

8. The method for intelligent monitoring of forest pests based on UAV remote sensing according to claim 1 is characterized in that: The method of screening the shooting positions based on the reference values ​​of each measuring point at different shooting positions, and monitoring the insect pests in the forest area to be tested based on the insect pest characteristic values ​​of each measuring point at the screened shooting positions, includes: For any measuring point, the shooting position corresponding to the maximum value of the reference value of the measuring point at all shooting positions is taken as the target position, and the pest characteristic value of the measuring point at the target position is taken as the final pest degree value of the measuring point; In the forest area to be tested, when the average of the final pest severity values ​​of all measuring points is greater than the preset pest threshold, the pest severity is considered to be high and an early warning is required; when the average of the final pest severity values ​​of all measuring points is less than or equal to the preset pest threshold, the pest severity is considered to be low and no early warning is required.

Images