Forest pest recognition and positioning method based on deep learning

Abstract

The invention discloses a forest pine nematode disease identification and positioning method based on deep learning, and the method comprises the steps: constructing a multi-modal data set through the fusion of LiDAR point cloud, image data and time sequence information, and employing the advantages of a convolutional neural network (CNN), a graph neural network (GNN) and a long short-term memory network (LSTM) in the method, thereby achieving the recognition and positioning of the forest pine nematode disease. According to the method, high-precision capture and analysis of tree geometric structure information and time change conditions are realized, and a scientific basis can be provided for tracking the origin of pathogens by determining an initial infection source, so that a more effective control strategy is formulated; by generating the infection probability graph and predicting the transmission path of the pine nematode disease, the transmission of the pine nematode disease can be predicted in advance, and the source of the pine nematode disease can be found; compared with the prior art, prevention measures can be taken more quickly, the risk of pine nematode disease diffusion is reduced, and the efficiency and accuracy of forest pine nematode disease monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the field of forest pests and diseases, and in particular to a method for identifying and locating forest pests and diseases based on deep learning. Background Art

[0002] Pinewood Nematode Disease, also known as pine wilt disease, is a disease caused by the pine wood nematode (Bursaphelenchus xylophilus) that seriously threatens the health of pine trees. Since it was first discovered in Japan in the early 20th century, the disease has spread to many countries and regions in Asia, Europe and North America, causing huge economic losses and ecological damage to pine forests around the world.

[0003] Traditional pest and disease monitoring relies on regular manual field inspections by forestry workers; workers will enter the forest area and confirm the presence of pine wood nematode disease through visual inspection, sampling and laboratory analysis. However, manual analysis makes it difficult to accurately predict the infection range of pine nematode disease and find the source of infection to implement prevention. Summary of the invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a method for identifying and locating forest pests and diseases based on deep learning.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a method for identifying and locating forest pests and diseases based on deep learning, comprising: S1, obtaining tree geometric structure information and time series information, and preprocessing them;

[0006] S2, generate training labels from preprocessed multi-source data;

[0007] S3. Establish a multimodal fusion deep learning model;

[0008] S4. Generate infection probability map through multimodal fusion deep learning model and predict transmission path;

[0009] S5. Determine the initial source of infection.

[0010] In a preferred embodiment of the present invention, in step S1, the geometric structure information includes parameters such as tree height, crown width, trunk diameter, etc., and the time series information is LiDAR point cloud data constructed by multiple scans in different seasons.

[0011] In a preferred embodiment of the present invention, in step S1, the preprocessing is specifically to classify the collected data and use a clustering algorithm to classify the LiDAR point cloud into ground, vegetation and other categories:

[0012]

[0013] Among them, p i =(x i ,y i ,z i ) is the coordinate of the i-th LiDAR point, p j =(x j ,y j ,z j ) is the coordinate of the jth LiDAR point;

[0014] Segment the point cloud into individual trees, ensuring that each pine tree has independent geometry:

[0015] C(p,q)=|I(p)-I(q)|<T, where I(p) is the reflection intensity or color value of point p, I(q) is the reflection intensity or color value of point q, and T is the threshold value;

[0016] The tree height, color features, and texture features are extracted: the tree height is determined by calculating the maximum height difference between the top of the tree crown and the ground point; texture features: GLCM (i, j, d, θ) = frequency.

[0017] In a preferred embodiment of the present invention, in step S2, the specific steps are: combining the preprocessed LiDAR point cloud data and time series data to form a multimodal data set.

[0018] In a preferred embodiment of the present invention, in step S3, the multimodal fusion deep learning model is designed and trained using the training labels in S2;

[0019] We select convolutional neural networks to process image data, graph neural networks to process LiDAR point cloud data, and long short-term memory networks to process time series data. By combining these three networks, we build a multimodal fusion model that can process image, point cloud, and time series data simultaneously. The specific algorithm steps are as follows:

[0020] W attention =[w CNN ,w GNN ,w LSTM ];

[0021] F fusion =w CNN ·F CNN +w GNN ·F GNN +w LSTM ·F LSTM ;

[0022] w CNN ,w GNN ,w LSTM for different attention weights.

[0023] In a preferred embodiment of the present invention, the specific steps of the convolutional neural network are:

[0024] CNN is used to process image features:

[0025] O(x,y) is the value of position (x,y) on the output feature map, K(m,n) is the convolution kernel weight, and its size is (2a+1)(2b+1);

[0026] F CNN =CNN(I;θ CNN );

[0027] Θ CNN is the parameter set of CNN.

[0028] In a preferred embodiment of the present invention, the specific steps of the graph neural network are:

[0029] GNN to model spatial relationships:

[0030]

[0031] F GNN =GNN(G; θ GNN );

[0032] Among them, Θ GNN is the parameter set of GNN, G = (V, E), G is the graph structure of LiDAR point cloud conversion;

[0033] is the hidden state of node v in layer l, N(v) is the set of neighbors of node v, W l is the weight matrix of the lth layer, and σ is the ReLU function.

[0034] In a preferred embodiment of the present invention, the specific steps of the long short-term memory network are:

[0035] LSTM captures temporal dependencies:

[0036] Input gate: i t :i t =σ(W i [h t-1 ,x t ]+b i );

[0037] Forget gate: f t :f t =σ(W f [h t-1 ,x t ]+b f );

[0038] Output gate: o t :o t =σ(W o [h t-1 ,x t ]+b o );

[0039] The final formula is: F LSTM =LSTM(T; θ LSTM );

[0040] Among them, Θ LSTM is the set of all parameters of LSTM, T is the input time series data, t is the current time step index, indicating the position in the time series, W is the weight matrix, b is the bias vector, [h t-1 ,x t ] is the hidden state h of the previous moment t-1 and the input x at the current time step t Concatenated into a vector, i, f, o are the states of the input gate, forget gate, and output gate.

[0041] In a preferred embodiment of the present invention, in step S4, the specific steps of generating the infection probability map are:

[0042] Through the Softmax function,

[0043] where P(y=jx; W,b) is the probability of class j given the input feature vector x, z j is the unnormalized prediction score for class j; W is the weight matrix and b is the bias vector.

[0044] In a preferred embodiment of the present invention, in step S5, the specific steps of determining the initial infection source are:

[0045] Getis-Ord Gi hot spot analysis statistics:

[0046]

[0047] Among them, G i (d) is the G of node i i Statistics, w ij is the spatial weight matrix, which depends on the distance d between two points, x j is the infection probability of node j, is the average value of all node attributes, S is the standard deviation, and n is the total number of nodes.

[0048] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0049] (1) By integrating LiDAR point cloud, image data and time series information, a multimodal dataset was constructed. This method took advantage of the advantages of convolutional neural networks (CNN), graph neural networks (GNN) and long short-term memory networks (LSTM) to achieve high-precision capture and analysis of tree geometric structure information and temporal changes. By determining the initial source of infection, it can provide a scientific basis for tracing the origin of the pathogen, thereby formulating more effective control strategies. By generating infection probability maps and predicting the transmission path of pine nematode disease, the spread of pine nematode disease can be predicted in advance and the source of pine nematode disease can be found. Compared with existing technologies, preventive measures can be taken more quickly to reduce the risk of pine nematode disease spreading, thereby improving the efficiency and accuracy of forest pine nematode disease monitoring.

[0050] (2) A multimodal dataset containing rich spatial information and changes in the temporal dimension is created by combining preprocessed LiDAR point cloud data and time series data. CNN is used to process image data, GNN is used to process LiDAR point cloud data, and LSTM is used to process time series data. A multimodal fusion model that can simultaneously process image, point cloud, and time series data is constructed. The dataset not only enriches the input of the model, but also reflects the changes of trees over time, which helps to improve the generalization ability of the model and integrate information from different sources, thereby improving the accuracy and efficiency of pine nematode diagnosis. Compared with existing technologies, it promotes more accurate pine nematode identification and positioning. Moreover, by combining the three networks, the model can maintain high performance in complex environments, thereby providing more reliable pine nematode monitoring services.

[0051] (3) The Softmax function is applied to calculate the probability of each category given the input feature vector x, and an infection probability map is generated. The Getis-Ord Gi* hotspot analysis statistic is used to determine the initial source of infection. This provides an intuitive understanding of which areas may have been affected by pine nematode disease and allows prediction of the future transmission path of pine nematode disease. At the same time, clustered areas with a high probability of infection, i.e., potential initial infection points, are identified. Compared with existing technologies, preventive measures can be taken in advance to reduce the risk of pine nematode disease spreading. Knowing the location of the initial infection source can help develop more effective control strategies to prevent the further spread of pine nematode disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 is a flow chart of a preferred embodiment of the present invention; DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0056] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the scope of protection of the present application. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, "multiple" means two or more.

[0057] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0058] like Figure 1 As shown, a forest pest identification and positioning method based on deep learning includes:

[0059] S1, obtaining tree geometric structure information and time series information, and preprocessing them;

[0060] S2, generate training labels from preprocessed multi-source data;

[0061] S3. Establish a multimodal fusion deep learning model;

[0062] S4. Generate infection probability map through multimodal fusion deep learning model and predict transmission path;

[0063] S5. Determine the initial source of infection;

[0064] Furthermore, in step S1, a ground-based LiDAR system is used to perform high-resolution scanning of the target forest area to collect geometric structural information such as tree height, crown width, trunk diameter, etc. At the same time, multiple scans are performed in different seasons (such as spring, summer, and autumn) to record the changes of pine trees over time and construct time series data, which are composed of LiDAR point clouds;

[0065] It is worth noting that point cloud here refers to a data set consisting of a large number of points in a three-dimensional coordinate system, each of which represents a measured position on the surface of an object. These points are obtained by 3D scanning equipment such as LiDAR. Each point in the point cloud contains its position information in space (X, Y, Z coordinates), as well as color (RGB value), reflection intensity or timestamp, etc.

[0066] In step S1, the preprocessing is to classify the collected data and use the clustering algorithm (DBSCAN) to divide the LiDAR point cloud into ground, vegetation and other categories:

[0067]

[0068] Among them, p i =(x i ,y i ,z i ) is the coordinate of the i-th LiDAR point, p j =(x j ,y j ,z j ) is the coordinate of the jth LiDAR point;

[0069] (Region Growing) Segment the point cloud into individual trees, ensuring that each pine tree has independent geometric features:

[0070] C(p,q)=|I(p)-I(q)|<T, where I(p) is the reflection intensity or color value of point p, I(q) is the reflection intensity or color value of point q, and T is the threshold value;

[0071] The features such as tree height, color features (color changes of needles), and texture features (reflecting the health of needles) are extracted: tree height is determined by calculating the maximum height difference between the top of the crown and the ground point;

[0072] Texture features (Gray Level Co-occurrence Matrix GLCM)

[0073] GLCM(i,j,d,θ) = frequency (the pixel pair (i,j) occurs at distance d and direction θ)

[0074] We use a ground-based LiDAR system to perform high-resolution scans of target forest areas, which helps to accurately capture the three-dimensional geometric structure information of trees, including key parameters such as height, crown width, and trunk diameter. These data form a LiDAR point cloud, which provides dynamic change information during the tree growth cycle. This process not only improves the accuracy of early detection of pests and diseases, but also provides an important basis for predicting the spread trend of pests and diseases.

[0075] In the preprocessing stage, a clustering algorithm (DBSCAN) was used to classify the LiDAR point cloud into ground, vegetation, and other categories, ensuring the accuracy of subsequent analysis. The point cloud was further segmented into individual trees using the Region Growing algorithm, so that each pine tree had independent geometric features, which was crucial for distinguishing individual trees and their health status. The extracted features such as tree height, color features (color changes of needles), and texture features (reflecting the health of needles) played an important role in identifying whether the trees were affected by pests and diseases. The gray-level co-occurrence matrix (GLCM) was used to quantify texture features, which enhanced the model's ability to assess the health status of trees.

[0076] In step S2, the specific steps are: combining the preprocessed LiDAR point cloud data and time series data to form a multimodal dataset. This dataset not only contains rich spatial information, but also reflects the changes of trees over time, which helps to improve the generalization ability of the model.

[0077] To make it easier to understand, let’s take an example where a pine tree is infected with pine wilt disease. The location of the tree is marked and given a label of 1 (indicating infection), while healthy trees are marked with 0.

[0078] In step S3, a multimodal fusion deep learning model is designed and trained using the training labels in S2;

[0079] In order to process different types of data, we select convolutional neural network (CNN) to process image data, graph neural network (GNN) to process LiDAR point cloud data, and long short-term memory network (LSTM) to process time series data. By combining these three networks, we build a multimodal fusion model that can process image, point cloud and time series data at the same time. The specific algorithm steps are as follows:

[0080] CNN (Convolutional Neural Network) is used to process image features:

[0081]

[0082] O(x,y) is the value of position (x,y) on the output feature map, K(m,n) is the convolution kernel weight, and its size is (2a+1)(2b+1).

[0083] F CNN =CNN(I;θ CNN );

[0084] Θ CNN is the parameter set of CNN;

[0085] GNN (Graph Neural Network) to model spatial relationships:

[0086]

[0087] F GNN =GNN(G; θ GNN ),

[0088] Among them, Θ GNN is the parameter set of GNN, G = (V, E), G is the graph structure of LiDAR point cloud conversion;

[0089] h u l is the hidden state of node v (each node represents a tree) at layer l, N(v) is the set of neighbors of node v, and W l is the weight matrix of the lth layer, and σ is the ReLU function.

[0090] LSTM (Long Short-Term Memory) captures temporal dependencies:

[0091] Input gate: i t :i t =σ(W i [h t-1 ,x t ]+b i )

[0092] Forget gate: f t :f t =σ(W f [ht-1 ,x t ]+b f )

[0093] Output gate: o t :o t =σ(W o [h t-1 ,x t ]+b o )

[0094] The final formula is: F LSTM =LSTM(T; θ LSTM )

[0095] Among them, Θ LSTM is the set of all parameters of LSTM, T is the input time series data, t is the current time step index, indicating the position in the time series, W is the weight matrix, b is the bias vector, [h t-1 ,x t ] is the hidden state h of the previous moment t-1 and the input x at the current time step t Concatenate into a vector, i, f, o are the states of the input gate, forget gate and output gate;

[0096] The final multimodal fusion deep learning model is:

[0097] W attention =[w CNN ,w GNN ,w LSTM ];

[0098] F fusion =w CNN ·F CNN +w GNN ·F GNN +w LSTM ·F LSTM ;

[0099] w CNN ,w GNN ,w LSTM for different attention weights.

[0100] In step S4, the specific steps of generating the infection probability map are as follows:

[0101] Through the Softmax function,

[0102] where P(y=jx; W,b) is the probability of class j given the input feature vector x, z j is the unnormalized prediction score for class j; W is the weight matrix and b is the bias vector.

[0103] The infection probability map helps us intuitively understand which areas may be affected by pests and diseases, and also allows us to predict the future spread of pests and diseases. By analyzing the infection probability map, forestry managers can take preventive measures in advance to reduce the risk of pests and diseases spreading.

[0104] In step S5, the specific steps of determining the initial infection source are:

[0105] Getis-Ord Gi hot spot analysis statistics:

[0106]

[0107] Among them, G i (d) is the G of node i i Statistics, w ij is the spatial weight matrix, which depends on the distance d between two points, x j is the infection probability of node j, is the average value of all node attributes, S is the standard deviation, and n is the total number of nodes.

[0108] By determining the initial source of infection, clustered areas with high infection probability, i.e. potential initial infection points, can be identified. It is based on the spatial weight matrix between nodes, taking into account the mean and standard deviation of all node attribute values, and provides a scientific basis for tracing the origin of pathogens. Knowing the location of the initial infection source can help develop more effective control strategies to prevent the further spread of pests and diseases.

[0109] The above is based on the ideal embodiment of the present invention. Through the above description, relevant personnel can make various changes and modifications without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.

Claims

1. A method for identifying and locating forest pests and diseases based on deep learning, characterized in that: include: S1, obtaining tree geometric structure information and time series information, and preprocessing them; S2, generate training labels from preprocessed multi-source data; S3, establish a multimodal fusion deep learning model; S4. Generate infection probability map through multimodal fusion deep learning model and predict transmission path; S5. Determine the initial source of infection.

2. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S1, the geometric structure information includes parameters such as tree height, crown width, trunk diameter, etc., while the time series information is LiDAR point cloud data constructed by multiple scans in different seasons.

3. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S1, the preprocessing is to classify the collected data and use a clustering algorithm to divide the LiDAR point cloud into ground, vegetation and other categories: Among them, p i =(x i ,y i ,z i ) is the coordinate of the i-th LiDAR point, p j =(x j ,y j ,z j ) is the coordinate of the jth LiDAR point; Segment the point cloud into individual trees, ensuring that each pine tree has independent geometry: C(p,q)=|I(p)-I(q)|<T, where I(p) is the reflection intensity or color value of point p, I(q) is the reflection intensity or color value of point q, and T is the threshold value; The tree height, color features, and texture features are extracted: the tree height is determined by calculating the maximum height difference between the top of the tree crown and the ground point; texture features: GLCM (i, j, d, θ) = frequency.

4. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S2, the specific steps are: combining the preprocessed LiDAR point cloud data and time series data to form a multimodal data set.

5. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S3, a multimodal fusion deep learning model is designed and trained using the training labels in S2; Select convolutional neural network to process image data, graph neural network to process LiDAR point cloud data, and long short-term memory network to process time series data; by combining these three networks, a multimodal fusion model is constructed that can process image, point cloud and time series data at the same time; the specific algorithm steps are as follows: IN attention =[in CNN ,In GNN ,In LSTM ]; F fusion =w CNN ·F CNN +w GNN ·F GNN +w LSTM ·F LSTM ; w CNN ,w GNN ,w LSTM for different attention weights.

6. The method for identifying and locating forest pests and diseases based on deep learning according to claim 5, characterized in that: The specific steps of the convolutional neural network are: CNN is used to process image features: O(x,y) is the value of position (x,y) on the output feature map, K(m,n) is the convolution kernel weight, and its size is (2a+1)(2b+1); F CNN =CNN(I;Θ CNN ); Θ CNN is the parameter set of CNN.

7. The method for identifying and locating forest pests and diseases based on deep learning according to claim 5, characterized in that: The specific steps of graph neural network are: GNN to model spatial relationships: F GNN =GNN(G;Θ GNN ); Among them, Θ GNN is the parameter set of GNN, G = (V, E), G is the graph structure of LiDAR point cloud conversion; is the hidden state of node v in layer l, N(v) is the set of neighbors of node v, W l is the weight matrix of the lth layer, and σ is the ReLU function.

8. The method for identifying and locating forest pests and diseases based on deep learning according to claim 5, characterized in that: The specific steps of long short-term memory network are: LSTM captures temporal dependencies: Input gate: i t :i t =σ(W i [h t-1 ,x t ]+b i ); Forget gate: f t :f t =σ(W f [h t-1 ,x t ]+b f ); Output gate: o t :o t =σ(W o [h t-1 ,x t ]+b o ); The final formula is: F LSTM =LSTM(T; θ LSTM ); Among them, Θ LSTM is the set of all parameters of LSTM, T is the input time series data, t is the current time step index, indicating the position in the time series, W is the weight matrix, b is the bias vector, [h t-1 ,x t ] is the hidden state h of the previous moment t-1 and the input x at the current time step t Concatenated into a vector, i, f, o are the states of the input gate, forget gate, and output gate.

9. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S4, the specific steps of generating the infection probability map are as follows: Through the Softmax function, where P(y=j|x; W,b) is the probability of class j given the input feature vector x, z j is the unnormalized prediction score for class j; W is the weight matrix and b is the bias vector.

10. The method for identifying and locating forest pests and diseases based on deep learning according to claim 1, characterized in that: In step S5, the specific steps of determining the initial infection source are: Getis-Ord Gi hot spot analysis statistics: Among them, G i (d) is the G of node i i Statistics, w ij is the spatial weight matrix, which depends on the distance d between two points, x j is the infection probability of node j, x is the average of all node attribute values, S is the standard deviation, and n is the total number of nodes.

Images