Forestry pest and disease damage intelligent monitoring system and method based on unmanned aerial vehicle inspection

Abstract

The invention relates to the field of forestry monitoring data processing, in particular to a forestry pest and disease damage intelligent monitoring system and method based on unmanned aerial vehicle inspection, and the system comprises a data interaction unit which is used for building a real-time communication channel between an unmanned aerial vehicle and a server through employing a WebSocket protocol; the multi-scale feature extraction unit is used for constructing a three-stage pyramid convolution structure, and respectively extracting microscopic, medium and macroscopic scale features of the blade through convolution kernels of different specifications and cavity convolution; the time sequence parameter adjusting unit is used for fusing Transform and GRU, analyzing a feature map through a multi-head self-attention mechanism, learning a tree phenological law through the GRU, performing condition normalization adjustment on convolutional layer parameters based on the tree phenological law, and distinguishing physiological and disease features; and the decision fusion unit is used for splicing and fusing the composite feature map and the time sequence feature vector, carrying out parallel processing on an SVM and a Softmax classifier, and outputting a disease and pest recognition conclusion according to a dynamic threshold decision.

Description

Technical Field

[0001] The present invention relates to the field of forestry monitoring data processing, and in particular to an intelligent forestry pest and disease monitoring system and method based on unmanned aerial vehicle inspections. Background Art

[0002] In the field of forestry pest and disease monitoring, traditional monitoring methods suffer from low efficiency and poor accuracy. Drone inspection technology has been widely adopted due to its efficiency and flexibility. However, tree phenology causes periodic changes in leaf morphology and spectral reflectance. Environmental factors such as soil fertility and rainfall can also affect vegetation health, and these changes can easily be confused with disease symptoms. Existing convolutional networks struggle to accurately distinguish between "physiological leaf drop" and "disease-related leaf drop" when processing relevant data. For example, the spectral characteristics of the natural yellowing of ginkgo leaves in autumn overlap with those caused by fungal diseases, making intelligent monitoring of forestry pests and diseases a technical challenge. Summary of the Invention

[0003] In order to solve the problems mentioned in the above background technology, the present invention provides an intelligent monitoring system for forestry pests and diseases based on drone inspections, and also provides an intelligent monitoring method for forestry pests and diseases based on drone inspections.

[0004] The first aspect discloses an intelligent forest pest monitoring system based on drone inspection, comprising: The drone side is used to collect forestry data in the inspection target area; The data interaction unit is used to establish a real-time communication channel between the drone and the server using the WebSocket protocol, process data using the Min-Max normalization algorithm, and provide standardized input for the convolutional network; The multi-scale feature extraction unit is used to construct a three-level pyramid convolution structure. Through convolution kernels of different specifications and dilated convolution, the micro-, medium-, and macro-scale features of leaves are extracted respectively. After fusion through the feature pyramid network, a composite feature map is formed, covering the full-scale morphological spectral changes of phenology and disease. The temporal parameter adjustment unit is used to integrate the Transformer and GRU. It uses a multi-head self-attention mechanism to analyze feature maps and uses the GRU to learn tree phenology patterns. Based on this, it conditionally normalizes the convolutional layer parameters to distinguish physiological and disease characteristics. The decision fusion unit is used to splice and fuse the composite feature map with the time series feature vector, use SVM and Softmax classifiers for parallel processing, make decisions based on dynamic thresholds, and output the pest and disease identification conclusion.

[0005] Furthermore, the WebSocket protocol is used to establish a real-time communication channel between the drone and the server, including: the drone and the server complete the initial connection through TCP, establish a full-duplex communication link after negotiating an HTTP upgrade request, and use a message fragmentation mechanism during the communication process to split the single-frame multispectral image and visible light image into fixed data segments, which are transmitted through message packets with sequence numbers; the communication protocol has a built-in heartbeat detection mechanism, and the drone sends a connection maintenance packet to the server every 2 seconds. If the server does not receive a response for more than 10 seconds, the reconnection logic is triggered, and the exponential backoff algorithm is used to gradually try to restore the connection; the transmitted data is encapsulated in a custom binary format, which contains data type identification, timestamp, and drone number metadata; The server-side receiving module adopts an asynchronous non-blocking IO model to handle concurrent connections of multiple drones at the same time. Each connection is allocated an independent buffer to temporarily store unprocessed data to avoid frame loss caused by data congestion. The data interaction process follows the "receive first, then process" principle. After the raw data reaches the server through the communication channel, it is first stored in a temporary cache queue in timestamp order. The data interaction unit extracts it in batches at a fixed frequency, synchronously triggering the Min-Max normalization preprocessing process.

[0006] Furthermore, the Min-Max normalization algorithm was used to process the data. Specifically, after receiving the multispectral image and visible light image from the UAV, a Min-Max normalization operation was performed on the pixel values ​​of each spectral channel. First, a sliding window was constructed for each image, and the window size was set. Then, the maximum and minimum pixel values ​​of each spectral channel were calculated within the window. Then, the pixel values ​​within the window were normalized and mapped to the interval [0, 1]. To ensure the timeliness of the normalization parameters, the sliding window moves with a fixed overlap rate. At the same time, to prevent interference from extreme values, abnormally high and low values ​​within the window are eliminated. When processing time series data, the normalization parameters of each window are retained and stored in the parameter cache queue together with the corresponding timestamp. When performing real-time inference, the most recent normalization parameters are searched from the cache queue based on the current timestamp to standardize the newly input data, ensuring that the input data entering the convolutional network remains strictly aligned in time series.

[0007] Furthermore, constructing a three-level pyramid convolution structure includes: constructing a three-layer pyramid convolution structure, the first layer adopts a parallel three-branch convolution structure, and configures convolution kernels of 3×3, 5×5, and 7×7 specifications respectively; the 3×3 convolution kernel is responsible for extracting microscopic features at the scale of 1-3 pixels, which can capture the ultra-micro morphological changes of subtle lesions and early chlorotic spots on the leaf surface; the 5×5 convolution kernel is responsible for extracting medium-sized features at the scale of 3-5 pixels, which is used to identify local discoloration areas and early lesions of small insect holes; the 7×7 convolution kernel extracts preliminary macroscopic features at the scale of 5-7 pixels, and detects early morphological differences such as slight curling of leaf edges and local leaf shape abnormalities; each branch is connected in series with a batch normalization layer and a ReLU activation function to standardize the input distribution and introduce nonlinear mapping.

[0008] Furthermore, constructing a three-level pyramid convolution structure includes: constructing a three-layer pyramid convolution structure, the second layer adopts void convolution with dilation rate, and setting three groups of 9×9 void convolution kernel branches with dilation rates of 2, 4, and 6; the void convolution kernel with a dilation rate of 2 has an equivalent receptive field of 17×17, which can extract medium-sized features at the scale of 8-10 pixels, which is used to identify medium-sized diseased areas on leaves and abnormal local vein color characteristics; the branch with a dilation rate of 4 realizes an equivalent receptive field of 33×33, capturing regional features at the scale of 12-16 pixels, such as local withering patches and leaf vein structure deformation; the branch with a dilation rate of 6 forms an ultra-large receptive field of 51×51, extracting composite features at the scale of 18-22 pixels, covering the diseased area of ​​leaf tissue; after the output of each branch is compressed by 1×1 convolution, a multi-scale fusion feature map is formed by channel splicing, thereby expanding the feature coverage without increasing the amount of computation.

[0009] Furthermore, the construction of a three-level pyramid convolutional structure includes: constructing a three-layer pyramid convolutional structure, the third layer uses a 15×15 convolution kernel with a stride of 2 to achieve macro-feature extraction of more than 20 pixels, capturing significant features such as large-scale discoloration, severe deformities, and full leaf lesion coverage; this layer introduces a residual connection structure to directly superimpose input features on the output. After the convolution operation, the feature map size is halved, and the number of channels is increased to 512. After batch normalization, it is input into the feature pyramid network; The output feature maps of the three-level convolutional structure are cross-layer fused through a feature pyramid network: the low-resolution macro feature map of the third layer is upsampled to the middle-layer size through bilinear interpolation, and semantically fused with the medium-sized feature map of the second layer through element-by-element addition; the fused feature map is upsampled to the first-layer resolution again, and channel-wise spliced ​​with the micro feature map of the first layer to finally form a composite feature map containing 256 channels; this composite feature map covers multi-scale morphological and spectral information from 1 pixel to the entire leaf, effectively distinguishing normal physiological changes in the phenological period from abnormal characteristics caused by pests and diseases.

[0010] Furthermore, the fusion of Transformer and GRU includes: taking the composite feature map as the input of Transformer, calculating the dependency of different spatial positions in the feature map through the multi-head self-attention mechanism; reshaping the feature map into a sequence form, mapping the feature vector of each position into query, key and value matrices, and calculating the attention weight in parallel through 8 heads. Position encoding is introduced during the weight calculation to retain spatial information, and finally the feature representation of the fused global context is obtained through weighted summation to determine the key features of pests and diseases at the edge of the lesion and the abnormal spectral area; at the same time, the GRU unit receives the attention-weighted feature map of each time step and interacts with the hidden states of several previous time steps. It controls the degree of retention of historical phenological features by updating the gate, and forgets the outdated periodic change pattern by resetting the gate, thereby learning the normal spectral and morphological change patterns of trees during the spring shoot germination and autumn leaf fall phenological period.

[0011] Furthermore, conditional normalization adjustment of convolutional layer parameters based on tree phenology includes: after the GRU establishes a tree phenology model by learning historical time series data, it outputs a hidden state vector containing the characteristics of the current phenological stage, which is mapped into the scaling factor and offset factor required for conditional normalization through the fully connected layer; the weight matrix and bias vector of each convolutional layer are adjusted by the scaling factor and offset factor.

[0012] Furthermore, the composite feature map and the time series feature vector are spliced ​​and fused, including: the composite feature map output by the multi-scale feature extraction unit is first passed through the global average pooling layer to compress the spatial dimension to 1×1 to obtain a 1024-dimensional global feature vector; then the dimension is reduced to 128 dimensions through the fully connected layer, and directly spliced ​​with the 64-dimensional time series feature vector output by the timing parameter adjustment unit in the channel dimension to form a 192-dimensional fused feature vector.

[0013] Furthermore, SVM and Softmax classifiers are used for parallel processing, and according to dynamic threshold decision-making, the pest and disease identification conclusion is output, including: the 192-dimensional fusion feature vector output by the decision fusion unit is input into the SVM and Softmax classifiers at the same time; the SVM performs binary classification on the input features based on the optimal hyperplane obtained by historical data training to determine whether the current tree has abnormal pests and diseases; the Softmax classifier outputs the probability distribution of the feature vector belonging to several common pest and disease categories; by analyzing historical data, the distribution of normal physiological changes and pest and disease characteristics under different phenological periods is statistically analyzed, and the error rate is calculated; when the SVM judges it to be abnormal and the probability of a certain type of pest and disease output by the Softmax classifier is greater than the threshold, the system determines that this type of pest and disease exists; if the SVM judges that there is no abnormality, or the Softmax output probability is lower than the threshold, it is judged to be normal physiological changes or no pests and diseases, and finally the pest and disease identification conclusion is output.

[0014] On the other hand, the present application discloses an intelligent monitoring method for forestry pests and diseases based on drone inspection, comprising the following steps: S100. The drone collects forestry data from the inspection target area; S101. The data exchange unit uses the WebSocket protocol to establish a real-time communication channel between the drone and the server, and uses the Min-Max normalization algorithm to process data and provide standardized input for the convolutional network. S102. The multi-scale feature extraction unit constructs a three-level pyramid convolutional structure. Using convolution kernels of different sizes and dilated convolution, it extracts micro-, meso-, and macro-scale features of leaves. These features are then fused through a feature pyramid network to form a composite feature map covering the full-scale morphological spectral variation of phenology and disease. S103. The temporal parameter adjustment unit integrates the Transformer and GRU, analyzes feature maps through a multi-head self-attention mechanism, and uses the GRU to learn tree phenology patterns. Based on this, conditional normalization is performed on the convolutional layer parameters to distinguish physiological and disease characteristics. S104. The decision fusion unit concatenates and fuses the composite feature map with the time series feature vector, uses SVM and Softmax classifiers for parallel processing, makes decisions based on dynamic thresholds, and outputs the pest and disease identification conclusion.

[0015] On the other hand, the present application discloses an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute one or more modules of the above-mentioned forestry pest and disease intelligent monitoring system based on drone inspection.

[0016] The beneficial effects of this invention are as follows: By modifying the convolutional network structure and algorithm, this application constructs an intelligent forest pest and disease monitoring system based on drone inspections. This system enables real-time interaction with new drone data and can dynamically modify convolutional network parameters. This system effectively responds to changes in tree phenology and environmental interference, accurately extracting and distinguishing physiological changes from pest and disease characteristics, thereby improving the accuracy of forest pest and disease identification and reducing false alarm rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the intelligent monitoring method for forestry pests and diseases based on drone inspections in this application. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] This application discloses an intelligent forestry pest and disease monitoring system based on drone inspection, including: The drone terminal 100 is used to collect forestry data of the inspection target area; In practice, drones equipped with multispectral imaging sensors and high-resolution visible-light cameras simultaneously capture multispectral images (covering eight spectral bands, including red, green, blue, and near-infrared) and visible-light images of trees in the target area. The drones are equipped with a high-precision POS positioning and orientation system, which records the drone's position coordinates (latitude, longitude, and altitude) and attitude data (heading, pitch, and roll) in real time during flight, ensuring accurate temporal and spatial reference for the captured images. During flight operations, inspection altitudes are set based on the forest terrain and tree heights, using autonomously planned grid-like routes or flight paths along forest paths to ensure comprehensive coverage. Based on the phenological characteristics of different tree species, inspection frequency is dynamically increased during critical periods such as spring shoot emergence, summer pest and disease peaks, and autumn leaf fall to ensure complete time-series data. Environmental parameters, including real-time light intensity, air humidity, and wind speed, are simultaneously recorded during data collection to provide context for subsequent data processing. The collected raw data is temporarily stored in the onboard storage device and transmitted back to the server in real time through the real-time communication link according to the WebSocket protocol standard for pre-processing by the data interaction unit.

[0020] The data interaction unit 200 is used to establish a real-time communication channel between the drone and the server using the WebSocket protocol; In practice, the WebSocket protocol is used to establish a real-time communication channel between the drone and the server. The initial connection is established via a TCP three-way handshake. After negotiation via an HTTP upgrade request, a full-duplex communication link is established, supporting bidirectional real-time data transmission. A message fragmentation mechanism is employed during communication, splitting single-frame multispectral and visible light images into 1024KB segments. These segments are transmitted via sequenced message packets, ensuring stable transmission even with large data volumes.

[0021] To ensure data integrity, the communication protocol incorporates a built-in heartbeat detection mechanism. The drone sends a connection-maintaining packet to the server every two seconds. If the server fails to receive a response for more than 10 seconds, a reconnection logic is triggered, gradually attempting to restore the connection using an exponential backoff algorithm (initial interval of one second, maximum interval of 30 seconds). Transmitted data is encapsulated in a custom binary format, including data type identifiers (multispectral imagery / visible light imagery / POS data / environmental parameters), timestamps, and drone ID metadata, enabling rapid parsing and classification on the server side.

[0022] The server-side receiving module utilizes an asynchronous, non-blocking I / O model to handle concurrent connections from multiple drones. Each connection is allocated a separate buffer to temporarily store unprocessed data, preventing frame drops caused by data congestion. The data exchange process adheres to the "receive first, then process" principle. Upon reaching the server via the communication channel, raw data is first stored in a temporary cache queue in timestamp order. The data exchange unit then extracts data in batches at a rate of 10 times per second, simultaneously triggering a Min-Max normalization preprocessing process to ensure strict temporal alignment of the input data entering the convolutional network, providing consistent, standardized data input for subsequent multi-scale feature extraction.

[0023] The data interaction unit 200 is used to process data using the Min-Max normalization algorithm to provide standardized input for the convolutional network; In practice, within the drone-based forestry pest and disease intelligent monitoring system, the data exchange unit uses the Min-Max normalization algorithm to process data. The specific steps are as follows: After receiving the multispectral and visible light images from the drone, a Min-Max normalization operation is performed on the pixel values ​​of each spectral channel. First, a sliding window is constructed for each image, and the window size is set, for example, to 256×256 pixels. The maximum and minimum pixel values ​​for each spectral channel are then calculated within the window. Next, the pixel values ​​within the window are normalized using the formula (x - min) / (max - min), mapping them to the [0, 1] interval to eliminate the impact of differences in light intensity on the data.

[0024] To ensure the timeliness of the normalization parameters, the sliding window moves with a 50% overlap. Furthermore, to prevent interference from extreme values, 0.5% of abnormally high values ​​and 0.5% of abnormally low values ​​within the window are removed. When processing time series data, the normalization parameters (maximum and minimum values) for each window are retained and stored in a parameter cache queue along with the corresponding timestamp. During real-time inference, the most recent normalization parameters are retrieved from the cache queue based on the current timestamp. This is used to normalize the newly input data, ensuring strict time series alignment of the input data entering the convolutional network.

[0025] The multi-scale feature extraction unit 300 is used to construct a three-level pyramid convolution structure. Through convolution kernels of different specifications and dilated convolution, the micro-, medium-, and macro-scale features of leaves are extracted respectively. After fusion through the feature pyramid network, a composite feature map is formed, covering the full-scale morphological spectral changes of phenology and disease. In the specific implementation, the first layer adopts a parallel three-branch convolution structure, and is configured with convolution kernels of 3×3, 5×5, and 7×7 specifications respectively.

[0026] The 3×3 convolution kernel extracts microscopic features at the scale of 1-3 pixels, capturing ultramicroscopic morphological changes such as subtle lesions and early chlorotic spots on the leaf surface measuring 0.5-1.5 mm. The 5×5 convolution kernel focuses on medium-sized features at the scale of 3-5 pixels, identifying localized discoloration areas measuring 2-3 mm and early-stage small insect holes. The 7×7 convolution kernel extracts preliminary macroscopic features at the scale of 5-7 pixels, detecting early morphological differences such as mild curling of leaf edges and localized leaf shape abnormalities. Each branch is connected in series with a batch normalization (BN) layer and a ReLU activation function to standardize the input distribution and introduce nonlinear mapping.

[0027] The second layer uses dilated convolution with dilation rates, using three sets of 9×9 dilated convolution kernel branches with dilation rates of 2, 4, and 6. The dilated convolution kernel with a dilation rate of 2 has an equivalent receptive field of 17×17, extracting medium-sized features at the scale of 8-10 pixels, useful for identifying medium-sized leaf lesions measuring 3-4 mm and localized vein color abnormalities. The branch with a dilation rate of 4 achieves an equivalent receptive field of 33×33, capturing regional features at the scale of 12-16 pixels, such as localized blight patches measuring 5-8 mm and deformed leaf veins. The branch with a dilation rate of 6 creates an extremely large receptive field of 51×51, extracting complex features at the scale of 18-22 pixels, covering leaf tissue lesions measuring 8-10 mm. After the outputs of each branch are compressed by 1×1 convolution to reduce the channel dimension, they are then concatenated to form a multi-scale fused feature map, expanding feature coverage without increasing computational complexity.

[0028] The third layer uses a 15×15 convolution kernel with a stride of 2 to extract macroscopic features at the full leaf scale (over 20 pixels). This layer focuses on capturing significant features such as large-scale discoloration (e.g., yellowing exceeding 50% of the leaf surface), severe deformities (leaf curling exceeding 30%), and full leaf coverage with lesions. This layer incorporates a residual connection structure, directly adding input features to the output to avoid gradient degradation during deep network training. After the convolution operation, the feature map size is halved, increasing the number of channels to 512. After batch normalization, it is fed into the Feature Pyramid Network (FPN).

[0029] The output feature maps of the three-level convolutional structure are fused across layers using a feature pyramid network. The low-resolution macroscopic feature map of the third layer is upsampled to the mid-level scale using bilinear interpolation and semantically fused with the mid-level feature map of the second layer through element-by-element addition. The fused feature map is then upsampled again to the first-level resolution and channel-wise concatenated with the microscopic feature map of the first layer, ultimately forming a composite feature map containing 256 channels. This composite feature map covers multi-scale morphological and spectral information, ranging from a single pixel to the entire leaf. It effectively distinguishes normal physiological changes during phenological periods from abnormal characteristics caused by pests and diseases, providing multi-dimensional input for subsequent temporal feature analysis.

[0030] The specific fusion process is as follows: The three layers of feature maps output by the three-level pyramid convolutional structure (the first-layer micro-feature map, the second-layer medium-size feature map, and the third-layer macro-feature map) are input to different levels of the Feature Pyramid Network (FPN). The third-layer macro-feature map is 1 / 8 the size of the input image (due to the use of stride-2 convolution operations) and has 512 channels. It is first upsampled to the size of the second-layer feature map (1 / 4 the size of the input image) through bilinear interpolation and then element-wise added to the second-layer medium-size feature map (after adjusting the number of channels to 512 through 1×1 convolution). This fusion of high-level semantic information and mid-level regional features highlights the association between medium-sized lesions and overall morphology.

[0031] The fused mid-layer feature map is upsampled again to the first-layer feature map size (half the input image size) and concatenated with the first-layer microscopic feature map (which has been resized from an initial 256 channels to 512 channels via a 1×1 convolution), resulting in a fused feature map containing 1024 channels. This fused feature map retains the microscopic details of subtle leaf lesions and spectral reflectance anomalies in the first layer, the medium-scale features of regional discoloration and venation changes in the second layer, and the macroscopic semantics of full-leaf discoloration trends and severe deformities in the third layer, achieving cross-layer fusion of morphological features from a single pixel to the entire leaf.

[0032] For the eight spectral bands of multispectral images (red, green, blue, near-infrared, etc.), the feature pyramid network performs the above operations independently on the feature map of each band during fusion to ensure the information integrity of the spectral dimension.

[0033] For example, the fusion of macroscopic feature maps from the near-infrared band and microscopic feature maps from the visible light band preserves the near-infrared spectrum's sensitivity to vegetation health while incorporating the texture details of visible light images, forming a composite feature map encompassing multidimensional spectral and spatial information. The resulting composite feature map has the same size as the input image (restored after upsampling), with each pixel corresponding to a 1024-dimensional feature vector. This fully captures periodic leaf changes during phenological periods (such as spectral variations caused by seasonal fluctuations in chlorophyll content) and pest and disease characteristics (such as spectral anomalies at lesion edges and morphological mutations), providing a multidimensional input foundation for subsequent time series parameter adjustment modules.

[0034] Timing parameter adjustment unit 400 is used to fuse the Transformer and GRU, analyze feature maps using a multi-head self-attention mechanism, and learn tree phenology patterns using the GRU. Specifically, the composite feature map output by the multi-scale feature extraction unit (with the same size as the input image, and each pixel corresponding to a 1024-dimensional feature vector) is used as the input of the Transformer. The multi-head self-attention mechanism calculates the dependencies between different spatial positions in the feature map. Specifically, the feature map is reshaped into a sequence, and the feature vector at each position is mapped into a query, key, and value matrix. Attention weights are calculated in parallel using eight heads. Positional encoding is introduced during the weight calculation to preserve spatial information. Finally, a weighted summation is performed to obtain a feature representation that incorporates the global context, highlighting key pest and disease characteristics such as lesion edges and abnormal spectral regions. At the same time, the GRU unit receives the attention-weighted feature map at each time step and interacts with the hidden state of the previous 10 time steps. It controls the degree of retention of historical phenological features by updating gates and forgets outdated cyclical change patterns by resetting gates. This allows it to learn the normal spectral and morphological changes of trees during phenological periods such as spring shoot budding and autumn leaf drop. For example, when processing ginkgo data, the GRU analyzes 30 consecutive days of autumn spectral data to model the normal changes in chlorophyll content over time. When the change in the yellow spectral feature detected at a certain time step matches the model's prediction, it determines that the tree is in the physiological leaf drop phase.

[0035] The time series parameter adjustment unit 400 is used to conditionally normalize the convolutional layer parameters based on tree phenology, distinguishing physiological and disease characteristics. Specifically, after the GRU builds a tree phenology model by learning from historical time series data, it outputs a hidden state vector (e.g., a 128-dimensional time series feature vector) containing the characteristics of the current phenological stage. This vector is then mapped by a fully connected layer into the scale and bias factors required for conditional normalization. For example, if the GRU determines that the current chlorophyll degradation phase is normal during the autumn leaf drop of a ginkgo tree, the generated scale factor multiplies the weight matrix elements associated with the yellow spectral channel (e.g., the red band at 650nm) in the convolutional layer by 0.6 (reducing the response by 40%) and multiplies the weight of the convolution kernel for lesion edge detection (e.g., a 3×3 gradient operator) by 1.6 (increasing the response by 60%). During the specific adjustment process, the conditional normalization operation performs element-wise operations on the weight matrix W and bias vector b of each convolutional layer: W'= scale × W, b'= bias + b, where scale and bias are generated by linear transformation of the GRU hidden state, and the adjustment parameters of each spectral channel are calculated independently, thereby dynamically suppressing the interference of normal spectral changes during the phenological period and enhancing the ability to identify pest and disease characteristics.

[0036] Decision fusion unit 500 is used to concatenate and fuse the composite feature map with the temporal feature vector. Specifically, the composite feature map output by the multi-scale feature extraction unit (with the same size as the input image, and each pixel corresponding to a 1024-dimensional feature vector) is first passed through a global average pooling layer to compress the spatial dimension to 1×1, resulting in a 1024-dimensional global feature vector. This is then reduced to 128 dimensions by a fully connected layer. The resulting vector is then directly concatenated with the 64-dimensional temporal feature vector output by the temporal parameter adjustment unit (generated by linearly transforming the GRU hidden state) in the channel dimension, forming a 192-dimensional fused feature vector. For example, the concatenation of the near-infrared spectral features in the composite feature vector and the phenological cycle information in the temporal vector preserves the spectral representation of leaf health while also incorporating the temporal context of seasonal variations, providing a comprehensive feature input that fuses spatial and temporal dimensions for subsequent classifiers.

[0037] The decision fusion unit 500 utilizes parallel processing using the SVM and Softmax classifiers, outputting pest and disease identification conclusions based on dynamic threshold decisions. Specifically, the 192-dimensional fused feature vector output by the decision fusion unit is fed simultaneously into the SVM and Softmax classifiers. The SVM uses an optimal hyperplane trained on historical data to perform a binary classification of the input features, determining whether the tree currently has pest or disease abnormalities. The Softmax classifier then outputs a probability distribution of the feature vector belonging to eight common pest and disease categories. The system analyzes historical data to statistically analyze the distribution of normal physiological changes and pest and disease characteristics during different phenological periods and calculates the error rate. For example, during the spring shoot emergence period, based on historical data, the error rate of misclassifying normal reddened new leaves as disease is 15%. Therefore, the classification decision threshold is set to 0.75. When the SVM judges it as abnormal and the probability of a certain type of pest and disease output by the Softmax classifier is greater than 0.75, the system determines that this type of pest and disease exists; if the SVM judges that there is no abnormality, or the Softmax output probability is lower than the threshold, it is judged as a normal physiological change or no pest and disease, and finally the pest and disease identification conclusion is output.

[0038] It can be understood that the present application discloses an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute one or more modules of the above-mentioned forestry pest and disease intelligent monitoring system based on drone inspection.

[0039] This application also discloses an intelligent monitoring method for forestry pests and diseases based on drone inspections, see Figure 1 Including steps: S100. The drone collects forestry data from the inspection target area; S101. The data exchange unit uses the WebSocket protocol to establish a real-time communication channel between the drone and the server, and uses the Min-Max normalization algorithm to process data and provide standardized input for the convolutional network. S102. The multi-scale feature extraction unit constructs a three-level pyramid convolutional structure. Using convolution kernels of different sizes and dilated convolution, it extracts micro-, meso-, and macro-scale features of leaves. These features are then fused through a feature pyramid network to form a composite feature map covering the full-scale morphological spectral variation of phenology and disease. S103. The temporal parameter adjustment unit integrates the Transformer and GRU, analyzes feature maps through a multi-head self-attention mechanism, and uses the GRU to learn tree phenology patterns. Based on this, conditional normalization is performed on the convolutional layer parameters to distinguish physiological and disease characteristics. S104. The decision fusion unit concatenates and fuses the composite feature map with the time series feature vector, uses SVM and Softmax classifiers for parallel processing, makes decisions based on dynamic thresholds, and outputs the pest and disease identification conclusion.

[0040] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0041] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. The intelligent monitoring system for forestry pests and diseases based on drone inspection is characterized by: include: The drone side is used to collect forestry data in the inspection target area; The data interaction unit is used to establish a real-time communication channel between the drone and the server using the WebSocket protocol, process data using the Min-Max normalization algorithm, and provide standardized input for the convolutional network; The multi-scale feature extraction unit is used to construct a three-level pyramid convolution structure. Through convolution kernels of different specifications and dilated convolution, the micro-, medium-, and macro-scale features of leaves are extracted respectively. After fusion through the feature pyramid network, a composite feature map is formed, covering the full-scale morphological spectral changes of phenology and disease. The temporal parameter adjustment unit is used to integrate the Transformer and GRU. It uses a multi-head self-attention mechanism to analyze feature maps and uses the GRU to learn tree phenology patterns. Based on this, it conditionally normalizes the convolutional layer parameters to distinguish physiological and disease characteristics. The decision fusion unit is used to splice and fuse the composite feature map with the time series feature vector, use SVM and Softmax classifiers for parallel processing, make decisions based on dynamic thresholds, and output the pest and disease identification conclusion.

2. The intelligent monitoring system for forestry pests and diseases based on drone inspection according to claim 1 is characterized in that: The WebSocket protocol is used to establish a real-time communication channel between the drone and the server. The process involves the drone and server completing the initial connection via TCP, then establishing a full-duplex communication link after negotiating an HTTP upgrade request. A message fragmentation mechanism is used during communication to split single-frame multispectral and visible light images into fixed data segments, which are transmitted via message packets with sequence numbers. The communication protocol has a built-in heartbeat detection mechanism. The drone sends a connection maintenance packet to the server every 2 seconds. If the server does not receive a response for more than 10 seconds, the reconnection logic is triggered, and an exponential backoff algorithm is used to gradually attempt to restore the connection. The transmitted data is encapsulated in a custom binary format that includes data type identifiers, timestamps, and drone number metadata. The server-side receiving module adopts an asynchronous non-blocking IO model to handle concurrent connections from multiple drones. Each connection is allocated an independent buffer to temporarily store unprocessed data to avoid frame loss caused by data congestion. The data interaction process follows the "receive first, then process" principle. After the raw data reaches the server through the communication channel, it is first stored in a temporary cache queue in timestamp order. The data interaction unit then extracts it in batches at a fixed frequency, synchronously triggering the Min-Max normalization preprocessing process.

3. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: The Min-Max normalization algorithm is used to process data. Specifically, after receiving multispectral and visible light images from drones, a Min-Max normalization operation is performed on the pixel values ​​of each spectral channel. First, a sliding window is constructed for each image, and the window size is set. Then, the maximum and minimum pixel values ​​of each spectral channel are calculated within the window. Next, the pixel values ​​within the window are normalized and mapped to the interval [0, 1]. To ensure the timeliness of the normalization parameters, the sliding window moves with a fixed overlap rate. At the same time, to prevent interference from extreme values, abnormally high and low values ​​within the window are eliminated. When processing time series data, the normalization parameters of each window are retained and stored in the parameter cache queue together with the corresponding timestamp. When performing real-time inference, the most recent normalization parameters are searched from the cache queue based on the current timestamp to standardize the newly input data, ensuring that the input data entering the convolutional network remains strictly aligned in time series.

4. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: Constructing a three-level pyramid convolution structure includes: constructing a three-layer pyramid convolution structure, the first layer adopts a parallel three-branch convolution structure, and configures convolution kernels of 3×3, 5×5, and 7×7 specifications respectively; the 3×3 convolution kernel is responsible for extracting microscopic features at the scale of 1-3 pixels, which can capture the ultra-micro morphological changes of subtle lesions and early chlorotic spots on the leaf surface; the 5×5 convolution kernel is responsible for extracting medium-sized features at the scale of 3-5 pixels, which is used to identify local discoloration areas and early lesions of small insect holes; the 7×7 convolution kernel extracts preliminary macroscopic features at the scale of 5-7 pixels, and detects early morphological differences such as slight curling of leaf edges and local leaf shape abnormalities; each branch is connected in series with a batch normalization layer and a ReLU activation function to standardize the input distribution and introduce nonlinear mapping.

5. The intelligent monitoring system for forestry pests and diseases based on drone inspection according to claim 1 is characterized in that: Constructing a three-level pyramid convolution structure includes: constructing a three-layer pyramid convolution structure, the second layer uses a dilated convolution with a dilation rate, and setting three groups of 9×9 dilated convolution kernel branches with dilation rates of 2, 4, and 6; the dilated convolution kernel with a dilation rate of 2 has an equivalent receptive field of 17×17, which can extract medium-sized features at the scale of 8-10 pixels, which is used to identify medium-sized diseased areas on leaves and abnormal local vein color characteristics; the branch with a dilation rate of 4 realizes an equivalent receptive field of 33×33, capturing regional features at the scale of 12-16 pixels, such as local withering patches and deformed vein structure; the branch with a dilation rate of 6 forms an ultra-large receptive field of 51×51, extracting composite features at the scale of 18-22 pixels, covering the diseased area of ​​leaf tissue; after the output of each branch is compressed by a 1×1 convolution, a multi-scale fusion feature map is formed by channel splicing, expanding the feature coverage without increasing the amount of computation.

6. The intelligent monitoring system for forestry pests and diseases based on drone inspection according to claim 1 is characterized in that: The construction of a three-level pyramid convolutional structure involves: The third layer uses a 15×15 convolution kernel with a stride of 2 to extract macroscopic features of more than 20 pixels, capturing significant features such as large-scale discoloration, severe deformities, and full leaf lesion coverage. This layer introduces a residual connection structure, which directly superimposes input features on the output. After the convolution operation, the feature map size is halved, and the number of channels is increased to 512. After batch normalization, it is input into the feature pyramid network. The output feature maps of the three-level convolutional structure are cross-layer fused through a feature pyramid network: the low-resolution macro feature map of the third layer is upsampled to the middle-layer size through bilinear interpolation, and semantically fused with the medium-sized feature map of the second layer through element-by-element addition; the fused feature map is upsampled to the first-layer resolution again, and channel-wise spliced ​​with the micro feature map of the first layer to finally form a composite feature map containing 256 channels; this composite feature map covers multi-scale morphological and spectral information from 1 pixel to the entire leaf, effectively distinguishing normal physiological changes in the phenological period from abnormal characteristics caused by pests and diseases.

7. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: The fusion of Transformer and GRU includes: taking the composite feature map as the input of Transformer, calculating the dependency of different spatial positions in the feature map through the multi-head self-attention mechanism; reshaping the feature map into a sequence form, mapping the feature vector of each position into query, key and value matrices, and calculating the attention weights in parallel through 8 heads. Position encoding is introduced during the weight calculation to retain spatial information, and finally the feature representation of the fused global context is obtained through weighted summation to determine the key features of pests and diseases at the edge of the lesion and the abnormal spectral area; at the same time, the GRU unit receives the attention-weighted feature map of each time step and interacts with the hidden states of the previous several time steps. It controls the degree of retention of historical phenological features by updating the gate, and forgets the outdated periodic change pattern by resetting the gate, thereby learning the normal spectral and morphological change patterns of trees during the spring shoot germination and autumn leaf fall phenological period.

8. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: Conditional normalization adjustment of convolutional layer parameters based on tree phenology includes: after the GRU establishes a tree phenology model by learning historical time series data, it outputs a hidden state vector containing the characteristics of the current phenological stage. This vector is mapped into the scaling factor and offset factor required for conditional normalization through the fully connected layer; the weight matrix and bias vector of each convolutional layer are adjusted by the scaling factor and offset factor.

9. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: The splicing and fusion of the composite feature map and the time series feature vector includes: the composite feature map output by the multi-scale feature extraction unit is first passed through the global average pooling layer to compress the spatial dimension to 1×1 to obtain a 1024-dimensional global feature vector; then it is reduced to 128 dimensions through the fully connected layer, and directly spliced ​​with the 64-dimensional time series feature vector output by the timing parameter adjustment unit in the channel dimension to form a 192-dimensional fused feature vector.

10. The intelligent forest pest monitoring system based on drone inspection according to claim 1 is characterized in that: Using SVM and Softmax classifiers for parallel processing, and based on dynamic threshold decisions, the output of pest and disease identification conclusions includes: the 192-dimensional fusion feature vector output by the decision fusion unit is input into the SVM and Softmax classifiers at the same time; the SVM performs binary classification on the input features based on the optimal hyperplane obtained by historical data training to determine whether the current tree has abnormal pests and diseases; the Softmax classifier outputs the probability distribution of the feature vector belonging to several common pest and disease categories; by analyzing historical data, the distribution of normal physiological changes and pest and disease characteristics under different phenological periods is statistically analyzed, and the error rate is calculated; when the SVM judges it to be abnormal and the probability of a certain type of pest and disease output by the Softmax classifier is greater than the threshold, the system determines that this type of pest and disease exists; if the SVM judges that there is no abnormality, or the Softmax output probability is lower than the threshold, it is judged to be normal physiological changes or no pests and diseases, and finally the pest and disease identification conclusion is output.

Images