Forestry intelligent spraying system and method based on multi-source sensor space perception

Abstract

The invention discloses a forestry intelligent spraying system and spraying method based on multi-source sensor space perception, and relates to the technical field of intelligent control. A target forest region is scanned through a laser radar and a visual sensor carried by an unmanned aerial vehicle, a three-dimensional forest region map is constructed, and a single tree is identified and positioned by using a tree body identification model; evaluating the leaf density of the canopy; identifying diseases and insect pests by utilizing multispectral imaging, and making a pesticide proportioning decision according to the disease and insect pest types and severity; planning and generating an optimal flight path for spraying of the unmanned aerial vehicle; adjusting nozzle parameters according to the leaf density of the canopy and the severity of diseases and pests, performing variable spraying, and storing the spraying operation parameters of the unmanned aerial vehicle. Through multi-source sensor fusion, an intelligent decision algorithm and precise spraying control, autonomous obstacle avoidance, high-precision map construction, tree body recognition and canopy analysis, pest and disease damage detection and dynamic pesticide dispensing and spraying in a forestry scene are realized, the forestry spraying efficiency is remarkably improved, and pesticide waste is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and more specifically, to a forestry intelligent spraying system and spraying method based on multi-source sensor spatial perception. Background Art

[0002] In forestry production, unmanned aerial vehicle (UAV) spraying technology has been widely applied to operations such as pollination, pest control, and nutrient supplementation. Traditional UAV spraying mainly relies on the downward airflow generated by the rotor to spray the medicament or pollen onto the top of the tree canopy. However, due to the complex structure and dense branches and leaves of the forest canopy, it is difficult for the liquid droplets of the medicament to penetrate the upper branches and leaves, resulting in insufficient coverage of the middle and lower layers and side branches of the tree body, seriously affecting the control or pollination effect. In addition, traditional UAV operations usually rely on preset flight routes or manual control, lacking the ability to perceive the spatial structure of the forest stand in real time, and it is difficult to dynamically adjust the spraying strategy according to the shape of the tree canopy, resulting in problems such as waste of medicament or missed spraying.

[0003] At present, some studies have tried to improve the droplet penetration by optimizing UAV flight parameters (such as flight height, speed) or improving nozzle design (such as increasing atomization pressure, adjusting spray angle). However, limited by the canopy occlusion effect, it is difficult to achieve precise and uniform three-dimensional spraying solely by mechanical adjustment. In addition, some forestry UAVs are equipped with lidar (LiDAR) or visible light cameras for canopy recognition, but the adaptability of a single sensor under complex lighting conditions is limited, and it is difficult to accurately reconstruct the three-dimensional structure of the tree crown and identify the internal spraying blind areas.

[0004] Therefore, there is an urgent need to develop a forestry intelligent spraying system based on multi-source sensor fusion. By combining sensing technologies such as lidar, multispectral imaging, and millimeter-wave radar, it is possible to achieve high-precision perception of the spatial morphology of the tree canopy, and use intelligent algorithms to dynamically plan the spraying path and parameters to ensure the uniform distribution of the medicament or pollen in each layer of the tree body, thereby improving the efficiency and ecological benefits of forestry operations. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a forestry intelligent spraying system and spraying method based on multi-source sensor spatial perception, which can significantly improve the efficiency of forestry spraying, reduce the waste of medicament, and achieve intelligent and precise operations for different tree structures.

[0006] The first aspect of the present invention provides a forestry intelligent spraying system based on multi-source sensor spatial perception, including a multi-source sensor perception module, an environmental modeling and map construction module, a tree recognition and canopy analysis module, a pest and disease recognition and intelligent medicine dispensing module, an autonomous path planning and obstacle avoidance module, and a precise spraying control module; The multi-source sensor perception module is responsible for collecting multi-source perception data through preset sensors and preprocessing the collected multi-source perception data; The environmental modeling and map construction module combines LiDAR, vision, and GPS data to generate a 3D forestry map, and identifies obstacle information for annotation in the 3D forestry map; The tree identification and canopy analysis module constructs a tree identification model to identify individual trees, calculates the height and canopy range, evaluates the density of branches and leaves in the canopy, and analyzes and generates a droplet penetration strategy; The pest and disease identification and intelligent drug dispensing module uses multispectral imaging combined with deep learning to identify pests and diseases, makes decisions on drug formulation ratio and variable spraying control according to the type and severity of pests and diseases; The autonomous path planning and obstacle avoidance module performs global planning of the flight path and local dynamic obstacle avoidance based on the 3D forestry map, and dynamically adjusts the altitude of the UAV according to the tree height; The precise spraying control module dynamically adjusts the nozzle parameters according to the environmental perception data, and stores the spraying data of each tree.

[0007] In this solution, in the environmental modeling and map construction module, LiDAR, vision, and GPS data are combined to generate a 3D forestry map, and obstacle information is identified for annotation in the 3D forestry map. Specifically: Obtain the preprocessed LiDAR point cloud data, vision data, and GPS data, extract edge points and plane points from the LiDAR point cloud data, select feature points based on curvature analysis, match adjacent frames according to the feature points, and establish an association relationship between the feature points of the current frame and the previous frame; Calculate the relative pose between adjacent frames through normal distribution transformation, and use IMU data for pose rationality verification to obtain a local point cloud map as a matching benchmark; Extract key points from the vision data, calculate descriptors for each key point, use descriptor matching for adjacent frame matching, and perform reprojection matching with the feature points in the local point cloud map, and optimize the pose according to the matching points combined with IMU data; Obtain the UAV pose information as nodes, construct a pose graph with sensor observation constraints as edges, match historical scenes through visual bag-of-words and LiDAR point clouds, perform loop detection, optimize the pose graph using the least squares method, and when GPS data is available, use the GPS position for global correction to obtain a corrected pose graph; Align the pose nodes in the pose graph with the corresponding sensor data time, perform coordinate transformation on the point cloud data, generate a dense 3D point cloud and convert the point cloud into an octree structure to construct a 3D forestry map; Use a 3D CNN network to identify obstacle information, use the obstacle information for target annotation, project the 3D point cloud in the 3D forestry map onto a 2D plane to generate an occupancy grid map, and identify the flyable area.

[0008] In this solution, in the tree recognition and canopy analysis module, a tree recognition model is constructed to recognize individual trees. Specifically: Synchronize the preprocessed LiDAR point cloud data and multi-spectral image data, construct a tree recognition model, use the PointNet++ network to set up a point cloud feature extraction branch, perform point cloud feature extraction in the 4 SA modules set, and realize feature upsampling through interpolation, gradually fusing point cloud features of different scales; Use the MobileNetV3 network to set up an image feature projection branch, obtain the global feature vector corresponding to the multi-spectral image as the image feature, project the image feature into the point cloud space, and splice it with the point cloud feature for feature association; Introduce an attention mechanism in the tree recognition model for bimodal feature fusion, perform weighted fusion on the associated point cloud features and image features, screen seed points according to the fusion features, adaptively adjust the preset multi-scale anchor boxes, encode the neighborhood features of each seed point, and predict and generate anchor boxes and anchor box categories; Obtain the geometric features and spectral features of each three-dimensional point in the anchor box as point features, calculate the covariance matrix for each point feature, calculate the Mahalanobis distance from the covariance matrix to the point cloud instance of the corresponding category of trees, and use Softmax for normalization processing to achieve probability conversion; Obtain the three-dimensional points that meet the probability threshold standard for deterministic assignment, mark them as tree instance members, obtain the three-dimensional points that do not meet the probability threshold standard, use the point features to obtain the classification score for tree classification, and if the classification score is greater than the preset threshold, mark them as tree instance members; Use the set of tree instance members to segment the trees in the anchor box and output the segmentation results of individual trees.

[0009] In this solution, calculate the height and canopy range, evaluate the density of branches and leaves in the canopy, and analyze and generate a fog droplet penetration strategy. Specifically: Select the point cloud positioning root point according to the recognition and segmentation results of individual trees, measure the tree height based on the root point, perform clustering analysis on the point cloud corresponding to the individual tree, and select the initial clustering center according to the point cloud distribution in combination with the tree point cloud instance; Calculate the Euclidean distance from each point cloud in the point cloud corresponding to the individual tree to the initial clustering center, assign each point cloud to the nearest initial clustering center to generate a clustering cluster, and through iterative clustering and updating of the clustering center, select the clustering cluster of the last clustering as the clustering result; Mark the point clouds involved in the canopy cluster according to the clustering result, and obtain the canopy range according to the horizontal projection and vertical stratification to extract the point cloud distribution of the marked point clouds; Perform three-dimensional grid division on the point cloud corresponding to the canopy range, calculate the point cloud density in each grid to quantify the spatial distribution characteristics of branches and leaves, and evaluate the density of branches and leaves according to the point cloud density distribution within the canopy range in combination with the preset density level division standard; Based on the canopy types with different degrees of branch and leaf density in the region, match spraying examples based on the canopy types. By counting the droplet size intervals with usage frequencies meeting the preset standards in the matching results, use the droplet size intervals as the recommended particle sizes to generate a droplet penetration strategy.

[0010] In this solution, in the pest and disease identification and intelligent medicine dispensing module, multi-spectral imaging combined with deep learning is used to identify pests and diseases. Specifically: Extract spectral features based on the preprocessed multi-spectral data, use an improved genetic algorithm to optimize the combination of spectral features, evaluate the importance of the feature combination using the recognition classification accuracy, and evaluate the redundancy of the feature combination according to the Pearson correlation coefficient; Calculate the fitness with the goal of maximizing importance and minimizing redundancy, select the spectral feature combination with the highest fitness for replication to determine the search direction, and perform feature recombination through crossover to accelerate the optimization selection process. When the termination condition is met, obtain the best spectral feature combination according to the optimal solution; Build a lightweight pest and disease identification model based on the EfficientDet-Lite model. Use the best spectral feature combination as the input of the pest and disease identification model, adopt EfficientNet-B3 as the backbone network, and introduce lightweight channel attention to obtain multi-spectral features; Reduce the number of BiFPN layers to 4 layers, generate anchor box adaptations for disease spots using the multi-spectral features, and predict the pest and disease types and damage stages corresponding to the anchor boxes through a pre-trained classification head.

[0011] In this solution, make a decision on the pesticide ratio and control variable spraying according to the pest and disease types and severity. Specifically: Extract the corresponding disease spot area according to the predicted anchor box, and project the spectral features and image features of the disease spot area onto a low-dimensional space using the t-SNE dimensionality reduction algorithm to generate a feature scatter plot; Retrieve pest and disease instances with severity labels annotated by using the pest and disease types and damage stages. Use the Wasserstein distance to quantify the distribution difference between the predicted anchor box and the feature scatter plot of the pest and disease instances, obtain the pest and disease instances with distribution differences meeting the preset difference threshold, and determine the severity according to the selected pest and disease instances; Construct a pest and disease portrait by combining the pest and disease type and severity with environmental characteristics, perform feature embedding encoding on the pest and disease portrait to generate corresponding pest and disease nodes, severity nodes, and environmental feature nodes, obtain corresponding spatio-temporal associations based on lesion adjacency and lesion state transitions, and use the nodes and spatio-temporal associations to construct a topological relationship graph; Use a graph convolutional network to perform representation learning on the topological relationship graph, use a message passing and neighbor aggregation mechanism to obtain corresponding graph embedding vectors, and perform KNN search in the domain knowledge graph through the graph embedding vectors to obtain a preset number of valid cases; Input a preset number of valid cases into a pre-trained multi-task learning head, obtain the probability distribution of the pesticide category and the dose prediction value to generate a pesticide formulation decision result, and determine the droplet size classification result and the nozzle angle regression result based on the droplet penetration strategy to generate a variable spraying control strategy.

[0012] In this solution, in the precise spraying control module, the nozzle parameters are dynamically adjusted according to the environmental perception data, and the spraying data of each tree is stored. Specifically: Obtain multi-source environmental perception data and spraying quality parameters during the UAV spraying process, construct a mapping relationship between the multi-source environmental perception data and the spraying quality parameters based on a neural network, and train a data mapping model under normal spraying quality; Import the real-time multi-source environmental perception data into the data mapping model under normal spraying quality for estimation, calculate the residual between the estimated multi-source environmental perception data and the real-time multi-source environmental perception data. When the residual is greater than the preset residual threshold, re-obtain the variable spraying control strategy and dynamically adjust the nozzle parameters; Standardize the single-tree basic data, spraying status data, and multi-source environmental perception data and store them in the database. Use the database to optimize the generation of the variable spraying control strategy and monitor the faults of the components of the UAV spraying device.

[0013] The second aspect of the present invention provides a forestry intelligent spraying method based on multi-source sensor spatial perception, including the following steps: Scan the target forest area through the lidar and visual sensors carried by the UAV, construct a three-dimensional forest map, use a tree recognition model to identify and locate individual trees, analyze the canopy structure, and evaluate the leaf density of the canopy; Use multi-spectral imaging combined with deep learning to identify pests and diseases, and make a pesticide formulation decision according to the pest and disease type and the severity of the pests and diseases; Based on the three-dimensional forest map, perform global planning of the flight path and local dynamic obstacle avoidance to generate the optimal flight path for UAV spraying; Adjust the nozzle parameters according to the leaf density of the canopy and the severity of the pests and diseases, perform variable spraying, and store the UAV spraying operation parameters.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By integrating multi-source sensor data, artificial intelligence algorithms, and precise control technology, this system achieves intelligent operation throughout the entire process, from environmental perception to variable-rate spraying. Through the collaborative perception of LiDAR and multispectral imaging, the system can achieve centimeter-level positioning of individual trees, ensuring that every spray target is reached. Adaptive droplet control based on three-dimensional canopy density analysis improves the uniformity of pesticide deposition across the upper, middle, and lower layers of the canopy, dynamically adjusting pesticide dosage based on lesion distribution, significantly improving control effectiveness and reducing pesticide waste.

[0015] From pest identification and pesticide dispensing decisions to spraying control, this system achieves unmanned, closed-loop operations. It also supports blockchain-based storage of individual tree-level records, meeting organic forestry certification requirements. Furthermore, this invention significantly reduces manual intervention, allowing drones to completely replace high-risk operations (such as those on steep slopes and in hazardous environments), improving both economic efficiency and social value. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or exemplary descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to these drawings without paying any creative work.

[0017] Figure 1 The block diagram of the forestry intelligent spraying system based on multi-source sensor spatial perception is shown.

[0018] Figure 2 A flowchart of constructing a tree recognition model to identify individual trees is shown in the embodiment; Figure 3 A flowchart illustrating the use of multispectral imaging combined with deep learning to identify pests and diseases is shown in an embodiment; Figure 4 A flow chart of a forestry intelligent spraying method based on multi-source sensor spatial perception is shown; DETAILED DESCRIPTION

[0019] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0020] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0021] Figure 1 A block diagram of a forestry intelligent spraying system based on multi-source sensor spatial perception is shown.

[0022] As Figure 1 shown, in the first embodiment of the present invention, a forestry intelligent spraying system 1 based on multi-source sensor spatial perception is provided, including a multi-source sensor perception module 101, an environmental modeling and map construction module 102, a tree recognition and canopy analysis module 103, a pest and disease recognition and intelligent drug dispensing module 104, an autonomous path planning and obstacle avoidance module 105, and a precise spraying control module 106; The multi-source sensor perception module 101 is responsible for collecting multi-source perception data through preset sensors and preprocessing the collected multi-source perception data; The environmental modeling and map construction module 102 combines LiDAR, vision, and GPS data to generate a three-dimensional forestry map and identifies obstacle information for annotation in the three-dimensional forestry map; The tree recognition and canopy analysis module 103 constructs a tree recognition model to identify individual trees, calculates the height and canopy range, evaluates the density of branches and leaves in the tree crown, and analyzes and generates a droplet penetration strategy; The pest and disease recognition and intelligent drug dispensing module 104 uses multi-spectral imaging combined with deep learning to identify pests and diseases, makes a decision on drug ratio and variable spraying control according to the type and severity of pests and diseases; The autonomous path planning and obstacle avoidance module 105 performs global planning of the flight path and local dynamic obstacle avoidance based on the three-dimensional forestry map, and dynamically adjusts the height of the unmanned aerial vehicle according to the tree height; The precise spraying control module 106 dynamically adjusts the nozzle parameters according to the environmental perception data and stores the spraying data of each tree.

[0023] It should be noted that the multi-source sensor perception module collects environmental data, provides information support for path planning, tree recognition, and pest and disease detection, and performs preprocessing such as calibration and fusion of multi-source perception data to improve the positioning and mapping accuracy. For example, point cloud denoising, visual feature extraction, and GPS and IMU data fusion. The multi-source sensor perception module includes a LiDAR lidar, a multi-spectral / hyperspectral camera, a millimeter-wave radar, an RGB-D depth camera, an RTK-GPS, an IMU, a wind speed sensor, a temperature and humidity sensor, etc. The LiDAR lidar scans the surrounding environment in real time to construct a high-precision three-dimensional point cloud map for obstacle detection, tree crown modeling, and spraying distance calculation, etc.; the multi-spectral / hyperspectral camera identifies the vegetation health status and detects pest and disease areas (such as leaf surface lesions and pest characteristics) to assist in intelligent drug dispensing decisions; the millimeter-wave radar compensates for the perception ability of the LiDAR in bad weather such as fog, rain, and snow, and enhances the robustness of obstacle detection; the RGB-D depth camera assists in the analysis of the tree crown structure at close range to improve spraying accuracy; the RTK-GPS and IMU provide centimeter-level positioning to ensure the precise flight and map matching of the drone in complex forest areas; preset sensors such as the wind speed sensor and the temperature and humidity sensor collect multi-source environmental perception parameters in real time to provide a data basis for adjusting spraying parameters.

[0024] It should be noted that in the environmental modeling and map construction module, the preprocessed LiDAR point cloud data, visual data, and GPS data are obtained. Edge points and plane points are extracted from the LiDAR point cloud data, and feature points are selected based on curvature analysis. Adjacent frames are matched according to the feature points, and an association relationship is established between the feature points of the current frame and the feature points of the previous frame. The relative pose between adjacent frames is calculated through normal distribution transformation, and the IMU data is used for pose rationality verification to obtain a local point cloud map as a matching reference. Key points are extracted from the visual data using algorithms such as FAST, ORB, or SIFT, descriptors are calculated for each key point, adjacent frames are matched using descriptor matching, and reprojection matching is performed with the feature points in the local point cloud map. The camera pose is solved using the PnP solution method based on the matching points, and the pose is optimized through visual reprojection error and combined with IMU data. Preferably, during the environmental modeling and map construction process, an adaptive weight allocation of the sensors is set. Through the adaptive weight allocation, LiDAR is mainly used in feature-rich areas, visual constraints are enhanced in open areas, the weight is automatically reduced in low light or motion blur, and the cumulative error is eliminated periodically using the GPS absolute position. The UAV pose information is obtained as nodes, and the sensor observation constraints (such as LiDAR matching, GPS measurement, etc.) are used as edges to construct a pose graph. Closed-loop detection of the historical scene is performed through DBoW2 or NetVLAD, and fast scene recognition is performed using LiDAR point cloud matching to detect whether it returns to the explored area and the LiDAR point cloud matching historical scene for closed-loop detection. The least squares method is used to optimize the pose graph to eliminate the cumulative error. When GPS data is available, the GPS position is used for global correction to obtain a corrected pose graph to avoid drift problems during large-scale mapping. Each pose node in the pose graph is time-aligned with the corresponding sensor data, the coordinate transformation of the point cloud data is performed, a dense three-dimensional point cloud is generated, and the point cloud is converted into an octree structure to construct a three-dimensional forestry map to support efficient storage and dynamic update. The 3D CNN network is used to identify obstacle information such as utility poles and rocks, and the obstacle information is used for target annotation. The three-dimensional point cloud in the three-dimensional forestry map is projected onto a two-dimensional plane to generate an occupancy grid map to identify the flyable area. Preferably, moving objects are removed through time series analysis or clustering algorithms for dynamic point cloud filtering. Different passage costs are set according to the tree density and obstacle distribution to generate a cost map for flight path planning.

[0025] Figure 2 FIG. shows the flow chart of identifying a single tree by constructing a tree recognition model in the embodiment.

[0026] According to an embodiment of the present invention, in the tree recognition and canopy analysis module, a tree recognition model is constructed to identify a single tree, specifically: S202. Synchronize the preprocessed LiDAR point cloud data and multi-spectral image data, construct a tree recognition model, use the PointNet++ network to set up a point cloud feature extraction branch, perform point cloud feature extraction in the set of 4 SA modules, and realize feature upsampling through interpolation, and gradually fuse point cloud features of different scales; S204. Use the MobileNetV3 network to set up an image feature projection branch, obtain the global feature vector corresponding to the multi-spectral image as the image feature, project the image feature into the point cloud space, and splice it with the point cloud feature for feature association; S206. Introduce an attention mechanism in the tree recognition model for bimodal feature fusion, perform weighted fusion on the associated point cloud feature and image feature, screen seed points according to the fusion feature, adaptively adjust the preset multi-scale anchor boxes, encode the neighborhood features of each seed point, and predict and generate anchor boxes and anchor box categories; S208. Obtain the geometric features and spectral features of each three-dimensional point in the anchor box as point features, calculate the covariance matrix for each point feature, calculate the Mahalanobis distance from the covariance matrix to the point cloud instance of the corresponding category of trees, and use Softmax for normalization processing to achieve probability conversion; S210. Obtain the three-dimensional points that meet the probability threshold standard for deterministic assignment, mark them as tree instance members, obtain the three-dimensional points that do not meet the probability threshold standard, use the point features to obtain the classification score for tree classification, and if the classification score is greater than the preset threshold, mark them as tree instance members; S212. Use the tree instance member set to segment the trees in the anchor box and output the segmentation results of individual trees.

[0027] It should be noted that an end-to-end double-branch feature extraction framework is constructed as the tree recognition model. In the point cloud feature extraction branch, the PointNet++ hierarchical sampling structure is adopted, 4 SA (Set Abstraction) modules are set for feature extraction, and the FP (Feature Propagation) module is used to realize feature upsampling through interpolation, and gradually fuse features of different scales. In the image feature projection branch, the lightweight MobileNetV3 is used to process the multi-spectral image, and the global feature vector is output. The image global feature vector is projected into the 3D point space for feature association. The image feature is merged at the SA1 layer of PointNet++, and weighted fusion is performed through the attention mechanism in the FP stage. The fusion feature is used to screen candidate points. For example, points with a height > 1.5m and a local curvature > 0.3 are screened according to geometric conditions, and points with an NDVI > 0.6 are screened according to spectral conditions. The farthest point sampling (FPS) is performed based on the candidate points to obtain the final seed points.

[0028] Set multi-scale anchor boxes according to tree species types, dynamically scale the sizes of the anchor boxes according to the local point cloud density, automatically rotate the directions of the anchor boxes for inclined tree trunks, extract geometric features and spectral features for the neighborhood of each seed point, including: height variance, plane fitting residuals, multi-spectral means, variances, etc., and predict and generate anchor boxes and anchor box categories through a regression network. Preferably, divide the generated anchor boxes into several regular grids, capture significant structural features and retain the overall distribution information through maximum pooling and average pooling to obtain aggregated features; project the anchor boxes onto the multi-spectral images for multi-modal feature fusion, use ROI Align to extract the corresponding regional features, introduce a self-attention mechanism to output a weight matrix to reveal key structural points, and use a classification branch and a regression branch to output tree categories and anchor boxes to achieve the optimization of anchor box prediction and improve the prediction accuracy.

[0029] Obtain the geometric features and spectral features of each three-dimensional point in the anchor box as point features, calculate the covariance matrix for each point feature, where the geometric features include the relative coordinates of the point to the center of the anchor box, the minimum distance from the point to each surface of the anchor box, the angle between the point normal vector and the main direction of the anchor box, etc., and the spectral features include multi-spectral reflectance, vegetation index, texture features, etc. Use the tree instance member set to segment the trees in the anchor box and add morphological constraints, such as prohibiting the generation of isolated single-point regions and forcing the minimum volume of connected regions, etc.

[0030] It should be noted that based on the recognition and segmentation results of individual trees, the point cloud is used to locate the root base point. For example, the average value of the Z values of the lowest 1% points of the point cloud is taken as the ground reference, and the first layer with a point density > 50 pts / 0.1m is searched upward along the Z axis as the root base surface. Based on the root base point, the tree height is measured. Cluster analysis is performed on the point cloud corresponding to the individual tree. According to the point cloud distribution and combined with the tree point cloud examples, the initial cluster centers are selected. The initial distance centers are located in different tree structures, such as the crown part, the trunk part, etc.; the Euclidean distance from each point cloud in the point cloud corresponding to the individual tree to the initial cluster center is calculated, and each point cloud is assigned to the nearest initial cluster center to generate cluster clusters. Through iterative clustering and updating the cluster centers, the cluster clusters of the last clustering are selected as the clustering result, and each cluster cluster corresponds to a different tree structure. According to the clustering result, the point clouds involved in the canopy cluster are marked, and the canopy range is obtained by extracting the point cloud distribution of the marked point cloud according to the horizontal projection and vertical stratification; the point cloud corresponding to the canopy range is divided into three-dimensional grids, and the point cloud density is calculated in each grid to quantify the spatial distribution characteristics of the branches and leaves. According to the point cloud density distribution within the canopy range and combined with the preset density level division standard, the density of the branches and leaves is evaluated. The preset density levels are divided into sparse, medium, and dense, corresponding to different point cloud density ranges respectively; according to the different canopy types in the area of the branch and leaf density, the spraying examples are matched based on the canopy type. By counting the droplet size intervals with the usage frequency meeting the preset standard in the spraying examples according to the matching result, the droplet size interval is used as the recommended particle size to generate the droplet penetration strategy. For example, for a sparse canopy, the recommended particle size is 150 - 200 μm, and small droplets are prone to drift in a sparse canopy; for a medium canopy, the recommended particle size is 200 - 250 μm to balance penetration and deposition; for a dense canopy, the recommended particle size is 250 - 350 μm, and large droplets have anti-drift properties.

[0031] Figure 3 Fig. shows the flow chart of identifying pests and diseases by using multi-spectral imaging combined with deep learning in the embodiment.

[0032] According to the embodiment of the present invention, in the pest and disease identification and intelligent drug dispensing module, multi-spectral imaging combined with deep learning is used to identify pests and diseases, specifically: S302, extract spectral features based on the preprocessed multi-spectral data, use an improved genetic algorithm to optimize the combination of spectral features, evaluate the importance of the feature combination using the recognition classification accuracy, and evaluate the redundancy of the feature combination according to the Pearson correlation coefficient; S304, calculate the fitness with the goal of maximizing the importance and minimizing the redundancy, select the spectral feature combination with the highest fitness for replication to determine the search direction, and perform feature recombination through crossover to accelerate the optimization selection process. When the termination condition is met, obtain the best spectral feature combination according to the optimal solution; S306. Build a lightweight pest and disease identification model based on the EfficientDet-Lite model. Use the optimal spectral feature combination as the input of the pest and disease identification model. Adopt EfficientNet-B3 as the backbone network and introduce lightweight channel attention to obtain multispectral features. S308. Reduce the number of BiFPN layers to 4. Generate anchor box adaptations for disease spots using the multispectral features, and predict the types and damage stages of pests and diseases corresponding to the anchor boxes through a pre-trained classification head.

[0033] It should be noted that spectral features such as NDVI, PRI, GLI, etc., as well as combinations such as ratio (R / NIR) and difference (NIR - R), are extracted based on the preprocessed multispectral data. Initialize the genetic algorithm with binary encoding. In each iteration, directly retain the top 5% of individuals with fitness to enter the next generation. Construct a fitness function through a precision term and a redundancy term. In the precision term, use the 10-fold cross-validation accuracy of the SVM classifier. In the redundancy term, calculate the Pearson correlation coefficient (absolute value mean) to characterize feature redundancy, and use weight coefficients to integrate the precision term and the redundancy term. During the genetic evolution process, perform two-point crossover on individuals with medium and high fitness, and increase the mutation probability on redundant feature bits. When the maximum number of iterations is reached or the fitness improvement is <1% for 20 consecutive generations, the termination condition is satisfied.

[0034] Build a lightweight pest and disease identification model with EfficientNet-B3 as the benchmark model. Increase the width coefficient to 1.2, increase the number of channels but reduce the network depth. Introduce lightweight channel attention to replace the original SE module, and use one-dimensional convolution for cross-channel interaction without dimensional compression, significantly reducing the computational load; reduce the number of BiFPN layers to 4, retain the key scales: P3(1 / 8), P5(1 / 32), P7(1 / 128), add P4(1 / 16) as an intermediate transition, and perform cross-scale connection during downsampling, and perform fast normalization weighting to further improve the computational speed. The classification head and the regression head share the first 3 convolutional layers, and only the last layer has an independent branch. In the regression head, adjust the anchor box density according to the NDVI value, and preferentially add 45° anchor boxes according to the sensitive direction of the disease spot; in the classification head, use the damage characteristics of pest and disease categories such as leaf spot, rust, and insect damage for pre-training. The feature reuse of the classification head and the regression head reduces the parameters while ensuring the accuracy, meeting the real-time processing requirements of the drone.

[0035] It should be noted that according to the predicted anchor boxes, the corresponding lesion areas are extracted, and the spectral features and image features of the lesion areas are projected into a low-dimensional space using the t-SNE dimensionality reduction algorithm to generate a feature scatter plot; pest and disease instances with severity label annotations are retrieved using the pest and disease type and damage stage, the Wasserstein distance is used to quantify the distribution difference between the predicted anchor boxes and the corresponding feature scatter plots of the pest and disease instances, pest and disease instances with distribution differences meeting the preset difference threshold are obtained, and the severity is determined according to the selected pest and disease instances; a pest and disease portrait is constructed by combining the pest and disease type, severity, and environmental characteristics, the pest and disease portrait is encoded by feature embedding to generate corresponding pest and disease nodes, severity nodes, and environmental feature nodes, the corresponding spatio-temporal associations are obtained according to lesion adjacency and lesion state transitions, and a topological relationship graph is constructed using the nodes and spatio-temporal associations; a graph convolutional network is used to perform representation learning on the topological relationship graph, a message passing and neighbor aggregation mechanism is used to obtain corresponding graph embedding vectors, KNN search is performed in the domain knowledge graph through the graph embedding vectors, and a preset number of effective cases are obtained according to the control effect; the preset number of effective cases are input into a pre-trained multi-task learning head, environmental data is introduced in the multi-task learning head for environmental adaptation, a probability distribution of chemical agent categories and a dose prediction value are obtained to generate a chemical agent ratio decision result, and a variable spraying control strategy is generated based on the droplet penetration strategy to determine the droplet size classification result and the nozzle angle regression result. Preferably, the chemical agent ratio decision result is adjusted according to the drug resistance record data, the actual control effect of the chemical agent ratio decision result is recorded, the edge weight of the graph is updated, and digital twins are used to simulate different strategies in the virtual canopy to optimize the hyperparameters of the graph convolutional network.

[0036] Configure the drone using the decision result of the chemical agent ratio and the variable spraying control strategy. Based on the adaptive droplet regulation analysis of the three-dimensional canopy density, improve the deposition uniformity of the chemical agent in the upper, middle, and lower layers of the tree crown. Dynamically adjust the chemical agent dosage according to the distribution of disease spots, and globally plan the flight path and perform local dynamic obstacle avoidance based on the three-dimensional forestry map using Stacking ensemble learning. Extract environmental features, including: terrain complexity (elevation variance + obstacle density), stand parameters (average tree height, crown width dispersion), meteorological records (wind speed, light time series data), etc., and vectorize the task parameters such as chemical agent dosage, droplet size requirements and resolution, and overlap rate requirements. Dynamically adjust the safety height according to the voxelized tree crown model, calculate the maximum tilt angle based on the DEM, and set spatial constraints; ensure the coverage integrity according to the path spacing, and train three base models corresponding to the A* algorithm, genetic algorithm, and RRT* algorithm. The output of each algorithm is uniformly converted into a 100×3 path matrix, and the stacked path matrix is used as the meta-dataset to train three meta-models, including the A* algorithm, genetic algorithm, and RRT* algorithm. Use the flight path sequences output by different meta-models as the input of the attention mechanism network, and weight the paths of the original models according to the current environmental features. For example, set the A* weight > 0.7 during emergency obstacle avoidance, set the genetic algorithm weight > 0.6 during long-distance cruising, and set the RRT* weight > 0.5 in the dense obstacle area. Combine the flight path sequences predicted by the meta-models with the weight information to generate the global planning of the flight path of the Stacking ensemble model, and combine algorithms such as the dynamic window method and artificial potential field method for local dynamic obstacle avoidance to generate the optimal flight path for drone spraying.

[0037] It should be noted that multi-source environmental perception data and spraying quality parameters during the drone spraying process are obtained, and the mapping relationship between the multi-source environmental perception data and the spraying quality parameters is constructed based on the neural network, and the data mapping model under normal spraying quality is trained; the real-time multi-source environmental perception data is imported into the data mapping model under normal spraying quality for prediction, and the residual between the predicted multi-source environmental perception data and the real-time multi-source environmental perception data is calculated. When the residual is greater than the preset residual threshold, the variable spraying control strategy is re-obtained, and the nozzle parameters are dynamically adjusted; the single-tree basic data, spraying status data, and multi-source environmental perception data are standardized and stored in the database to achieve full-process digital traceability, such as tree ID, positioning coordinates (longitude + latitude + elevation), spraying timestamp, actual chemical agent dosage (divided into upper, middle, and lower layers), environmental parameters (wind speed / temperature, etc.), nozzle status (pressure / flow / angle), and operation quality score, etc. Use the database to optimize the generation of the variable spraying control strategy and monitor the faults of the components of the drone spraying device.

[0038] Figure 4 The flowchart of the forestry intelligent spraying method based on multi-source sensor spatial perception is shown.

[0039] In the second aspect of the present invention, a forestry intelligent spraying method based on multi-source sensor spatial perception is provided, including the following steps: S402, scanning the target forest area through the lidar and vision sensors carried by the unmanned aerial vehicle (UAV) to construct a three-dimensional forest map, identifying and positioning individual trees using a tree recognition model, analyzing the canopy structure, and evaluating the leaf density of the canopy; S404, identifying pests and diseases using multi-spectral imaging combined with deep learning, and making a decision on the chemical agent ratio according to the type and severity of the pests and diseases; S406, performing global planning of the flight path and local dynamic obstacle avoidance based on the three-dimensional forest map to generate the optimal flight path for UAV spraying; S408, adjusting the nozzle parameters according to the leaf density of the canopy and the severity of the pests and diseases for variable spraying, and storing the UAV spraying operation parameters.

[0040] The third embodiment of the present invention provides a computer-readable storage medium, which includes a program for the forestry intelligent spraying method based on multi-source sensor spatial perception. When the program for the forestry intelligent spraying method based on multi-source sensor spatial perception is executed by a processor, the steps of the forestry intelligent spraying method based on multi-source sensor spatial perception are implemented.

[0041] In several embodiments provided by the present application, it should be understood that the disclosed method can be implemented in other ways. The above-described embodiments are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For example, multiple modules or components can be combined, or integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0042] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the foregoing storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks and other various media that can store program codes.

[0043] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention.

Claims

1. A forestry intelligent spraying system based on multi-source sensor spatial perception, characterized in that, It includes a multi-source sensor perception module, an environmental modeling and map construction module, a tree identification and canopy analysis module, a pest and disease identification and intelligent drug dispensing module, an autonomous path planning and obstacle avoidance module, and a precise spraying control module; The multi-source sensor perception module is responsible for collecting multi-source perception data through preset sensors and preprocessing the collected multi-source perception data; The environmental modeling and map construction module combines LiDAR, vision, and GPS data to generate a three-dimensional forestry map and identifies obstacle information for annotation in the three-dimensional forestry map; The tree identification and canopy analysis module constructs a tree identification model to identify individual trees, calculates the height and canopy range, evaluates the density of branches and leaves in the canopy, and analyzes and generates a droplet penetration strategy; The pest and disease identification and intelligent drug dispensing module uses multi-spectral imaging combined with deep learning to identify pests and diseases, makes decisions on drug formulation ratio and variable spraying control according to the type and severity of pests and diseases; The autonomous path planning and obstacle avoidance module conducts global planning of the flight path and local dynamic obstacle avoidance based on the three-dimensional forestry map, and dynamically adjusts the altitude of the unmanned aerial vehicle according to the tree height; The precise spraying control module dynamically adjusts the nozzle parameters according to the environmental perception data and stores the spraying data of each tree.

2. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 1, characterized in that In the environmental modeling and map construction module, LiDAR, vision, and GPS data are combined to generate a three-dimensional forestry map, and obstacle information is identified and annotated in the three-dimensional forestry map. Specifically: Obtain the preprocessed LiDAR point cloud data, vision data, and GPS data, extract edge points and plane points from the LiDAR point cloud data, select feature points based on curvature analysis, match adjacent frames according to the feature points, and establish an association relationship between the feature points of the current frame and the feature points of the previous frame; Calculate the relative pose between adjacent frames through normal distribution transformation and use IMU data for pose rationality verification to obtain a local point cloud map as a matching reference; Extract key points from the vision data, calculate descriptors for each key point, use descriptor matching for adjacent frame matching, and perform reprojection matching with the feature points in the local point cloud map, and optimize the pose according to the matching points combined with IMU data; Obtain the unmanned aerial vehicle pose information as nodes, construct a pose graph with sensor observation constraints as edges, match historical scenes through visual bag-of-words and LiDAR point clouds, perform loop closure detection, optimize the pose graph using the least squares method, and use the GPS position for global correction when GPS data is available to obtain a corrected pose graph; Align each pose node in the pose graph with the corresponding sensor data time, perform coordinate transformation on the point cloud data, generate a dense three-dimensional point cloud, convert the point cloud into an octree structure, and construct a three-dimensional forestry map; Use a 3D CNN network to identify obstacle information, use the obstacle information for target annotation, project the three-dimensional point cloud in the three-dimensional forestry map onto a two-dimensional plane to generate an occupancy grid map, and identify the flyable area.

3. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 1, characterized in that, In the tree identification and canopy analysis module, a tree identification model is constructed to identify individual trees. Specifically: Synchronize the preprocessed LiDAR point cloud data and multi-spectral image data, construct a tree recognition model, use the PointNet++ network to set up a point cloud feature extraction branch, perform point cloud feature extraction with 4 SA modules, and realize feature upsampling through interpolation, gradually fusing point cloud features of different scales; Use the MobileNetV3 network to set up an image feature projection branch, obtain the global feature vector corresponding to the multi-spectral image as the image feature, project the image feature into the point cloud space, and splice it with the point cloud feature for feature association; Introduce an attention mechanism in the tree recognition model for bimodal feature fusion, perform weighted fusion on the associated point cloud feature and image feature, screen seed points according to the fusion feature, adaptively adjust the preset multi-scale anchor boxes, encode the neighborhood features of each seed point, and predict and generate anchor boxes and anchor box categories; Obtain the geometric features and spectral features of each three-dimensional point within the anchor box as point features, calculate the covariance matrix for each point feature, calculate the Mahalanobis distance from the covariance matrix to the corresponding tree point cloud instance of the category, and use Softmax for normalization processing to achieve probability conversion; Obtain the three-dimensional points that meet the probability threshold standard for deterministic assignment and mark them as tree instance members. Obtain the three-dimensional points that do not meet the probability threshold standard, and use the point features to obtain the classification score for tree classification. If the classification score is greater than the preset threshold, mark it as a tree instance member; Use the tree instance member set to segment the trees within the anchor box and output the segmentation results of individual trees.

4. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 3, wherein Calculate the height and canopy range, evaluate the density of branches and leaves in the canopy, and analyze and generate a fog droplet penetration strategy, specifically: Select the point cloud positioning root point according to the recognition and segmentation results of individual trees, measure the tree height based on the root point, perform clustering analysis on the point cloud corresponding to individual trees, and select the initial clustering center according to the point cloud distribution combined with the tree point cloud instance; Calculate the Euclidean distance from each point cloud in the point cloud corresponding to individual trees to the initial clustering center, assign each point cloud to the nearest initial clustering center to generate a clustering cluster, and through iterative clustering and updating of the clustering center, select the clustering cluster of the last clustering as the clustering result; Mark the point cloud involved in the canopy cluster according to the clustering result, and obtain the canopy range according to the horizontal projection and vertical stratification to extract the point cloud distribution of the marked point cloud; Perform three-dimensional grid division on the point cloud corresponding to the canopy range, calculate the point cloud density in each grid to quantify the spatial distribution characteristics of branches and leaves, and evaluate the density of branches and leaves according to the point cloud density distribution within the canopy range combined with the preset density level division standard; Based on the canopy types with different degrees of branch and leaf density in the area, match the spraying instances based on the canopy type, count the fog droplet size intervals with usage frequencies meeting the preset standard in the matching results, and use the fog droplet size intervals as the recommended particle sizes to generate a fog droplet penetration strategy.

5. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 1, characterized in that In the pest and disease identification and intelligent drug dispensing module, use multi-spectral imaging combined with deep learning to identify pests and diseases, specifically: Extract spectral features based on the preprocessed multispectral data, use the improved genetic algorithm to optimize the combination of spectral features, evaluate the importance of the feature combination using the recognition classification accuracy, and evaluate the redundancy of the feature combination according to the Pearson correlation coefficient; Calculate the fitness with the goal of maximizing the importance and minimizing the redundancy, select the spectral feature combination with the highest fitness for replication to determine the search direction, and perform feature recombination through crossover to accelerate the optimization selection process. When the termination condition is met, obtain the best spectral feature combination according to the optimal solution; Build a lightweight pest and disease identification model based on the EfficientDet-Lite model, use the best spectral feature combination as the input of the pest and disease identification model, adopt EfficientNet-B3 as the backbone network and introduce lightweight channel attention to obtain multispectral features; Reduce the number of BiFPN layers to 4 layers, generate anchor box adaptations for the disease spots using the multispectral features, and predict the pest and disease types and damage stages corresponding to the anchor boxes through the pre-trained classification head.

6. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 5, characterized in that Make decisions on the chemical agent ratio and variable spraying control according to the pest and disease types and severity. Specifically: Extract the corresponding disease spot areas according to the predicted anchor boxes, project the spectral features and image features of the disease spot areas into a low-dimensional space using the t-SNE dimensionality reduction algorithm to generate a feature scatter plot; Retrieve pest and disease instances with severity labels annotated by using the pest and disease types and damage stages, use the Wasserstein distance to quantify the distribution difference between the predicted anchor boxes and the feature scatter plots of the pest and disease instances, obtain the pest and disease instances whose distribution differences meet the preset difference threshold, and determine the severity according to the selected pest and disease instances; Construct a pest and disease portrait by combining the pest and disease types, severity and environmental features, perform feature embedding encoding on the pest and disease portrait to generate corresponding pest and disease nodes, severity nodes and environmental feature nodes, obtain the corresponding spatio-temporal associations according to the disease spot adjacency and disease spot state transition, and use the nodes and spatio-temporal associations to construct a topological relationship graph; Use the graph convolutional network to perform representation learning on the topological relationship graph, use the message passing and neighbor aggregation mechanisms to obtain the corresponding graph embedding vectors, and perform KNN search in the domain knowledge graph through the graph embedding vectors to obtain a preset number of valid cases; Input the preset number of valid cases into the pre-trained multi-task learning head, obtain the probability distribution of the chemical agent categories and the dose prediction values to generate the chemical agent ratio decision result, and determine the fog droplet size classification result and the nozzle angle regression result based on the fog droplet penetration strategy to generate the variable spraying control strategy.

7. The forestry intelligent spraying system based on multi-source sensor spatial perception according to claim 1, characterized in that In the precise spraying control module, dynamically adjust the nozzle parameters according to the environmental perception data, and store the spraying data of each tree. Specifically: Obtain the multi-source environmental perception data and spraying quality parameters during the UAV spraying process, construct the mapping relationship between the multi-source environmental perception data and spraying quality parameters based on the neural network, and train the data mapping model under normal spraying quality; Importing real-time multi-source environmental perception data into a data mapping model under normal spraying quality for estimation, calculating the residual between the estimated multi-source environmental perception data and the real-time multi-source environmental perception data, and re-obtaining the variable spraying control strategy when the residual is greater than a preset residual threshold, and dynamically adjusting the nozzle parameters; The basic data of individual trees, spraying status data and multi-source environmental perception data are standardized and stored in a database. The database is used to optimize the generation of variable spraying control strategies and to monitor the faults of the components of the drone spraying device.

8. A forestry intelligent spraying method based on multi-source sensor spatial perception, applied to the forestry intelligent spraying system based on multi-source sensor spatial perception as claimed in any one of claims 1 to 7, comprising the following steps: The drone uses lidar and visual sensors to scan the target forest area, construct a three-dimensional forest map, and use a tree recognition model to identify and locate individual trees, analyze the canopy structure, and assess the density of leaves in the canopy. Use multispectral imaging combined with deep learning to identify pests and diseases, and make pesticide ratio decisions based on the type and severity of the pests and diseases; Performing global planning of the flight path and local dynamic obstacle avoidance based on the three-dimensional forestry map to generate an optimal flight path for the drone spraying; Adjust nozzle parameters according to the density of leaves in the canopy and the severity of pests and diseases, perform variable spraying, and store drone spraying operation parameters.

Images