Forest pruning auxiliary decision-making method and system based on artificial intelligence

Abstract

The invention discloses a forest pruning auxiliary decision-making method and system based on artificial intelligence, and relates to the field of forest visual analysis. Images and point cloud data of a target tree are collected through a camera shooting and laser radar device; matching a tree species database by using a target detection model, extracting a target morphological feature, identifying a pruning area, performing feature description and difference comparison by combining a Sobel operator, judging a pruning probability and extracting a two-dimensional image feature; performing three-dimensional feature extraction on the point cloud data by adopting an FPFH local feature descriptor, evaluating differences in combination with morphological features, and generating three-dimensional point cloud features; the two-dimensional and three-dimensional features are subjected to weighted fusion through a DCA feature fusion algorithm, a decision-making tree decision-making model is constructed after the pruning probability is associated, and efficient judgment and decision-making assistance of a pruning mode are carried out on real-time tree data. According to the method, the multi-modal data and the intelligent algorithm are fused, and the accuracy and the automation level of pruning decision making are improved.

Description

Technical Field

[0001] This invention relates to the field of visual analysis of trees, and more specifically, to a method and system for assisting decision-making in tree pruning based on artificial intelligence. Background Technology

[0002] Pruning and shaping of economic forests is a key technical measure to ensure high yields of fruit trees. However, at present, pruning and shaping techniques still mainly rely on two methods: textual learning and on-site guidance. However, textual learning has drawbacks such as high threshold, slow updates, and narrow dissemination. Frontline workers find it difficult to identify the correctness and advancement of knowledge, and grassroots forestry workers find it difficult to master it quickly. On-site guidance relies more on the skills and explanation level of the instructors. Differences in the experience of different instructors lead to inconsistent pruning results. Insufficient human resources during the peak season of autumn and winter pruning also affect planting efficiency, thus becoming a bottleneck restricting the development of the forestry industry.

[0003] In recent years, technologies such as artificial intelligence and visual analytics have been gradually applied in the agricultural field. However, existing technologies are mostly focused on pest and disease identification and yield prediction, while intelligent auxiliary decision-making systems for forest pruning are still in their infancy. How to utilize artificial intelligence technology for efficient and rapid decision analysis and forest adjustment in forest pruning is a crucial problem that urgently needs to be solved. Summary of the Invention

[0004] This invention overcomes the shortcomings of existing technologies and proposes an artificial intelligence-based auxiliary decision-making method and system for forest pruning.

[0005] The first aspect of this invention provides an artificial intelligence-based forest pruning auxiliary decision-making method, comprising: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

[0006] In this solution, S1 specifically refers to: Multiple camera angles are set based on the shape and location of the target trees; The camera device acquires multi-angle images of the target tree, and the multi-angle images are integrated to obtain the first image set; The LiDAR scanner scans point cloud data corresponding to multiple camera angles, and uses radius filtering to remove discrete noise points from the multi-angle point cloud data. The point cloud is then aligned and 3D modeled to define a 3D space. Spatial location marking is performed on the positional information of multi-angle images in three-dimensional space.

[0007] In this solution, S2 specifically refers to: The first image set is preprocessed with image denoising, smoothing and grayscale conversion; The YOLOv7 model is introduced to identify the target tree species in the first image set, and multiple tree species identification results are generated based on multi-angle images. Based on the same target, tree species retrieval information is constructed using the most frequently occurring recognition results. The target pruning morphology of tree species is retrieved from the database, and the target morphological features based on two-dimensional images and three-dimensional point clouds are obtained.

[0008] In this solution, S3 specifically refers to: Based on the YOLOv7 model, the tree trunk, branches, canopy, and leaf edges are identified and marked from the first image set, and multiple target pruning regions are generated. The Sobel operator is used to perform morphological and contour feature analysis and feature description on the target trimmed region to generate the first feature. Based on Mahalanobis distance, the first feature is compared with the two-dimensional features in the target morphology features, and the pruning probability is determined based on the degree of difference. Calculate and analyze the first feature and trimming probability of all target trimming regions.

[0009] In this solution, S4 specifically refers to: Map the target trimmed region onto the point cloud data region; In point cloud data, the FPFH local descriptor is introduced to extract local features for each target trimmed region, thus obtaining the second feature. Based on Mahalanobis distance, the differences between the second feature and the three-dimensional features in the target morphology feature are compared, and the pruning probability is determined based on the degree of difference.

[0010] In this solution, S5 specifically refers to: Based on the DCA feature fusion algorithm, the first and second features of the target trimming region are vectorized into feature vectors to form a first feature vector and a second feature vector. The first feature vector and the second feature vector are dimensionally aligned, and the first feature map and the second feature map are generated through a CNN network. Based on the pruning probabilities of the first feature and the second feature, weight coefficients are set, and the first feature map and the second feature map are spatially weighted and fused to generate fused features. Calculate the mean of the pruning probabilities of the first feature and the second feature, label them as the judgment probabilities, and associate and bind the fused features with the judgment probabilities; Construct a decision-making model based on decision trees; For each target pruning region, a corresponding fusion feature and judgment probability are generated. Based on the fusion feature, the condition transformation of feature judgment is performed to generate condition nodes. The corresponding judgment probability is bound to the condition node as the classification condition of the decision tree, resulting in multiple classification nodes.

[0011] In this solution, S6 specifically refers to: A node is randomly selected as the root node based on the classification nodes. The internal nodes are determined by the C4.5 heuristic algorithm, and leaf nodes and all classification conditions are generated step by step to build a complete decision model. Historical tree pruning feature data from the database is used as training data to classify and train the decision-making model and optimize its nodes.

[0012] In this solution, S6 further includes: Real-time visual data of the second target tree is obtained through camera devices and lidar. Based on real-time tree visual data, real-time fusion features based on two-dimensional images and three-dimensional point clouds are obtained. The real-time fusion features are then imported into a decision model for prediction and classification, and judgment probabilities are generated. Based on a preset probability range, the pruning mode is classified according to the judgment probability, a real-time pruning mode is set and applied to the second target tree.

[0013] A second aspect of the present invention also provides an artificial intelligence-based forest pruning auxiliary decision-making system, the system comprising: a memory and a processor, wherein the memory includes an artificial intelligence-based forest pruning auxiliary decision-making program, and the artificial intelligence-based forest pruning auxiliary decision-making program, when executed by the processor, performs the following steps: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

[0014] A third aspect of the present invention also provides a computer-readable storage medium comprising an artificial intelligence-based forest pruning auxiliary decision-making program, wherein when the artificial intelligence-based forest pruning auxiliary decision-making program is executed by a processor, it implements the steps of the artificial intelligence-based forest pruning auxiliary decision-making method as described in any of the preceding claims.

[0015] This invention discloses an artificial intelligence-based method and system for assisting forest pruning decisions, relating to the field of forest visual analysis. It acquires images and point cloud data of target trees using cameras and lidar devices; utilizes a target detection model to match a tree species database, extracts target morphological features, and identifies pruning areas; combines the Sobel operator for feature description and difference comparison to determine pruning probabilities and extract two-dimensional image features; employs the FPFH local feature descriptor to extract three-dimensional features from the point cloud data, combining morphological features to evaluate differences and generate three-dimensional point cloud features; and uses the DCA feature fusion algorithm to weightedly fuse the two-dimensional and three-dimensional features, correlates them with pruning probabilities, and constructs a decision tree model for efficient determination and decision assistance of pruning patterns based on real-time tree data. This invention integrates multimodal data and intelligent algorithms to improve the accuracy and automation level of pruning decisions. Attached Figure Description

[0016] Figure 1 A flowchart of an artificial intelligence-based forest pruning auxiliary decision-making method according to the present invention is shown; Figure 2 A flowchart of the visual data acquisition process of the present invention is shown; Figure 3 A block diagram of an artificial intelligence-based forest pruning auxiliary decision-making system of the present invention is shown. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0019] Figure 1 A flowchart of an artificial intelligence-based forest pruning auxiliary decision-making method according to the present invention is shown.

[0020] like Figure 1 As shown, the first aspect of the present invention provides an artificial intelligence-based forest pruning auxiliary decision-making method, comprising: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

[0021] It should be noted that the camera device can be a professional high-resolution camera unit or a mobile camera terminal, such as a smartphone. The target trees are generally a group of trees based on data collection needs, used to collect a certain amount of data for feature analysis and decision model construction.

[0022] In the process of pruning pattern analysis, the decision model constructed for the same type of tree can be effectively applied to the visual feature analysis and real-time classification of pruning patterns of other tree groups of the same type.

[0023] Figure 2 A flowchart of the visual data acquisition process of the present invention is shown.

[0024] According to an embodiment of the present invention, S1 specifically includes: S202, based on the shape and location of the target tree, sets multiple camera angles; S204: Using a camera device, acquire multi-angle images of the target tree, and integrate the multi-angle images to obtain a first image set; S206 uses LiDAR to scan point cloud data corresponding to multiple camera angles, and uses radius filtering to remove discrete noise points from the multi-angle point cloud data. It also aligns and models the point cloud in three dimensions, and sets up a three-dimensional space. S208 marks the spatial location of the multi-angle image's position information in three-dimensional space.

[0025] It should be noted that the multi-angle images are two-dimensional image data, storing relevant camera angle information consistent with the LiDAR shooting position and angle. Two-dimensional images can be bound to the three-dimensional space of the point cloud, allowing for subsequent mapping and association of three-dimensional spatial regions through the two-dimensional image areas. The point cloud data is measured using LiDAR technology for multi-directional and multi-angle data. Both the first image set and the point cloud data are visual data.

[0026] According to an embodiment of the present invention, step S2 specifically includes: The first image set is preprocessed with image denoising, smoothing and grayscale conversion; The YOLOv7 model is introduced to identify the target tree species in the first image set, and multiple tree species identification results are generated based on multi-angle images. Based on the same target, tree species retrieval information is constructed using the most frequently occurring recognition results. The target pruning morphology of tree species is retrieved from the database, and the target morphological features based on two-dimensional images and three-dimensional point clouds are obtained.

[0027] It should be noted that the YOLOv7 model is an AI object detection model that uses deep learning technology to identify objects in images or videos and output their category and location information. Depending on the detection requirements, other models based on fast object detection algorithms can be used. The YOLOv7 model is trained using an existing tree-type image database. Target morphological features include two types of feature data: the first is based on two-dimensional image target morphological features, described using the Soble descriptor; the second is based on three-dimensional point cloud target morphological features, described using the FPFH local descriptor. These are used for subsequent difference evaluation between two-dimensional and three-dimensional features, respectively. According to an embodiment of the present invention, step S3 specifically includes: Based on the YOLOv7 model, the tree trunk, branches, canopy, and leaf edges are identified and marked from the first image set, and multiple target pruning regions are generated. The Sobel operator is used to perform morphological and contour feature analysis and feature description on the target trimmed region to generate the first feature. Based on Mahalanobis distance, the first feature is compared with the two-dimensional features in the target morphology features, and the pruning probability is determined based on the degree of difference. Calculate and analyze the first feature and trimming probability of all target trimming regions.

[0028] It should be noted that different target pruning regions correspond to independent regionalized features. The pruning probability is proportional to the feature difference value, i.e., the distance value. The Sobel operator is an edge detection operator, which in this invention is used to describe tree features in a two-dimensional image and generate feature data based on the two-dimensional image.

[0029] For the two-dimensional and three-dimensional data analysis process, there are two pruning probabilities, namely S3 and S4, which correspond to the pruning probabilities of the first feature and the second feature, respectively. Both pruning probabilities can be used as a reference for pruning decisions in a certain tree (forest) area. Generally, the pruning probability of the second feature corresponding to the lidar has a larger reference weight.

[0030] According to an embodiment of the present invention, S4 specifically includes: Map the target trimmed region onto the point cloud data region; In point cloud data, the FPFH local descriptor is introduced to extract local features for each target trimmed region, thus obtaining the second feature. Based on Mahalanobis distance, the differences between the second feature and the three-dimensional features in the target morphology feature are compared, and the pruning probability is determined based on the degree of difference.

[0031] It should be noted that the FPFH local descriptor specifically uses fast point feature histograms for feature extraction, which is used to extract point cloud features within a certain spatial range.

[0032] According to an embodiment of the present invention, S5 specifically includes: Based on the DCA feature fusion algorithm, the first and second features of the target trimming region are vectorized into feature vectors to form a first feature vector and a second feature vector. The first feature vector and the second feature vector are dimensionally aligned, and the first feature map and the second feature map are generated through a CNN network. Based on the pruning probabilities of the first feature and the second feature, weight coefficients are set, and the first feature map and the second feature map are spatially weighted and fused to generate fused features. Calculate the mean of the pruning probabilities of the first feature and the second feature, label them as the judgment probabilities, and associate and bind the fused features with the judgment probabilities; Construct a decision-making model based on decision trees; For each target pruning region, a corresponding fusion feature and judgment probability are generated. Based on the fusion feature, the condition transformation of feature judgment is performed to generate condition nodes. The corresponding judgment probability is bound to the condition node as the classification condition of the decision tree, resulting in multiple classification nodes.

[0033] It should be noted that the feature map is graph structure data generated based on a CNN network, possessing high-level semantic information. In the setting of weight coefficients for the pruning probabilities based on the first and second features, the pruning probability is proportional to the weight coefficient. In the DCA weighted fusion process, this invention adopts a dynamic weighting method, dynamically setting weight coefficients for the pruning probabilities obtained in steps S3 and S4 for subsequent DCA weighted fusion. The weight coefficients differ for different pruning scenarios.

[0034] According to an embodiment of the present invention, S6 specifically includes: A node is randomly selected as the root node based on the classification nodes. The internal nodes are determined by the C4.5 heuristic algorithm, and leaf nodes and all classification conditions are generated step by step to build a complete decision model. Historical tree pruning feature data from the database is used as training data to classify and train the decision-making model and optimize its nodes.

[0035] It should be noted that the historical tree pruning feature data includes first and second features based on two-dimensional and three-dimensional data, as well as corresponding fused features, pruning probabilities, pruning patterns, and other data. The decision-making model can perform conditional classification based on the real-time collected two-dimensional and three-dimensional fused features of trees, determine the pruning probability, and achieve rapid and efficient pruning decision analysis for the target area.

[0036] According to an embodiment of the present invention, S6 further includes: Real-time visual data of the second target tree is obtained through camera devices and lidar. Based on real-time tree visual data, real-time fusion features based on two-dimensional images and three-dimensional point clouds are obtained. The real-time fusion features are then imported into a decision model for prediction and classification, and judgment probabilities are generated. Based on a preset probability range, the pruning mode is classified according to the judgment probability, a real-time pruning mode is set and applied to the second target tree.

[0037] It should be noted that the real-time fusion feature refers to the fusion feature of a specific target region of the second target tree. The process of classifying the judgment probability into pruning modes, setting real-time pruning modes, and applying them to the second target tree can be specifically divided into three modes, with corresponding probability ranges as follows: [0-0.3], (0.3-0.6], and (0.6-1] correspond to three levels of trimming modes: red, yellow, and green, respectively. Green areas require direct trimming, with suggestions on trimming location, intensity, and method. Yellow areas require secondary judgment by the system, which can continuously collect more detailed visual data for analysis until the system completes the judgment. Red areas do not require trimming. The color of specific areas can be visualized using 3D point cloud data, displayed on user mobile terminals or other computer terminals.

[0038] It is worth mentioning that existing tree pruning strategies are difficult to implement multi-dimensional spatial fusion analysis, resulting in low efficiency in pruning analysis. Moreover, they are often based on human experience, making it difficult to achieve efficient pruning. At the same time, existing online image analysis technologies have a lag in real-time tree pruning decisions, and the analysis dimension of the pruning area is low. They are often based on simple image dimension analysis and lack the means to fuse and analyze the three-dimensional features of trees. As a result, the accuracy of current visual analysis in judging the pruning status is low, making it difficult to improve economic benefits.

[0039] In addition, existing technologies are inefficient in feature recognition and trimming matching of point cloud data, have low recognition accuracy, and suffer from long recognition times. They also have low accuracy and efficiency in analyzing 3D point cloud data and exhibit decision lag.

[0040] This invention effectively solves the aforementioned problems. It involves acquiring two-dimensional images and three-dimensional point cloud data of target trees through visual analysis for regional visual feature analysis. Pruning probabilities are set based on the differences between two-dimensional and three-dimensional features. A fusion algorithm is then used to fuse and analyze relevant two-dimensional and three-dimensional recognition features based on the target tree region. A decision tree is constructed based on the fused features to efficiently classify real-time visual features, generate judgment probabilities, and make pruning decisions. This achieves efficient and rapid forestry decision analysis and pruning pattern setting, improving the economic benefits of forestry.

[0041] This invention utilizes mobile camera equipment and LiDAR, combined with big data analysis and AI-assisted decision-making algorithms, to achieve visualized and accurate presentation of decisions. It directly addresses the pain point of learning pruning techniques for grassroots forestry workers, enabling autonomous tree species identification, multi-dimensional tree model construction, and dynamic assisted decision-making for pruning within forestry scenarios. This significantly improves the efficiency of forestry pruning, reduces learning costs for grassroots workers, and enhances fruit quality and yield.

[0042] This invention, based on fusion feature analysis and classification decision-making, can improve the real-time pruning area detection capability. By classifying through feature fusion, it can support a certain degree of fuzzy feature input detection and improve the applicability of different visual information acquisition devices and complex forest environments. For example, it can still achieve efficient forest auxiliary decision-making when the lighting is poor or when it is difficult to collect two-dimensional and three-dimensional features of trees in a comprehensive manner.

[0043] According to an embodiment of the present invention, it further includes: The S5 also includes: The first and second features of multiple target trimmed regions are vectorized into feature vectors to form multiple first feature vectors and multiple second feature vectors. The difference between multiple first feature vectors is calculated based on Mahalanobis distance, and the variance value is used to evaluate the degree of dispersion of the multiple first feature vectors. At the same time, the dispersion of multiple second feature vectors is evaluated, and the weight coefficients for the first and second features are set based on the dispersion of the data. DCA feature fusion is performed based on the set weight coefficients.

[0044] It should be noted that setting weight coefficients based on pruning probabilities is applicable to pruning tasks and decision analysis involving relatively simple tree types. However, for more complex forest situations, such as pruning decision analysis involving multiple tree species, a dynamic setting of weight coefficients based on the feature differences of multiple target regions, as described above, can be used to construct a decision model capable of accurately classifying pruning regions with high feature differences. The larger the variance and the greater the dispersion, the smaller the weight coefficients should be.

[0045] Figure 3 A block diagram of an artificial intelligence-based forest pruning auxiliary decision-making system of the present invention is shown.

[0046] A second aspect of the present invention also provides an artificial intelligence-based forest pruning auxiliary decision-making system 3, the system comprising: a memory 31 and a processor 32, wherein the memory includes an artificial intelligence-based forest pruning auxiliary decision-making program, and the artificial intelligence-based forest pruning auxiliary decision-making program, when executed by the processor, performs the following steps: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

[0047] It should be noted that the camera device can be a professional high-resolution camera unit or a mobile camera terminal, such as a smartphone. The target trees are generally a group of trees based on data collection needs, used to collect a certain amount of data for feature analysis and decision model construction.

[0048] In the process of pruning pattern analysis, the decision model constructed for the same type of tree can be effectively applied to the visual feature analysis and real-time classification of pruning patterns of other tree groups of the same type.

[0049] According to an embodiment of the present invention, S1 specifically includes: Multiple camera angles are set based on the shape and location of the target trees; The camera device acquires multi-angle images of the target tree, and the multi-angle images are integrated to obtain the first image set; The LiDAR scanner scans point cloud data corresponding to multiple camera angles, and uses radius filtering to remove discrete noise points from the multi-angle point cloud data. The point cloud is then aligned and 3D modeled to define a 3D space. Spatial location marking is performed on the positional information of multi-angle images in three-dimensional space.

[0050] It should be noted that the multi-angle images are two-dimensional image data, storing relevant camera angle information consistent with the LiDAR shooting position and angle. Two-dimensional images can be bound to the three-dimensional space of the point cloud, allowing for subsequent mapping and association of three-dimensional spatial regions through the two-dimensional image areas. The point cloud data is measured using LiDAR technology for multi-directional and multi-angle data. Both the first image set and the point cloud data are visual data.

[0051] According to an embodiment of the present invention, step S2 specifically includes: The first image set is preprocessed with image denoising, smoothing and grayscale conversion; The YOLOv7 model is introduced to identify the target tree species in the first image set, and multiple tree species identification results are generated based on multi-angle images. Based on the same target, tree species retrieval information is constructed using the most frequently occurring recognition results. The target pruning morphology of tree species is retrieved from the database, and the target morphological features based on two-dimensional images and three-dimensional point clouds are obtained.

[0052] It should be noted that the YOLOv7 model is an AI object detection model that uses deep learning technology to identify objects in images or videos and output their category and location information. Depending on the detection requirements, other models based on fast object detection algorithms can be used. The YOLOv7 model is trained using an existing tree-type image database. Target morphological features include two types of feature data: the first is based on two-dimensional image target morphological features, described using the Soble descriptor; the second is based on three-dimensional point cloud target morphological features, described using the FPFH local descriptor. These are used for subsequent difference evaluation between two-dimensional and three-dimensional features, respectively. According to an embodiment of the present invention, step S3 specifically includes: Based on the YOLOv7 model, the tree trunk, branches, canopy, and leaf edges are identified and marked from the first image set, and multiple target pruning regions are generated. The Sobel operator is used to perform morphological and contour feature analysis and feature description on the target trimmed region to generate the first feature. Based on Mahalanobis distance, the first feature is compared with the two-dimensional features in the target morphology features, and the pruning probability is determined based on the degree of difference. Calculate and analyze the first feature and trimming probability of all target trimming regions.

[0053] It should be noted that different target pruning regions correspond to independent regionalized features. The pruning probability is proportional to the feature difference value, i.e., the distance value. The Sobel operator is an edge detection operator, which in this invention is used to describe tree features in a two-dimensional image and generate feature data based on the two-dimensional image.

[0054] For the two-dimensional and three-dimensional data analysis process, there are two pruning probabilities, namely S3 and S4, which correspond to the pruning probabilities of the first feature and the second feature, respectively. Both pruning probabilities can be used as a reference for pruning decisions in a certain tree (forest) area. Generally, the pruning probability of the second feature corresponding to the lidar has a larger reference weight.

[0055] According to an embodiment of the present invention, S4 specifically includes: Map the target trimmed region onto the point cloud data region; In point cloud data, the FPFH local descriptor is introduced to extract local features for each target trimmed region, thus obtaining the second feature. Based on Mahalanobis distance, the differences between the second feature and the three-dimensional features in the target morphology feature are compared, and the pruning probability is determined based on the degree of difference.

[0056] It should be noted that the FPFH local descriptor specifically uses fast point feature histograms for feature extraction, which is used to extract point cloud features within a certain spatial range.

[0057] According to an embodiment of the present invention, S5 specifically includes: Based on the DCA feature fusion algorithm, the first and second features of the target trimming region are vectorized into feature vectors to form a first feature vector and a second feature vector. The first feature vector and the second feature vector are dimensionally aligned, and the first feature map and the second feature map are generated through a CNN network. Based on the pruning probabilities of the first feature and the second feature, weight coefficients are set, and the first feature map and the second feature map are spatially weighted and fused to generate fused features. Calculate the mean of the pruning probabilities of the first feature and the second feature, label them as the judgment probabilities, and associate and bind the fused features with the judgment probabilities; Construct a decision-making model based on decision trees; For each target pruning region, a corresponding fusion feature and judgment probability are generated. Based on the fusion feature, the condition transformation of feature judgment is performed to generate condition nodes. The corresponding judgment probability is bound to the condition node as the classification condition of the decision tree, resulting in multiple classification nodes.

[0058] It should be noted that the feature map is graph structure data generated based on a CNN network, possessing high-level semantic information. In the setting of weight coefficients for the pruning probabilities based on the first and second features, the pruning probability is proportional to the weight coefficient. In the DCA weighted fusion process, this invention adopts a dynamic weighting method, dynamically setting weight coefficients for the pruning probabilities obtained in steps S3 and S4 for subsequent DCA weighted fusion. The weight coefficients differ for different pruning scenarios.

[0059] According to an embodiment of the present invention, S6 specifically includes: A node is randomly selected as the root node based on the classification nodes. The internal nodes are determined by the C4.5 heuristic algorithm, and leaf nodes and all classification conditions are generated step by step to build a complete decision model. Historical tree pruning feature data from the database is used as training data to classify and train the decision-making model and optimize its nodes.

[0060] It should be noted that the historical tree pruning feature data includes first and second features based on two-dimensional and three-dimensional data, as well as corresponding fused features, pruning probabilities, pruning patterns, and other data. The decision-making model can perform conditional classification based on the real-time collected two-dimensional and three-dimensional fused features of trees, determine the pruning probability, and achieve rapid and efficient pruning decision analysis for the target area.

[0061] According to an embodiment of the present invention, S6 further includes: Real-time visual data of the second target tree is obtained through camera devices and lidar. Based on real-time tree visual data, real-time fusion features based on two-dimensional images and three-dimensional point clouds are obtained. The real-time fusion features are then imported into a decision model for prediction and classification, and judgment probabilities are generated. Based on a preset probability range, the pruning mode is classified according to the judgment probability, a real-time pruning mode is set and applied to the second target tree.

[0062] It should be noted that the real-time fusion feature refers to the fusion feature of a specific target region of the second target tree. The process of classifying the judgment probability into pruning modes, setting real-time pruning modes, and applying them to the second target tree can be specifically divided into three modes, with corresponding probability ranges as follows: [0-0.3], (0.3-0.6], and (0.6-1] correspond to three levels of trimming modes: red, yellow, and green, respectively. Green areas require direct trimming, with suggestions on trimming location, intensity, and method. Yellow areas require secondary judgment by the system, which can continuously collect more detailed visual data for analysis until the system completes the judgment. Red areas do not require trimming. The color of specific areas can be visualized using 3D point cloud data, displayed on user mobile terminals or other computer terminals.

[0063] It is worth mentioning that existing tree pruning strategies are difficult to implement multi-dimensional spatial fusion analysis, resulting in low efficiency in pruning analysis. Moreover, they are often based on human experience, making it difficult to achieve efficient pruning. At the same time, existing online image analysis technologies have a lag in real-time tree pruning decisions, and the analysis dimension of the pruning area is low. They are often based on simple image dimension analysis and lack the means to fuse and analyze the three-dimensional features of trees. As a result, the accuracy of current visual analysis in judging the pruning status is low, making it difficult to improve economic benefits.

[0064] In addition, existing technologies are inefficient in feature recognition and trimming matching of point cloud data, have low recognition accuracy, and suffer from long recognition times. They also have low accuracy and efficiency in analyzing 3D point cloud data and exhibit decision lag.

[0065] This invention effectively solves the aforementioned problems. It involves acquiring two-dimensional images and three-dimensional point cloud data of target trees through visual analysis for regional visual feature analysis. Pruning probabilities are set based on the differences between two-dimensional and three-dimensional features. A fusion algorithm is then used to fuse and analyze relevant two-dimensional and three-dimensional recognition features based on the target tree region. A decision tree is constructed based on the fused features to efficiently classify real-time visual features, generate judgment probabilities, and make pruning decisions. This achieves efficient and rapid forestry decision analysis and pruning pattern setting, improving the economic benefits of forestry.

[0066] This invention utilizes mobile camera equipment and LiDAR, combined with big data analysis and AI-assisted decision-making algorithms, to achieve visualized and accurate presentation of decisions. It directly addresses the pain point of learning pruning techniques for grassroots forestry workers, enabling autonomous tree species identification, multi-dimensional tree model construction, and dynamic assisted decision-making for pruning within forestry scenarios. This significantly improves the efficiency of forestry pruning, reduces learning costs for grassroots workers, and enhances fruit quality and yield.

[0067] This invention, based on fusion feature analysis and classification decision-making, can improve the real-time pruning area detection capability. By classifying through feature fusion, it can support a certain degree of fuzzy feature input detection and improve the applicability of different visual information acquisition devices and complex forest environments. For example, it can still achieve efficient forest auxiliary decision-making when the lighting is poor or when it is difficult to collect two-dimensional and three-dimensional features of trees in a comprehensive manner.

[0068] A third aspect of the present invention also provides a computer-readable storage medium comprising an artificial intelligence-based forest pruning auxiliary decision-making program, wherein when the artificial intelligence-based forest pruning auxiliary decision-making program is executed by a processor, it implements the steps of the artificial intelligence-based forest pruning auxiliary decision-making method as described in any of the preceding claims.

[0069] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0070] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0071] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0072] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0074] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A forest pruning auxiliary decision-making method based on artificial intelligence, characterized in that, include: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

2. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, Specifically, S1 is: Multiple camera angles are set based on the shape and location of the target trees; The camera device acquires multi-angle images of the target tree, and the multi-angle images are integrated to obtain the first image set; The LiDAR scanner scans point cloud data corresponding to multiple camera angles, and uses radius filtering to remove discrete noise points from the multi-angle point cloud data. The point cloud is then aligned and 3D modeled to define a 3D space. Spatial location marking is performed on the positional information of multi-angle images in three-dimensional space.

3. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, Specifically, S2 is: The first image set is preprocessed with image denoising, smoothing and grayscale conversion; The YOLOv7 model is introduced to identify the target tree species in the first image set, and multiple tree species identification results are generated based on multi-angle images. Based on the same target, tree species retrieval information is constructed using the most frequently occurring recognition results. The target pruning morphology of tree species is retrieved from the database, and the target morphological features based on two-dimensional images and three-dimensional point clouds are obtained.

4. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 3, characterized in that, Specifically, S3 is: Based on the YOLOv7 model, the tree trunk, branches, canopy, and leaf edges are identified and marked from the first image set, and multiple target pruning regions are generated. The Sobel operator is used to perform morphological and contour feature analysis and feature description on the target trimmed region to generate the first feature. Based on Mahalanobis distance, the first feature is compared with the two-dimensional features in the target morphology features, and the pruning probability is determined based on the degree of difference. Calculate and analyze the first feature and trimming probability of all target trimming regions.

5. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, Specifically, S4 is: Map the target trimmed region onto the point cloud data region; In point cloud data, the FPFH local descriptor is introduced to extract local features for each target trimmed region, thus obtaining the second feature. Based on Mahalanobis distance, the differences between the second feature and the three-dimensional features in the target morphology feature are compared, and the pruning probability is determined based on the degree of difference.

6. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, Specifically, S5 is: Based on the DCA feature fusion algorithm, the first and second features of the target trimming region are vectorized into feature vectors to form a first feature vector and a second feature vector. The first feature vector and the second feature vector are dimensionally aligned, and the first feature map and the second feature map are generated through a CNN network. Based on the pruning probabilities of the first feature and the second feature, weight coefficients are set, and the first feature map and the second feature map are spatially weighted and fused to generate fused features. Calculate the mean of the pruning probabilities of the first feature and the second feature, label them as the judgment probabilities, and associate and bind the fused features with the judgment probabilities; Construct a decision-making model based on decision trees; For each target pruning region, a corresponding fusion feature and judgment probability are generated. Based on the fusion feature, the condition transformation of feature judgment is performed to generate condition nodes. The corresponding judgment probability is bound to the condition node as the classification condition of the decision tree, resulting in multiple classification nodes.

7. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, Specifically, S6 is: A node is randomly selected as the root node based on the classification nodes. The internal nodes are determined by the C4.5 heuristic algorithm, and leaf nodes and all classification conditions are generated step by step to build a complete decision model. Historical tree pruning feature data from the database is used as training data to classify and train the decision-making model and optimize its nodes.

8. The forest pruning auxiliary decision-making method based on artificial intelligence according to claim 1, characterized in that, S6 further includes: Real-time visual data of the second target tree is obtained through camera devices and lidar. Based on real-time tree visual data, real-time fusion features based on two-dimensional images and three-dimensional point clouds are obtained. The real-time fusion features are then imported into a decision model for prediction and classification, and judgment probabilities are generated. Based on a preset probability range, the pruning mode is classified according to the judgment probability, a real-time pruning mode is set and applied to the second target tree.

9. An artificial intelligence-based forest pruning auxiliary decision-making system, characterized in that, The system includes a memory and a processor. The memory contains an AI-based forest pruning auxiliary decision-making program. When executed by the processor, the AI-based forest pruning auxiliary decision-making program performs the following steps: S1: Using a camera device and lidar, multi-angle images and point cloud data of the target tree are collected to obtain the first image set and point cloud data; S2: Using the target detection model, target detection and database tree matching are performed on the first image set, and target morphological features based on images and point clouds are retrieved from the database; S3: Based on the target detection model, tree pruning areas are identified from the first image set and marked as multiple target pruning areas. The Sobel operator is introduced to describe the features of the target pruning areas and compare them with the target morphological features to determine the pruning probability. The first feature based on the two-dimensional image is collected. S4: Using the FPFH local descriptor, local features are extracted from the point cloud data in the target pruning region. The difference is evaluated in combination with the target morphological features, and the pruning probability is determined. The local features are collected to obtain the second feature based on the 3D point cloud. S5: Using the DCA feature fusion algorithm, the first and second features of multiple target pruning regions are weighted and fused to obtain multiple fused features. These multiple fused features are then associated with the pruning probability and used as classification conditions. Multiple classification nodes based on decision trees are then set. S6: Based on the decision tree construction process, the decision model is constructed using classification nodes as the original classification conditions and heuristic algorithms. The decision model is then used to determine the pruning pattern of real-time tree visual data.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes an artificial intelligence-based forest pruning auxiliary decision-making program, which, when executed by a processor, implements the steps of the artificial intelligence-based forest pruning auxiliary decision-making method as described in any one of claims 1 to 8.

Images