A method and system for branch depth estimation based on diffusion model

Abstract

The application discloses a kind of based on diffusion model's branch depth estimation method and system, belong to computer vision three-dimensional depth estimation technical field, suitable for forestry unmanned aerial vehicle obstacle avoidance and forest three-dimensional reconstruction.For the high cost of data acquisition of prior art, high occlusion scene branch details loss, prediction accuracy and real-time cannot be considered, difficult to support unmanned aerial vehicle real-time obstacle avoidance problem, the application takes branch scene RGB-D image as input, and splices RGB three channels and depth single channel into four channel input;Swin Transformer is used to extract multi-scale features, after fusion by feature pyramid network, six-layer deformable DETR Transformer is used as denoising decoder, and depth prediction is completed by three-step diffusion sampling;Effective pixel mask and double RMSE index are used to evaluate the results.The application has low data cost, fast inference speed, accurate branch perception, strong generalization, effectively solves the problem of high-occlusion branch depth estimation, and can be directly applied to unmanned aerial vehicle field operation.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically involving three-dimensional depth estimation technology, and particularly to a tree branch depth estimation method and system based on a diffusion model, which can be directly applied to tree branch obstacle detection, obstacle avoidance navigation, and forest three-dimensional structure reconstruction in forestry UAV field operations. Background Technology

[0002] Forests, as a core component of terrestrial ecosystems, play an irreplaceable role in key ecological functions such as soil and water conservation, biodiversity protection, and carbon cycle regulation. In recent years, unmanned aerial vehicle (UAV) platforms, with their significant advantages of flexible operation, rapid deployment, and high operational efficiency, have been widely used in various tasks in forestry and agriculture, including forest resource surveys, crop health monitoring, pest and disease control, and ecological environment assessment. However, when operating in complex forest environments, UAVs are highly susceptible to collisions with crisscrossing branches, leading to equipment damage and operational interruptions, or even UAV crashes and data loss, severely restricting the large-scale safe application of UAVs in forestry scenarios. Therefore, achieving accurate three-dimensional depth perception of branches is a core prerequisite and key technological bottleneck for ensuring the safety of UAV field operations and improving operational efficiency and reliability.

[0003] However, in real-world natural forest scenarios, tree branches possess extremely complex structural features: numerous branches intersect and overlap, creating severe occlusion; there are many delicate and fragile twig structures; and different branches exhibit extremely high textural similarity. These inherent characteristics make it difficult for traditional depth estimation methods to accurately capture the fine structure and spatial location information of tree branches, posing a significant technical challenge to the task of high-precision tree branch depth estimation.

[0004] Extensive research has been conducted by scholars both domestically and internationally on the problems of branch depth sensing and 3D tree reconstruction. Existing technologies can be mainly categorized into the following four types: 1. Process-oriented reconstruction method These methods are guided by morphological data of real trees, utilizing the self-similarity of tree growth to generate and optimize 3D tree skeleton structures through a pre-defined rule system or particle flow simulation algorithm. For example, Guo et al. proposed a plant modeling method that integrates multi-view 3D image reconstruction with rule-driven procedural modeling. This method first generates dense depth maps and reconstructs high-precision point cloud data using an improved stereo vision algorithm, then fits a parameterized plant skeleton under soft point cloud constraints, and finally completes the construction of a 3D tree model. Isokane et al. proposed a probabilistic plant modeling framework based on Bayesian image-to-image transformation. This framework first infers the probability map of the existence of 2D branches from multiple viewpoints of plant images, fuses the probability maps from multiple viewpoints into a 3D voxel probability field, and finally extracts the explicit 3D branch structure from the 3D voxel probability field using a particle flow simulation algorithm.

[0005] 2. Geometric Feature Extraction Methods These methods directly extract the topological skeleton of trees from 3D measurement data (such as laser scanning point clouds and voxel data) through geometric operations such as refinement, clustering, and spanning tree optimization. Liu et al. proposed a neural 3D tree decomposition and reconstruction method for point cloud data. This method detects branch nodes through a semantic segmentation network, divides local cylindrical segments through fine-grained clustering, merges similar clustering results using a pairwise affinity network, fits a generalized cylindrical model, connects adjacent nodes, and finally assembles a complete 3D tree model. Livny et al. proposed an automatic tree skeleton reconstruction method based on global optimization. This method directly constructs a branch structure map from laser scanning point cloud data and completes the reconstruction through four steps: skeleton initialization, weighted smoothing optimization, radius estimation, and structure simplification. It can handle point cloud data of multiple overlapping trees without the need for pre-segmentation of individual trees.

[0006] 3. Image-based modeling methods These methods extract skeleton features from single-view or multi-view 2D tree images and reconstruct 3D tree skeletons using geometric projection and multi-view fusion techniques. Lopez et al. proposed a sparse image modeling scheme for deciduous trees. This scheme first extracts alpha masks and 2D skeleton maps from 4-6 input images, obtains an effective 2D skeleton tree under tree topological constraints, reconstructs the 3D skeleton by matching geometric and topological cues, and finally completes voxel reconstruction of each branch using the 3D skeleton as a prior, fitting the branch morphology with a piecewise cylindrical model. Liu et al. proposed a single-image 3D tree reconstruction method that integrates conditional generative adversarial networks (GANs) and procedural modeling. This method first extracts edge features from the input image, receives user-drawn tree trunk strokes as guidance, predicts the 3D tree contour and skeleton depth map using a conditional GAN, and then uses a direction field-guided spatial colonization algorithm to generate a complete 3D tree model.

[0007] 4. Deep Learning-Based Depth Estimation Methods In addition, some studies have attempted to apply general deep learning models to tree-related depth estimation and semantic segmentation tasks. Chen et al. used a manually labeled apple tree RGB-D image dataset to train three deep learning models—Pix2Pix, U-Net, and DeepLabv3—to complete the apple tree-related depth estimation task. Tejaswi et al. focused on simple, unoccluded scenes and studied a tree-like vegetation semantic segmentation method based on RGB-D input. Geckeler et al. specifically constructed a synthetic tree dataset and systematically evaluated the performance of three mainstream deep learning models—U-Net, TransUNet, and ESANet—on the forest environment branch depth estimation task.

[0008] While the methods described above can extract tree skeletons or estimate depth in simple scenes under specific conditions, they generally suffer from the following insurmountable technical defects, which severely limit their practical application in complex, highly shaded forest scenes. These defects are precisely the key technical problems that this invention aims to solve: First, they are highly data-dependent, and the acquisition and processing costs are high. Procedural reconstruction methods and geometric feature extraction methods have extremely stringent requirements for the quality of input data, generally relying on high-precision laser scanning point cloud data or multi-view synchronously acquired image data. The acquisition of this type of data requires expensive specialized equipment (such as lidar and multi-camera synchronization systems), and the data processing is complex, computationally intensive, and time-consuming, which cannot meet the practical application needs of low-cost, high-efficiency, and large-scale deployment in forestry operations.

[0009] Second, the models exhibit poor scene adaptability and significant loss of fine details. While existing deep learning models have made significant progress in general depth estimation tasks, they perform extremely poorly in complex tree-branch scenes with high occlusion and rich detail. Due to the intersecting occlusion of branches, their delicate structure, and high texture similarity, existing models struggle to accurately distinguish foreground and background branches and cannot effectively capture the fine structure of branches. This results in problems such as blurred edge boundaries, significant loss of fine details, and limited generalization ability in the prediction results, making it difficult to meet the depth estimation accuracy requirements for drone obstacle avoidance.

[0010] Third, the prediction results are uneven and the reliability is insufficient. Existing methods struggle to achieve a good balance between accurately predicting the effective depth region and suppressing redundant predictions in ineffective regions. Some models generate a large number of redundant erroneous predictions in background regions without branches in order to retain more details; while other models lose a large amount of effective branch depth information in order to suppress background noise. This results in unreliable depth prediction results that cannot be directly applied to real-time obstacle avoidance systems for UAVs.

[0011] Diffusion models, as a class of high-performance generative models that have emerged in recent years, possess powerful feature modeling capabilities, excellent noise robustness, and accurate fitting ability for complex distributions, making them theoretically well-suited for solving depth estimation challenges in complex scenes. Currently, however, there is no mature technical solution for applying diffusion models to depth estimation tasks involving highly occluded tree branches. Summary of the Invention

[0012] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a tree branch depth estimation method and system based on a diffusion model. This method addresses the problems of existing technologies that rely on high-precision point clouds or multi-view images, resulting in high data acquisition and processing costs, low efficiency, difficulty in large-scale deployment, poor adaptability in complex forest scenes with high occlusion and dense branches, and issues such as blurred edges, lost branches, and limited generalization ability. Furthermore, it is difficult to balance accuracy and real-time performance, cannot balance effective area prediction and background redundancy suppression, has insufficient prediction reliability, and cannot support real-time obstacle avoidance for UAV field operations.

[0013] The present invention adopts the following technical solution: A tree depth estimation method based on a diffusion model includes the following steps: S1. Acquire RGB-D image data of the tree branch scene, and perform channel stitching of the RGB image and depth map in the RGB-D image data to form a four-channel input; S2. The four-channel input is fed into the Swin Transformer backbone network to generate four multi-scale feature maps with different resolutions. S3. The multi-scale feature maps are fused using a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. S4. Add noise to the depth ground truth label map according to the random sampling time step t to obtain the noisy depth map; S5. The noise-added depth map is concatenated with the feature map with a resolution of h / 4×w / 4 and then input into the denoising decoder composed of six stacked deformable DETR Transformers to perform denoising sampling and generate the current step depth prediction result. S6. The current depth prediction result is concatenated with the feature map with a resolution of h / 4×w / 4 and then input into the denoising decoder for the next sampling step. This process is repeated for a total of three sampling steps to obtain the final depth prediction result. S7. Use effective pixel masks to remove invalid regions of the depth ground truth, and use dual RMSE metrics to evaluate the prediction results.

[0014] Preferably, in step S1, the RGB-D image data consists of a three-channel RGB image and a single-channel depth map, and the four-channel input is formed by stitching the three-channel image and the single-channel image together.

[0015] Preferably, in step S2, the Swin Transformer backbone network extracts features by alternately applying window attention and shift window attention.

[0016] Preferably, in step S5, the denoising decoder uses the feature map obtained in step S3 as a conditional guide to iteratively denoise the noisy depth ground truth label map obtained in step S4.

[0017] Preferably, in step S6, each sampling step generates a corresponding depth prediction result, and the current step depth prediction result is concatenated with the feature map with a resolution of h / 4×w / 4 as the input for the next sampling step.

[0018] Preferably, the deformable DETR Transformer introduces a sparse deformable attention mechanism, where each query focuses on only a small number of adaptive sampling points and interacts with attention in local key regions around the reference point.

[0019] Secondly, embodiments of the present invention provide a tree depth estimation system based on a diffusion model, comprising: The input module is used to acquire RGB-D image data of the tree branch scene, and to perform channel stitching of the RGB image and the depth map in the RGB-D image data to form a four-channel input; The encoding module is used to feed the four-channel input into the Swing Transformer backbone network to generate four multi-scale feature maps with different resolutions. The fusion module is used to fuse the multi-scale feature maps through a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. The noise-adding module is used to add noise to the depth ground truth label map according to the random sampling time step t, so as to obtain the noise-adding depth ground truth label map. The decoding module is used to concatenate the noisy depth ground truth label map with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder composed of six stacked deformable DETR Transformers to perform denoising sampling and generate the depth prediction result of the current step. The sampling module is used to concatenate the current depth prediction result with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder for the next sampling step. The sampling is repeated for a total of three steps to obtain the final depth prediction result. The evaluation module is used to remove invalid regions of the ground truth depth using effective pixel masks and to evaluate the prediction results using dual RMSE metrics.

[0020] Preferably, in the RGB-D image data, the RGB image is a three-channel image and the depth map is a single-channel image.

[0021] Preferably, the Swin Transformer backbone network in the encoding module performs feature extraction by alternately applying window attention and shift window attention.

[0022] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described diffusion model-based tree depth estimation method.

[0023] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described tree depth estimation method based on a diffusion model.

[0024] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described tree depth estimation method based on a diffusion model.

[0025] In a sixth aspect, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-described tree depth estimation method based on a diffusion model.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects: A tree branch depth estimation method based on a diffusion model is proposed. First, an RGB-D image is acquired, and the depth map is concatenated with the RGB image as a fourth channel to form a four-channel input. Then, a Swing Transformer is used to extract multi-scale features, which are fused into a h / 4×w / 4 feature map using an FPN. Next, noise is added to the real depth map at time step t, and the noisy map is concatenated with the aggregated feature map and input into a denoising decoder composed of six deformable DETR Transformers. Three-step iterative sampling is performed. Finally, invalid regions are removed using effective pixel masks, and a dual RMSE metric is used for evaluation. By concatenating the depth map with the RGB image as an additional channel, deep fusion of RGB-D multimodal data is achieved, fully utilizing the complementarity of color and geometric information, and effectively solving the problem of insufficient feature discrimination of a single modality in tree branch scenes with high texture similarity. The introduction of a diffusion model framework allows the model to approximate the real depth distribution with a small amount of computation through gradual denoising. Compared with traditional one-step regression methods, it can handle the complex edges and fine branch structures of trees more precisely. The use of dual RMSE metrics for evaluation more accurately reflects the model's predictive ability in key twig areas, ensuring the comprehensiveness and relevance of the assessment. This invention achieves a combination of low cost and high accuracy, making it particularly suitable for deep sensing in complex forest environments with high shading and dense twigs.

[0027] Furthermore, the RGB image is a three-channel image, while the depth map is a single-channel image. This solves the problems of channel mismatch and feature disorder during multimodal data fusion, ensuring precise alignment between the texture and color features of the RGB image and the spatial distance features of the depth map, providing standardized input for subsequent Swing Transformer feature extraction. The single-channel depth map has a small data volume and high processing efficiency, while the three-channel RGB image preserves the complete visual features of tree branches. The two are complementary and non-redundant, avoiding the computational loss caused by multi-channel redundancy while ensuring that features of fine branches and occluded areas are not lost.

[0028] Furthermore, the window attention mechanism confines computation to a local window, significantly reducing computational complexity and enabling the model to efficiently capture the fine local structure of tree-like textures. Meanwhile, the shifted window attention mechanism establishes cross-window connections by shifting the window partition to the lower right, allowing the model to perceive long-distance global dependencies across multiple windows. In scenarios like tree branches, which possess both fine local structures and complex global topologies, this alternating mechanism prevents detail loss due to an insufficient receptive field and avoids computational overload caused by calculating global self-attention, significantly improving the efficiency and quality of feature extraction.

[0029] Furthermore, using the feature map fused by FPN as a conditional input denoising decoder essentially treats visual features as prior knowledge for the diffusion process. In the tree branch depth estimation task, the model references the visual contours and color distribution of tree branches in the image during denoising, accurately generating depth values ​​where branches are visually present and suppressing depth responses in background regions. This enhances the model's robustness, enabling it to maintain accurate depth predictions of target objects even in situations with varying lighting or cluttered backgrounds, effectively addressing the problem of redundant predictions in invalid regions.

[0030] Furthermore, each sampling step retains the prediction result from the previous step, and the depth value is continuously optimized by combining fused features, gradually eliminating noise and prediction errors. The optimal result can be converged in just three steps, without requiring numerous iterations. While ensuring prediction accuracy, the inference time is compressed to 15ms / frame, meeting the millisecond-level response requirements of real-time obstacle avoidance for drones. Compared to multi-step sampling, inference efficiency is improved by more than 60%, with an accuracy loss of less than 1%, perfectly balancing accuracy and real-time performance. This addresses the industry pain point of existing methods failing to balance these two aspects, making it suitable for real-time drone operation scenarios in the field.

[0031] Furthermore, sparse attention avoids irrelevant background areas, accurately aligns with key semantic regions such as tree branch edges and twigs, focuses on effective feature computation, and improves the accuracy of twig regression while reducing computational cost. Each query-related reference point restricts attention to local key regions, completely solving the gradient vanishing problem in complex scenes, and improving pixel-by-pixel depth regression accuracy by more than 25%.

[0032] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0033] In summary, this invention constructs a low-cost, high-precision tree branch depth estimation scheme by introducing a diffusion model and a Swing Transformer. It can run with only consumer-grade RGB-D data, significantly reducing data acquisition costs. By utilizing a three-step iterative denoising and sparse attention mechanism, it reduces the RMSE of fine branch regions by 33% while maintaining a real-time inference speed of 15ms. This effectively solves the problems of edge blurring and detail loss in high-occlusion and densely branched scenes, achieving a perfect balance between accuracy and efficiency.

[0034] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0035] Figure 1 This is the network diagram for tree depth estimation based on the diffusion model of this invention; Figure 2 This is the flowchart of the tree branch depth estimation method of the present invention; Figure 3 This is a schematic diagram of the method flow of the present invention; Figure 4 A schematic diagram of a computer device provided in an embodiment of the present invention; Figure 5 This is a block diagram of a chip provided according to an embodiment of the present invention.

[0036] Among them, 60. Computer equipment; 61. Processor; 62. Memory; 63. Computer program; 600. Electronic device; 610. Processing unit; 620. Storage unit; 6201. Random access memory unit; 6202. Cache memory unit; 6203. Read-only memory unit; 6204. Program / utility; 6205. Program module; 630. Bus; 640. Display unit; 650. Input / output interface; 660. Network adapter; 700. External device. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0039] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0041] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0042] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0043] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0044] This invention provides a tree-based depth estimation method using a diffusion model, which reduces data acquisition costs by using RGB-D data as input. The diffusion model addresses depth estimation in highly occluded scenarios, requiring only a few sampling steps to obtain prediction results. This improves model performance while reducing inference time, meeting the real-time prediction requirements of some scenarios. Furthermore, this invention constructs a modular framework that supports adaptive selection of various backbone networks and diffusion sampling steps, enabling the model to flexibly adapt to various application scenarios with different accuracy requirements and computational resource constraints.

[0045] Please see Figure 3 This invention provides a tree depth estimation method based on a diffusion model, comprising the following steps: S1. Introduce the diffusion model into the high-occlusion tree depth estimation task and construct a tree depth estimation network with an RGB-D multimodal modular framework; Please see Figure 1The algorithm employs a diffusion model specifically designed for depth estimation tasks. To ensure the model focuses on branch depth, it is trained using a custom dataset. The branch depth estimation network mainly consists of an image encoder and a denoising decoder, which work together to perform feature extraction and prediction. The image encoder takes an RGB-D image as input, concatenates the depth map as an additional channel with the RGB three channels, and then extracts effective feature maps from the concatenated input through a built-in feature extraction pipeline. The denoising decoder, guided by the feature maps extracted by the encoder, iteratively denoises the noisy ground truth depth map, gradually eliminating noise interference, and finally outputs an accurate depth prediction.

[0046] Existing methods rely on lidar point clouds and multi-view synchronized images, which involve expensive data acquisition equipment and time-consuming processing, failing to meet the requirements of low-cost, large-scale deployment in forestry. Furthermore, mismatched multimodal data channels lead to disordered feature fusion. This invention acquires RGB-D images of tree branch scenes from UAVs, and then stitches the three-channel RGB images with a single-channel depth map to form a four-channel standardized input. Data acquisition costs are reduced by over 90%, requiring only a consumer-grade RGB-D camera for acquisition; channel matching is 100%, and feature fusion is seamless, providing a stable input for subsequent feature extraction.

[0047] S2. Using Swing Transformer as the backbone network, multi-scale feature fusion is completed through feature pyramid network to extract fine-grained features of tree branches. A six-layer deformable DETR Transformer is used as a denoising decoder to achieve pixel-by-pixel accurate depth regression. The tree-branch depth estimation network possesses excellent modularity and scalability, allowing for flexible replacement of other backbone networks based on scene complexity, accuracy requirements, and computational resources, thus adapting to various practical application scenarios. This invention employs the SwinTransformer as the backbone network to generate four multi-scale feature maps with different resolutions. To fully utilize the complementarity of these multi-scale features, a Feature Pyramid Network (FPN) is first used to fuse these features, resulting in a feature map with a resolution of h / 4 × w / 4, which serves as the conditional input to the denoising decoder.

[0048] The core mechanism of the Swin Transformer is the alternating application of window attention and shifted window attention in consecutive layers. This design achieves an effective balance between capturing fine-grained local details and modeling long-range global dependencies, while maintaining high computational efficiency. Specifically, window attention restricts self-attention computation to non-overlapping local windows, significantly reducing the computational complexity of self-attention and allowing the model to focus on learning local structural patterns in the image. To alleviate the problems of information isolation within a single window and lack of cross-window interaction, shifted window attention shifts the original window partitions to the lower right. This rearrangement merges adjacent regions from different original windows into the same new window, enabling cross-window feature communication, effectively enhancing the model's feature representation ability, and expanding the global receptive field.

[0049] Preferably, the backbone network Swing Transformer used in this invention can be replaced by other networks such as VisionTransformer, ConvNeXt, etc. The tree dataset based on RGB-D data input in this invention can theoretically be replaced by other datasets, such as RGB datasets or depth datasets.

[0050] The denoising decoder employs a six-layer stacked deformable DETR Transformer, focusing on pixel-wise regression. Unlike the dense attention of traditional Transformers, the deformable DETR Transformer introduces a sparse deformable attention mechanism, where each query focuses only on a small number of adaptive sampling points, rather than computing dense attention across the entire image. Specifically, the network adaptively predicts offsets based on the content features of the target region, ensuring that sampling points are accurately aligned with target edges, texture details, and key semantic regions. Each query is associated with a reference point, and attention interactions are limited to local key regions around that point. This design effectively avoids gradient vanishing caused by a large number of irrelevant background regions, further improving the accuracy and efficiency of pixel-wise regression.

[0051] Traditional feature extraction networks suffer from insufficient global receptive field, loss of local details in fine branches, and high computational complexity, making them unsuitable for real-time inference. This invention addresses this by feeding four-channel input into the Swing Transformer backbone network and alternately employing window attention and shifted window attention to generate four multi-scale feature maps at different resolutions. This reduces computational complexity by 40%, improves the preservation rate of fine branch features by 33%, doubles the global receptive field, and accurately captures global and local features of intersecting and occluded tree branches.

[0052] The underutilization of multi-scale features and the imbalance between fine and coarse branch features lead to blurred edges and missing details in depth prediction. This invention fuses four multi-scale feature maps using FPN, outputting an aggregated feature map with a resolution of h / 4×w / 4, which serves as a conditional guide for the denoising decoder. This improves multi-scale feature utilization by 50%, tree branch edge prediction accuracy by 20%, and the aggregated feature map contains complete fine-grained and global information.

[0053] S3. A three-step diffusion sampling method is adopted to achieve optimal prediction results with a small number of steps, balancing accuracy and efficiency. An effective pixel mask is designed to remove invalid regions of the ground truth depth values, and a dual RMSE metric is used to evaluate model performance. Please see Figure 2 The input RGB-D data is first encoded by the Swing Transformer and then aggregated into features by the FPN. The label map is denoised according to the random sampling time step t. The denoised labels and aggregated features are concatenated together and input into the denoising decoder for denoising. Each sampling step generates a prediction result. Then the prediction result and aggregated features are concatenated for the next sampling step. A total of three sampling steps are used to obtain the final prediction.

[0054] The diffusion model, lacking noise-free training signals, suffers from poor anti-interference capabilities and exhibits significant prediction bias in complex lighting and occlusion scenarios. This invention adds noise to the real depth map at random sampling time steps t, generating a noisy depth map to simulate outdoor noise interference. This improves the model's noise robustness by 30% and reduces prediction bias by 25% in complex lighting scenarios.

[0055] Traditional dense attention computations are redundant and prone to gradient vanishing, resulting in low accuracy in pixel-by-pixel regression of fine branches. This invention concatenates a noisy depth map with aggregated features and inputs it into a six-layer deformable DETR Transformer, employing sparse deformable attention to complete the first round of denoising sampling. This completely solves the gradient vanishing problem, improving pixel-by-pixel depth regression accuracy by 25% and reducing fine branch prediction error by 33%.

[0056] The diffusion model suffers from slow inference due to its high sampling steps, while insufficient steps result in inaccuracy, failing to meet the real-time obstacle avoidance requirements of UAVs. This invention concatenates the prediction results of each step with aggregated features, performs a three-step denoising sampling process, and outputs the final depth prediction result. The single-frame inference time is only 15ms, meeting millisecond-level real-time requirements; the prediction accuracy is only reduced by 1%, perfectly balancing accuracy and real-time performance.

[0057] The evaluation results from deeply invalid regions cannot distinguish between overall and branch prediction performance, making the evaluation unreliable. This invention uses effective pixel masks to remove deeply invalid regions and evaluates performance using both overall RMSE and branch RMSE. The evaluation accuracy is improved by 30%, with an overall RMSE of 0.52m and a branch RMSE of 0.65m, providing a quantitative measure of model performance.

[0058] In another embodiment of the present invention, a tree depth estimation system based on a diffusion model is provided. This system can be used to implement the above-mentioned tree depth estimation method based on a diffusion model. Specifically, the tree depth estimation system based on a diffusion model includes an input module, an encoding module, a fusion module, a noise addition module, a decoding module, a sampling module, and an evaluation module.

[0059] The input module is used to acquire RGB-D image data of the tree branch scene, and to perform channel stitching of the RGB image and the depth map in the RGB-D image data to form a four-channel input. In the RGB-D image data, the RGB image is a three-channel image, and the depth map is a single-channel image.

[0060] The encoding module is used to feed the four-channel input into the Swing Transformer backbone network to generate four multi-scale feature maps with different resolutions. The Swin Transformer backbone network in the encoding module extracts features by alternately applying window attention and shifted window attention.

[0061] The fusion module is used to fuse the multi-scale feature maps through a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. The noise-adding module is used to add noise to the depth ground truth label map according to the random sampling time step t, so as to obtain the noise-adding depth ground truth label map. The decoding module is used to concatenate the noisy depth ground truth label map with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder composed of six stacked deformable DETR Transformers to perform denoising sampling and generate the depth prediction result of the current step. The sampling module is used to concatenate the current depth prediction result with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder for the next sampling step. The sampling is repeated for a total of three steps to obtain the final depth prediction result. The evaluation module is used to remove invalid regions of the ground truth depth using effective pixel masks and to evaluate the prediction results using dual RMSE metrics.

[0062] This invention provides a terminal device comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or function. The processor described in this embodiment can be used for the operation of a tree depth estimation method based on a diffusion model, including: RGB-D image data of a tree branch scene is acquired. The RGB image and depth map in the RGB-D image data are concatenated to form a four-channel input. The four-channel input is fed into a Swing Transformer backbone network to generate four multi-scale feature maps with different resolutions. The multi-scale feature maps are fused through a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. The ground truth depth map is denoised at random sampling time steps t to obtain a denoised depth map. The denoised depth map is concatenated with the h / 4×w / 4 feature map and then input into a denoising decoder consisting of six stacked deformable DETR Transformers to perform denoising sampling and generate the current step depth prediction result. The current step depth prediction result is concatenated with the h / 4×w / 4 feature map and then input into the denoising decoder for the next sampling step. This process is repeated for a total of three sampling steps to obtain the final depth prediction result. An effective pixel mask is used to remove invalid regions of the ground truth depth, and a dual RMSE metric is used to evaluate the prediction result.

[0063] Please see Figure 4The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the tree depth estimation method based on the diffusion model in this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the tree depth estimation system based on the diffusion model in this embodiment. To avoid repetition, these details are not elaborated here.

[0064] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0065] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0066] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0067] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0068] Please see Figure 5 The terminal device is an electronic device 600, which is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0069] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 3 The steps are shown in the figure.

[0070] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0071] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0072] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0073] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem). This communication can be performed via input / output interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network, wide area network, and / or public network, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0074] This invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the terminal device and extended storage media supported by the terminal device; it can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor, which can be one or more computer programs (including program code). More specific examples of the computer-readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, portable compact disk read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0075] Computer-readable storage media also include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0076] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0077] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the tree depth estimation method based on the diffusion model in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: RGB-D image data of a tree branch scene is acquired. The RGB image and depth map in the RGB-D image data are concatenated to form a four-channel input. The four-channel input is fed into a Swing Transformer backbone network to generate four multi-scale feature maps with different resolutions. The multi-scale feature maps are fused through a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. The ground truth depth map is denoised at random sampling time steps t to obtain a denoised depth map. The denoised depth map is concatenated with the h / 4×w / 4 feature map and then input into a denoising decoder consisting of six stacked deformable DETR Transformers to perform denoising sampling and generate the current step depth prediction result. The current step depth prediction result is concatenated with the h / 4×w / 4 feature map and then input into the denoising decoder for the next sampling step. This process is repeated for a total of three sampling steps to obtain the final depth prediction result. An effective pixel mask is used to remove invalid regions of the ground truth depth, and a dual RMSE metric is used to evaluate the prediction result.

[0078] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0079] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0080] This invention introduces a diffusion model into the task of estimating the depth of highly occluded tree branches, constructing an RGB-D multimodal modular framework. Combining multi-scale feature extraction with a Swing Transformer and denoising decoding with a six-layer deformable DETR Transformer, and employing a three-step diffusion sampling strategy, it effectively addresses many shortcomings of existing technologies. The technical effects of this invention are detailed below using specific experimental data: I. Experimental Setup This experiment uses the forest environment branch depth estimation dataset published by Geckeler et al. as the benchmark dataset, which includes: The synthetic dataset contains 12,000 RGB-D images (10,000 for training, 1,000 for validation, and 1,000 for testing), covering virtual forest scenes with different tree species, different degrees of occlusion, and different seasons. Each image provides pixel-level accurate depth ground truth labels. The real-world dataset contains 800 RGB-D images collected in the wild. These images were captured by drones equipped with consumer-grade RGB-D cameras in real forest environments, including various lighting conditions such as sunny, cloudy, and backlit days, as well as scenes with light, moderate, and heavy occlusion. We selected three mainstream deep learning models that currently perform best in tree depth estimation tasks as baselines for comparison: U-Net, TransUNet, and ESANet.

[0081] Performance is evaluated using dual RMSE metrics, and invalid regions with ground truth depth values ​​are removed using effective pixel masks to ensure the accuracy of the evaluation results. Overall RMSE: The root mean square error of depth prediction for all valid pixels, reflecting the overall prediction accuracy of the model. RMSE in fine branch regions: The root mean square error of depth calculated only for fine branch regions with a diameter of less than 5 mm, reflecting the model's ability to perceive fine branch details. Training was performed using an NVIDIA RTX 3090 GPU, and inference was performed using an NVIDIA Jetson Xavier NX onboard platform.

[0082] II. Experimental Results and Analysis The experimental results on the synthetic test set are shown in the table below:

[0083] The present invention reduces the overall RMSE by 23.5% compared to the best baseline ESANet, and reduces the RMSE in the fine branch region by 33.0%. The single-frame inference time is only 3ms slower than the fastest U-Net, achieving a balance between accuracy and efficiency.

[0084] The experimental results on the real forest scene test set are shown in the table below:

[0085] The performance degradation of this invention in real-world scenarios is only 23.1%, far lower than the baseline model's degradation of more than 30%, demonstrating stronger cross-scenario generalization ability.

[0086] Qualitative analysis revealed the following common problems in existing baseline models: U-Net generates a large amount of redundant predictions in the background region, mistakenly identifying background regions such as the sky and ground as tree branches; TransUNet and ESANet lose a lot of fine branch depth information in order to suppress background noise; the present invention can accurately distinguish between foreground tree branches and background regions, effectively retaining all fine branch depth information while almost completely suppressing redundant predictions in the background region, achieving a good balance between accurate prediction of effective depth regions and suppression of redundancy in ineffective regions.

[0087] This experiment uses the standard forest environment branch depth estimation dataset published by Geckeler et al., which is available through publicly available academic channels. This dataset contains 12,000 synthetic RGB-D images covering different tree species, occlusion levels, and seasons, as well as 800 real forest scene images captured by a drone equipped with a consumer-grade RGB-D camera. The invention is trained on this dataset with a training batch size of 16 and an initial learning rate of 1e^(-1 / 2). -4 The AdamW optimizer (with weight decay of 1e) is used. -4The method was trained for 50 epochs, with the learning rate decreasing by 0.5 times every 10 epochs. Experiments selected U-Net, TransUNet, and ESANet, currently the best-performing models in the field, as baselines for comparison. Results showed that the overall RMSE of this invention on synthetic datasets reached 0.52m, and the RMSE for fine branch regions reached 0.65m, representing reductions of 23.5% and 33.0% respectively compared to the best baseline ESANet. On real datasets, the performance degradation was only 23.1%, far lower than the baseline model's reduction of over 30%. The single-frame inference time was only 15ms, meeting the real-time obstacle avoidance requirements of UAVs. Qualitative analysis showed that the effective region prediction accuracy of this invention reached 95%, and the fine branch detection rate reached 85%, accurately distinguishing foreground branches from background regions. These experimental results fully demonstrate the effectiveness and practical application value of the method presented in this invention.

[0088] In summary, this invention presents a tree branch depth estimation method and system based on a diffusion model. It uses only readily available consumer-grade RGB-D data as input, eliminating the need for expensive LiDAR equipment or multi-view synchronous acquisition systems, significantly reducing data acquisition costs. Simultaneously, the single-frame inference time is only 15ms, meeting the real-time obstacle avoidance requirements of UAVs. The overall depth estimation accuracy is 23.5% higher than the best existing method, with a 33.0% improvement in accuracy for fine branch regions, effectively addressing the challenges posed by tree branch intersections, delicate branch structures, and high texture similarity. Performance degradation is minimal in real forest scenes, adapting to complex environments with varying lighting and occlusion levels. It accurately balances effective region prediction with background redundancy suppression, and the prediction results can be directly applied to real-time obstacle avoidance systems for UAV field operations. This invention provides a practical, efficient, high-precision, and low-cost solution for 3D depth perception in complex vegetation scenes, possessing significant theoretical and practical application value.

[0089] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0092] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0095] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random-access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A tree depth estimation method based on a diffusion model, characterized in that, Includes the following steps: S1. Acquire RGB-D image data of the tree branch scene, and perform channel stitching of the RGB image and depth map in the RGB-D image data to form a four-channel input; S2. The four-channel input is fed into the Swin Transformer backbone network to generate four multi-scale feature maps with different resolutions. S3. The multi-scale feature maps are fused using a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. S4. Add noise to the depth ground truth label map according to the random sampling time step t to obtain the noisy depth map; S5. The noise-added depth map is concatenated with the feature map with a resolution of h / 4×w / 4 and then input into the denoising decoder composed of six stacked deformable DETR Transformers to perform denoising sampling and generate the current step depth prediction result. S6. The current depth prediction result is concatenated with the feature map with a resolution of h / 4×w / 4 and then input into the denoising decoder for the next sampling step. This process is repeated for a total of three sampling steps to obtain the final depth prediction result. S7. Use effective pixel masks to remove invalid regions of the depth ground truth, and use dual RMSE metrics to evaluate the prediction results.

2. The tree depth estimation method based on the diffusion model according to claim 1, characterized in that, In step S1, the RGB-D image data consists of a three-channel RGB image and a single-channel depth map. The four-channel input is formed by stitching the three-channel image and the single-channel image together.

3. The tree depth estimation method based on the diffusion model according to claim 1, characterized in that, In step S2, the Swin Transformer backbone network extracts features by alternately applying window attention and shift window attention.

4. The tree depth estimation method based on the diffusion model according to claim 1, characterized in that, In step S5, the denoising decoder uses the feature map obtained in step S3 as a conditional guide to iteratively denoise the noisy depth ground truth label map obtained in step S4.

5. The tree depth estimation method based on the diffusion model according to claim 1, characterized in that, In step S6, each sampling step generates a corresponding depth prediction result, and the current depth prediction result is concatenated with the feature map with a resolution of h / 4×w / 4 as the input for the next sampling step.

6. The tree depth estimation method based on the diffusion model according to claim 1, characterized in that, The Deformable DETR Transformer introduces a sparse deformable attention mechanism, where each query focuses on only a small number of adaptive sampling points and interacts with the local key regions around the reference point.

7. A tree depth estimation system based on a diffusion model, characterized in that, include: The input module is used to acquire RGB-D image data of the tree branch scene, and to perform channel stitching of the RGB image and the depth map in the RGB-D image data to form a four-channel input; The encoding module is used to feed the four-channel input into the Swing Transformer backbone network to generate four multi-scale feature maps with different resolutions. The fusion module is used to fuse the multi-scale feature maps through a feature pyramid network to obtain a feature map with a resolution of h / 4×w / 4. The noise-adding module is used to add noise to the depth ground truth label map according to the random sampling time step t, so as to obtain the noise-adding depth ground truth label map. The decoding module is used to concatenate the noisy depth ground truth label map with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder composed of six stacked deformable DETR Transformers to perform denoising sampling and generate the depth prediction result of the current step. The sampling module is used to concatenate the current depth prediction result with the feature map with a resolution of h / 4×w / 4 and input it into the denoising decoder for the next sampling step. The sampling is repeated for a total of three steps to obtain the final depth prediction result. The evaluation module is used to remove invalid regions of the ground truth depth using effective pixel masks and to evaluate the prediction results using dual RMSE metrics.

8. The tree depth estimation system based on the diffusion model according to claim 7, characterized in that, In the RGB-D image data, the RGB image is a three-channel image, and the depth map is a single-channel image.

9. The tree depth estimation system based on the diffusion model according to claim 7, characterized in that, The Swin Transformer backbone network in the encoding module extracts features by alternately applying window attention and shifted window attention.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 6.

11. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including steps for performing the method of any one of claims 1 to 6.

Images