Tree crown three-dimensional reconstruction method and system based on multi-scale feature fusion

Abstract

The invention provides a tree crown three-dimensional reconstruction method and system based on multi-scale feature fusion, and the method comprises the steps: firstly collecting a tree crown collection data set which comprises laser radar scanning point cloud data, multispectral image data and ambient light field distribution data, and then carrying out the multi-scale feature extraction of the tree crown collection data set; a tree crown point cloud local structure feature, a multispectral image spectral reflection feature and an ambient light field illumination attenuation feature are obtained, then a feature fusion network is used for carrying out dynamic weight distribution fusion on the features, and a tree crown multi-scale fusion feature is generated. Training a three-dimensional reconstruction model based on the features to generate a crown three-dimensional grid model, finally calling the crown three-dimensional grid model to perform three-dimensional reconstruction on real-time acquired data of a target crown, and outputting a dynamic three-dimensional model of the target crown and a structure parameter optimization suggestion, so as to obtain a dynamic three-dimensional model of the target crown through multi-scale feature fusion. And the three-dimensional reconstruction of the tree crown can be realized more accurately and optimization suggestions can be given.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for three-dimensional reconstruction of tree crowns based on multi-scale feature fusion. Background Art

[0002] In the field of forestry resource management and ecological monitoring, tree crown 3D reconstruction technology is gaining increasing attention as a key means of obtaining tree spatial structure information, assessing growth status, and evaluating ecological and environmental effects. Traditional tree crown 3D reconstruction methods mainly rely on a single data source, such as LiDAR scanning point cloud data or optical image data. Although these methods can reconstruct tree crown morphology to a certain extent, they are often limited by the single data source and often cannot fully capture the complex structure and spectral characteristics of the tree crown.

[0003] Furthermore, existing technologies often rely on simple data splicing or fixed-weight fusion strategies when processing multi-source heterogeneous data. These strategies fail to fully account for the complementarities and differences between different data sources, thus limiting the reconstruction model's ability to capture the complex characteristics of tree crowns. This is particularly true for species like the oil-tea canopy, which exhibits unique growth morphology and spectral characteristics. Traditional methods struggle to accurately capture subtle structural variations and spectral response characteristics, making them difficult to meet the demands of precise forestry management and ecological assessment. Summary of the Invention

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a method for 3D reconstruction of tree crowns based on multi-scale feature fusion, the method comprising:

[0005] Collecting a tree crown data set, wherein the tree crown data set includes laser radar scanning point cloud data, multispectral image data, and ambient light field distribution data;

[0006] Performing multi-scale feature extraction processing on the tree crown collection data set to obtain local structural features of the tree crown point cloud set, spectral reflectance features of the multispectral image data, and light attenuation features of the ambient light field distribution data;

[0007] The local structural features, the spectral reflectance features and the light attenuation features are dynamically weighted and fused through a feature fusion network to generate a multi-scale fusion feature of the crown;

[0008] Training a three-dimensional reconstruction model based on the multi-scale fusion features of the tree crown to generate a three-dimensional grid model of the tree crown;

[0009] The three-dimensional reconstruction model is called to perform three-dimensional reconstruction processing on the real-time collected data of the target tree crown, and a dynamic three-dimensional model of the target tree crown and structural parameter optimization suggestions are output.

[0010] On the other hand, an embodiment of the present invention also provides a tree crown three-dimensional reconstruction system based on multi-scale feature fusion, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0011] Based on the above aspects, the embodiment of the present application significantly improves the accuracy and dynamic adaptability of the three-dimensional reconstruction of the tree crown by integrating the laser radar scanning point cloud data, multi-spectral image data and ambient light field distribution data. Specifically, through multi-scale feature extraction and processing, not only the local structural characteristics of the tree crown point cloud set are captured, but also the spectral reflectance characteristics of the multi-spectral image and the light attenuation law of the ambient light field distribution are deeply excavated, thereby achieving a comprehensive characterization of the tree crown morphology, material and lighting environment. Furthermore, the feature fusion network organically integrates the above-mentioned multi-source heterogeneous features into the multi-scale fusion features of the tree crown through a dynamic weight distribution mechanism, effectively overcoming the limitations of a single data source in describing complex tree crown structures, and enhancing the robustness and generalization ability of feature expression. The three-dimensional reconstruction model trained based on this fusion feature can generate a highly detailed three-dimensional grid model of the tree crown that conforms to actual physical properties. By dynamically reconstructing the real-time collected data of the target tree crown and outputting structural parameter optimization suggestions, it not only achieves the leap from static modeling to dynamic monitoring, but also provides an intelligent decision-making basis for applications such as tree crown growth status assessment and pest and disease warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the execution flow of the tree crown three-dimensional reconstruction method based on multi-scale feature fusion provided by an embodiment of the present invention.

[0013] Figure 2 Schematic diagram of exemplary hardware and software components of a tree crown 3D reconstruction system based on multi-scale feature fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This is a flow chart of a method for three-dimensional reconstruction of tree crowns based on multi-scale feature fusion provided by an embodiment of the present invention. The method for three-dimensional reconstruction of tree crowns based on multi-scale feature fusion is introduced in detail below.

[0015] Step S110: collecting a tree crown collection data set, wherein the tree crown collection data set includes lidar scanning point cloud data, multispectral image data and ambient light field distribution data.

[0016] In the application scenario of this embodiment, a typical tea tree in a tea tree planting base in a mountainous area is taken as an example. First, a laser radar device is used to scan the tea tree crown. The laser radar can emit 5,000 laser beams per second to detect the tree crown in all directions. During the scanning process, the laser beam hits various parts of the crown and reflects back. The laser radar device accurately calculates the distance information of each reflection point based on the reflection time, and then generates the laser radar scanning point cloud data. For example, during the scanning process, the laser radar device records a reflection point 2 meters away from the device. Its coordinates in the three-dimensional coordinate system established with the laser radar as the origin may be (1.5, 1.2, 2.0). As the scanning continues, a large number of such points converge into point cloud data, which details the outline of the crown and the spatial position of each part, forming a point cloud set of the crown.

[0017] Next, a multispectral camera equipped with multiple lenses for specific wavelengths can be used to collect multispectral image data. The multispectral camera is set to automatically adapt to different lighting conditions and can automatically adjust exposure parameters according to the ambient brightness and color distribution. When shooting, the multispectral camera simultaneously captures light information from multiple bands, covering the red, green, blue and near-infrared bands of visible light. For example, at a certain moment in the shooting, the camera records the reflectance data of the red band, which reflects the reflectance characteristics of the tree canopy under red light. Similarly, the green, blue and near-infrared bands also record corresponding reflectance information. The reflectance data of the above different bands together constitute the multispectral image data.

[0018] Finally, ambient light field distribution data can be collected by placing multiple ambient light field sensors around the oil tea tree. The ambient light field sensors are distributed at different heights and angles around the tree canopy, and can monitor light intensity, light direction, and light scattering in real time. For example, an ambient light field sensor located 1 meter above the tree canopy records a light intensity of 800 lux, with the light direction being horizontal from due south. It also records a light scattering coefficient of 0.6. After integrating the data collected by multiple sensors, ambient light field distribution data is formed that can fully reflect the ambient light field conditions of the tree canopy.

[0019] Step S120: performing multi-scale feature extraction processing on the tree crown collection data set to obtain local structural features of the tree crown point cloud set, spectral reflectance features of the multispectral image data, and light attenuation features of the ambient light field distribution data.

[0020] In this embodiment, step S120 may include:

[0021] Step S121: performing density adaptive clustering processing on the laser radar scanning point cloud data, dividing it into multiple point cloud clusters, and extracting the geometric center coordinates, normal vector distribution and curvature change gradient of each point cloud cluster to generate the local structural features of the crown point cloud set.

[0022] A density-adaptive clustering algorithm is used to process LiDAR scanned point cloud data. Specifically, using this oil-tea tree crown point cloud data as an example, this density-adaptive clustering algorithm automatically identifies different point cloud clusters based on the density of the point cloud distribution. For example, the point cloud distribution is relatively sparse at the edge of the crown, while the point cloud distribution is denser in the inner area near the trunk. After computational analysis, the algorithm segmented the entire point cloud data into 10 clusters.

[0023] For each point cloud cluster, local structural features are extracted. For example, for a point cloud cluster containing 200 points, the geometric center coordinates are calculated by summing the x, y, and z coordinates of these 200 points and dividing by the number of points, 200. Assuming the sum of the x coordinates of these 200 points is 300, the sum of the y coordinates is 400, and the sum of the z coordinates is 500, the geometric center coordinates are (300 ÷ 200, 400 ÷ 200, 500 ÷ 200), or (1.5, 2.0, 2.5).

[0024] Normal vector distribution extraction involves applying a mathematical calculation method to each point in a point cloud cluster to obtain the normal vector at that point. For example, for point (1.6, 2.1, 2.6), according to the normal vector calculation rules, its normal vector is (0.4, 0.5, 0.7). This gives the entire point cloud cluster a series of normal vector data reflecting the surface orientation of each point, forming the normal vector distribution.

[0025] The curvature gradient is calculated by comparing the curvature differences between adjacent points. For example, if the curvature of a point in a point cloud cluster is 0.3 and the curvature of its adjacent point is 0.4, then the curvature gradient between these two points is 0.4 - 0.3 = 0.1. This calculation is performed for all adjacent points in the entire point cloud cluster to obtain the curvature gradient data. The geometric center coordinates, normal vector distribution, and curvature gradient together constitute the local structural characteristics of the point cloud cluster and are also part of the local structural characteristics of the tree crown point cloud collection.

[0026] Step S122: performing band separation processing on the multispectral image data, extracting the reflectance difference between the visible light band and the near-infrared band, and generating spectral reflectance characteristics of the multispectral image data in combination with a vegetation index calculation model.

[0027] In this embodiment, when processing multispectral image data, a band separation operation is first performed. For example, image processing software can be used to separate the individual bands in the collected multispectral image data. For example, the red band image, the green band image, the blue band image, and the near-infrared band image can be separately extracted.

[0028] Next, calculate the difference in reflectivity between the visible light band and the near-infrared band. For example, if the red band reflectivity is 0.3 and the near-infrared band reflectivity is 0.7, the difference in reflectivity between them is 0.7-0.3=0.4.

[0029] Next, the spectral reflectance characteristics are generated by combining the vegetation index calculation model. The common normalized difference vegetation index (NDVI) calculation model can be used here, and the formula is NDVI = (near-infrared band reflectance - red band reflectance) ÷ (near-infrared band reflectance + red band reflectance). Substituting the above red band reflectance of 0.3 and the near-infrared band reflectance of 0.7 into the formula, we get NDVI = (0.7-0.3) ÷ (0.7+0.3) = 0.4 ÷ 1 = 0.4. At the same time, similar calculations are performed on the green band and near-infrared band, and the blue band and near-infrared band to obtain a series of vegetation index data with different combinations. The vegetation index data calculated from the above different band combinations, as well as the reflectance difference data calculated previously, together constitute the spectral reflectance characteristics of the multispectral image data.

[0030] Step S123: performing ray tracing simulation processing on the ambient light field distribution data, calculating the light penetration depth according to the spatial occlusion relationship between the light source position and the tree crown point cloud, and generating the light attenuation characteristics of the ambient light field distribution data.

[0031] When processing ambient light field distribution data, a ray tracing simulation algorithm was used. First, the light source position was determined. Assuming the sun is the primary light source, its position relative to the canopy of the tea tree at a certain moment is from the southeast at an angle of 30 degrees.

[0032] Then, based on the spatial position information of the canopy point cloud, the occlusion relationship between the light source and the canopy is analyzed. For example, some branches and leaves on the outer layer of the canopy may block some light, preventing it from directly reaching the inner canopy. Using a ray tracing simulation algorithm, the penetration depth of light from the light source as it passes through various parts of the canopy is calculated.

[0033] Imagine a ray of light traveling from a light source toward a tree canopy and encountering a branch or leaf represented by a point cloud. Based on the positional relationship and angle between the ray and the branch or leaf, we determine whether the ray is obstructed. If so, we record the distance the ray penetrated before being blocked. This distance represents the ray's penetration depth. By performing numerous such calculations for light rays from different directions and positions, we generate a series of ray penetration depth data. This data reflects the ray's penetration into different parts of the canopy, thereby generating the illumination attenuation characteristics of the ambient light field distribution data.

[0034] Step S124: aligning the local structural features, the spectral reflectance features, and the light attenuation features according to spatial coordinates and storing them in a fusion feature cache queue.

[0035] After obtaining the local structure features, spectral reflectance features, and light attenuation features, they need to be aligned according to spatial coordinates. For example, for the geometric center coordinates (1.5, 2.0, 2.5) of a point cloud cluster in the local structure features, find the spectral reflectance feature data and light attenuation feature data at the corresponding spatial location in the multispectral image data and ambient light field distribution data.

[0036] Assume that at this spatial location, the spectral reflectance feature data includes reflectance differences and vegetation index data for multiple bands, such as a reflectance difference of 0.4 between the red and near-infrared bands and a normalized vegetation index of 0.4; and that the light attenuation feature data has a light penetration depth of 1.2 meters. These different types of feature data corresponding to spatial locations can then be organized together and stored in a fusion feature cache queue in a specific format. This fusion feature cache queue can be understood as a temporary data storage area for subsequent feature fusion processing, ensuring that different types of feature data accurately correspond to each other in space.

[0037] Step S130: The local structural features, the spectral reflectance features and the light attenuation features are fused by dynamic weight allocation through a feature fusion network to generate a multi-scale fusion feature of the crown.

[0038] Step S131: Initialize the local structure feature weight coefficient, the spectral reflection feature weight coefficient and the light attenuation feature weight coefficient, wherein the local structure feature weight coefficient is positively correlated with the point cloud density of the crown point cloud set, the spectral reflection feature weight coefficient is negatively correlated with the vegetation index of the multispectral image data, and the light attenuation feature weight coefficient is exponentially related to the light penetration depth of the ambient light field distribution data.

[0039] Before performing feature fusion, this embodiment first initializes the weight coefficient of each feature. In detail, for the weight coefficient of the local structural feature, since it is positively correlated with the point cloud density of the crown point cloud set. Assume that in this oil-tea canopy, after statistical analysis, the average value of the overall point cloud density is 100 points per cubic meter. For a region with a point cloud density of 150 points per cubic meter, according to the pre-set positive correlation rule, the local structural feature weight coefficient of the region is initialized to 0.6 (the higher the point cloud density, the larger the weight coefficient).

[0040] As for the spectral reflectance feature weight coefficient, since it is negatively correlated with the vegetation index of the multispectral image data, taking the previously calculated normalized vegetation index of 0.4 as an example, according to the negative correlation rule, assuming that the higher the vegetation index, the smaller the weight coefficient, the spectral reflectance feature weight coefficient is initialized to 0.3.

[0041] The light attenuation feature weight coefficient is exponentially related to the light penetration depth of the ambient light field distribution data. Assuming that the light penetration depth at a certain location is 1.5 meters, according to the pre-set exponential relationship formula (assuming that the weight coefficient = 0.5^light penetration depth), the light attenuation feature weight coefficient at that location is calculated to be 0.5^1.5≈0.35 (this is just an example to illustrate the exponential relationship calculation process, and the formula in actual application can be more complex). In this way, the weight coefficients of each feature at different spatial positions are initialized, providing a basis for subsequent dynamic weight adjustment and feature fusion.

[0042] Step S132: performing the following operations on each spatial coordinate unit in the fusion feature cache queue:

[0043] Step S1321: Dynamically adjust the weight coefficient of the local structural feature according to the curvature change gradient of the local structural feature.

[0044] In the fusion feature cache queue, for each spatial coordinate unit, the weight coefficient is dynamically adjusted based on the curvature gradient of the local structural feature. For example, in a certain spatial coordinate unit, the current curvature gradient is calculated to be 0.2, and the global average curvature is known to be 0.15. The ratio of the curvature gradient of the current spatial coordinate unit to the global average curvature is calculated, that is, 0.2÷0.15≈1.33. Assuming the first threshold is set to 1.2, since this ratio is greater than the first threshold, the local structural feature weight coefficient is increased by the first gain according to the rule. Assuming that the first gain is dynamically adjusted based on the point cloud density distribution of the crown point cloud set, and the current point cloud density is 0.1, the first gain is adjusted. Then, the adjusted local structural feature weight coefficient changes from the initial 0.6 to 0.6+0.1=0.7. The first gain and first attenuation here must satisfy the balance constraint between the total gain and the total attenuation to ensure the rationality and stability of the weight coefficient adjustment.

[0045] Step S1322: Dynamically adjust the spectral reflection feature weight coefficient according to the band reflectivity difference of the spectral reflection feature.

[0046] For the same spatial coordinate unit, the weight coefficient is then adjusted according to the difference in the band reflectance of the spectral reflectance feature. The deviation between the vegetation index of the current spatial coordinate unit and the healthy vegetation index baseline value is calculated. Assuming that the healthy vegetation index baseline value is 0.5 and the currently calculated vegetation index is 0.3, the deviation is (0.5-0.3)÷0.5=0.4. Assuming that the third threshold is set to 0.3, since the deviation exceeds the third threshold, the spectral reflectance feature weight coefficient is increased by the second gain according to the rule. Assuming that the second gain is dynamically corrected according to the time series data of the multispectral image data, in the case of the current time series data, the second gain is 0.1. The adjusted spectral reflectance feature weight coefficient changes from the initialized 0.3 to 0.3+0.1=0.4. If the vegetation index is lower than the withering warning threshold, for example, the withering warning threshold is 0.2, when the vegetation index is 0.1, the spectral reflectance feature weight coefficient is reduced by the second attenuation amount. Assuming that the second attenuation amount is 0.15 after dynamic correction, the adjusted weight coefficient is 0.3-0.15=0.15.

[0047] Step S1323: Dynamically adjust the light attenuation feature weight coefficient according to the light penetration depth of the light attenuation feature.

[0048] Similarly, within this spatial coordinate unit, the weight coefficient of the illumination attenuation feature is adjusted based on the light penetration depth. The light penetration depth for the current spatial coordinate unit is calculated to determine the effective illumination coverage area ratio. Assuming the light penetration depth for this spatial coordinate unit is 1.0 meter, the calculated effective illumination coverage area ratio is 0.4. Assuming the fourth threshold is set to 0.5, since this ratio is below the fourth threshold, the light attenuation feature weight coefficient is increased by the third gain amount according to the rule. Assuming the third gain amount is periodically adjusted based on seasonal variations in the ambient light field distribution data, and in the current season, the third gain amount is 0.1, the adjusted light attenuation feature weight coefficient changes from the initial value of 0.35 to 0.35 + 0.1 = 0.45. If the light penetration depth reaches the saturation threshold, for example, 2.0 meters, the light attenuation feature weight coefficient is reduced by the third attenuation amount. Assuming the third attenuation amount is 0.2 after periodic adjustment, the adjusted weight coefficient becomes 0.35 - 0.2 = 0.15.

[0049] Step S1324: Based on the adjusted local structure feature weight coefficient, the spectral reflection feature weight coefficient and the light attenuation feature weight coefficient, the local structure feature, the spectral reflection feature and the light attenuation feature are weightedly fused through a feature fusion network to generate a fusion feature vector of the current spatial coordinate unit.

[0050] After dynamically adjusting the weight coefficients of each feature, the local structure features, spectral reflectance features, and light attenuation features are weighted and fused through the feature fusion network. For example, for the current spatial coordinate unit, the local structure feature vector can be expressed as (geometric center coordinate vector, normal vector distribution vector, curvature change gradient vector), assuming the geometric center coordinate vector is (1.5, 2.0, 2.5), the normal vector distribution vector is (0.4, 0.5, 0.7), and the curvature change gradient vector is (0.2); the spectral reflectance feature vector is (reflectance difference vector, vegetation index vector), assuming the reflectance difference vector is (0.4) and the vegetation index vector is (0.3); the light attenuation feature vector is (light penetration depth vector), assuming the light penetration depth vector is (1.0).

[0051] The adjusted weight coefficients for the local structure feature, spectral reflectance feature, and illumination attenuation feature are 0.7, 0.4, and 0.45, respectively. Through weighted fusion, each dimension of the local structure feature vector is multiplied by its weight coefficient of 0.7, each dimension of the spectral reflectance feature vector is multiplied by its weight coefficient of 0.4, and each dimension of the illumination attenuation feature vector is multiplied by its weight coefficient of 0.45. These weighted vectors are then concatenated in sequence. For example, the fused vector is (0.7×1.5, 0.7×2.0, 0.7×2.5, 0.7×0.4, 0.7×0.5, 0.7×0.7, 0.7×0.2, 0.4×0.4, 0.4×0.3, 0.45×1.0), yielding the fused feature vector for the current spatial coordinate unit.

[0052] Step S1325: Connect the fused feature vectors of all spatial coordinate units according to the topological relationship to generate the crown multi-scale fusion feature.

[0053] The fused feature vectors generated by all spatial coordinate units in the fused feature cache queue are connected according to their topological relationship in the canopy space. For example, the fused feature vectors of adjacent spatial coordinate units are sequentially connected based on the actual structure and spatial position relationship of the canopy. This is similar to building a three-dimensional structural framework, in which each fused feature vector is a node. Through reasonable connection methods, a complete fused feature set that reflects the multi-scale characteristics of the canopy is formed. This fused feature set is the canopy multi-scale fusion feature, which integrates multiple feature information such as the local structure, spectral reflectance, and light attenuation of the canopy, providing a rich and comprehensive data foundation for subsequent 3D reconstruction model training.

[0054] Step S140: training a three-dimensional reconstruction model based on the multi-scale fusion features of the tree crown to generate a three-dimensional mesh model of the tree crown.

[0055] Step S141: constructing a 3D reconstruction network architecture including an encoder and a decoder, wherein the encoder is composed of a multi-scale convolution module and a feature pyramid module, and the decoder is composed of a voxel generation module and a surface refinement module.

[0056] When building the 3D reconstruction network architecture, the encoder is constructed first. The multi-scale convolution module uses convolution kernels of varying sizes to convolve the input multi-scale fused features of the tree crown. For example, three different convolution kernel sizes are used: 3×3, 5×5, and 7×7. For example, a 3×3 convolution kernel slides over the fused feature data, performing a weighted sum of the data within a 3×3 region with each slide to generate new feature data. In this way, feature information at different scales is extracted.

[0057] The feature pyramid module is used to fuse features of different scales generated by the multi-scale convolution module. It combines feature maps of different scales according to certain rules, such as upsampling or downsampling small-scale feature maps to make them dimensionally matched. It then concatenates the data at corresponding locations to achieve cross-level feature fusion. This allows the encoder to fully extract and integrate various information from the multi-scale fused features of the tree crown.

[0058] Next, the decoder is constructed. The voxel generation module generates an initial voxel model based on the fused three-dimensional spatial features output by the encoder. For example, the fused feature data can be mapped into a three-dimensional voxel space, and the state of each voxel (such as whether it is part of the tree canopy) can be determined based on the distribution of the features. Assuming that in a 10×10×10 voxel space, based on the analysis of the fused features, some voxels are determined to be "part of the tree canopy", these voxels form the basis of the initial voxel model.

[0059] The surface refinement module smoothes the edges of voxels. In the initial voxel model, the boundaries between voxels may be relatively rigid. The surface refinement module will adjust the above boundaries through a set algorithm. For example, for two adjacent voxels, if one is marked as "with crown part" and the other as "without crown part", the surface refinement module will make an appropriate smooth transition on the boundary between the two voxels based on the situation of the surrounding voxels and the information in the fusion features, so that the generated model surface is more natural and in line with the actual crown shape.

[0060] Step S142: Input the crown multi-scale fusion feature into the encoder, extract three-dimensional spatial features of different levels through the multi-scale convolution module, and perform cross-level feature fusion through the feature pyramid module to generate fused three-dimensional spatial features.

[0061] The previously obtained multi-scale fused features of the tree crown are input into the constructed encoder. The multi-scale convolution module begins operation. Taking a 3×3 convolution kernel as an example, it slides over the fused feature data with a stride of 1. For each sliding 3×3 region, each element in the convolution kernel is multiplied by the fused feature data at the corresponding position, and these products are then added together to obtain a new value. For example, in a certain 3×3 region, the values ​​of the fused feature data are (0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0), and the elements in the 3×3 convolution kernel are (0.1, 0.2, 0.1, 0.2, 0.4, 0.2, 0.1, 0.2, 0.1). Then, through multiplication and addition, the new value is (0.2×0.1+0.3×0.2+0.4×0.1+0.5×0.2+0.6×0.4+0.7×0.2+0.8×0.1+0.9×0.2+1.0×0.1)=0.68. In this way, the 3×3 convolution kernel slides over the entire fused feature data to generate a new feature map that extracts information of the fused feature at a smaller scale.

[0062] Similarly, the 5×5 and 7×7 convolution kernels also perform similar operations to generate feature maps at different scales. These feature maps at different scales contain information on the multi-scale fusion features of the tree crown at different levels of detail.

[0063] The feature pyramid module then performs cross-level feature fusion on the feature maps of these different scales. Assume that the feature map generated by the 3×3 convolution kernel is denoted as F1, the feature map generated by the 5×5 convolution kernel is denoted as F2, and the feature map generated by the 7×7 convolution kernel is denoted as F3. First, F2 is downsampled to the same size as F1. Downsampling can be achieved through methods such as average pooling, for example, by averaging the values ​​within each 2×2 region in F2 to obtain a new value, thereby reducing the size of F2. The downsampled F2 is then concatenated with F1, arranging the elements at corresponding positions in sequence to form a new feature map F12. Similarly, F3 is downsampled to the same size as F12 and then concatenated with F12 to generate a fused three-dimensional spatial feature. This 3D spatial feature integrates feature information at different scales and more comprehensively reflects the structure and characteristics of the tree canopy.

[0064] Step S143: inputting the fused three-dimensional spatial features into the decoder, generating an initial voxel model through the voxel generation module, and smoothing the voxel edges through the surface refinement module.

[0065] The fused three-dimensional spatial features are input into the decoder's voxel generation module. Based on the aforementioned feature information, the voxel generation module determines the state of each voxel in a pre-set three-dimensional voxel space. For example, for a 20×20×20 voxel space, the voxel generation module will traverse each voxel. Based on information about the crown structure, density, and other aspects of the fused three-dimensional spatial features, each voxel is determined to be a tree crown. Assuming that at a certain voxel location, the fused feature data indicates that the location has a high crown feature value, the voxel generation module will mark the voxel as "with a crown"; conversely, if the feature value is low, it will be marked as "without a crown." By processing the entire voxel space, an initial voxel model is generated.

[0066] Next, the surface refinement module smoothes the voxel edges of the initial voxel model. In the initial voxel model, there may be noticeable discontinuities between voxel boundaries. For example, two adjacent voxels, one labeled "with a canopy" and the other labeled "without a canopy," have a hard boundary. The surface refinement module adjusts based on the conditions of surrounding voxels and information from the fused features. It analyzes the state of voxels within a certain range, such as a 3×3×3 area centered on the current boundary. If it finds that most of the surrounding voxels tend to be "with a canopy," the surface refinement module adjusts the boundary appropriately, shifting the voxels labeled "without a canopy" slightly toward "with a canopy," thereby achieving a smoother transition at the boundary. By performing this processing on all voxel edges throughout the initial voxel model, the generated model surface becomes more natural and resembles the morphology of a real tree canopy.

[0067] Step S144: Calculate the geometric difference loss between the initial voxel model and the true tree crown 3D model, and train the parameters of the 3D reconstruction network architecture in combination with the surface curvature consistency loss.

[0068] After generating the initial voxel model, it is necessary to calculate the geometric difference loss between it and the true canopy 3D model. The true canopy 3D model can be obtained using high-precision measurement equipment or through detailed modeling by professionals. When calculating the geometric difference loss, the geometric information of the corresponding positions in the initial voxel model and the true canopy 3D model are compared. For example, for a voxel position (x, y, z) in the initial voxel model, the corresponding position in the true canopy 3D model is found and the difference in their spatial coordinates is compared. Assuming that the coordinates of the true canopy 3D model at this position are (x+0.1, y-0.05, z+0.08), then this coordinate difference is part of the geometric difference. By traversing all voxel positions in the initial voxel model, the coordinate differences of all corresponding positions are calculated, and the above differences are comprehensively calculated (for example, by taking the square sum and then averaging), the value of the geometric difference loss is obtained.

[0069] At the same time, the surface curvature consistency loss is combined to train the network parameters. Surface curvature reflects the degree of curvature of the model surface. For the initial voxel model and the true tree crown 3D model, the curvature of their surfaces is calculated separately. For example, the curvature of a voxel surface in the initial voxel model is calculated to be 0.2 using a specific algorithm, and the curvature of the corresponding position in the true tree crown 3D model is 0.25. Then, the curvature difference between them is part of the surface curvature consistency loss. Similarly, the entire model surface is traversed, the curvature differences of all corresponding positions are calculated, and a comprehensive calculation is performed to obtain the value of the surface curvature consistency loss.

[0070] The geometric difference loss and surface curvature consistency loss are combined to form the basis for training the parameters of the 3D reconstruction network architecture. For example, an optimization algorithm such as stochastic gradient descent is used to adjust the weight parameters in the network based on these two loss values. If the geometric difference loss is large, it means that the initial voxel model generated by the current network differs significantly in geometry from the true 3D tree canopy model. In this case, the optimization algorithm will adjust the weight parameters so that the network can generate a model closer to the true geometry the next time it generates the model. Similarly, for the surface curvature consistency loss, the weight parameters will be adjusted so that the surface curvature of the generated model is closer to that of the true 3D tree canopy model.

[0071] Step S145: When the geometric difference loss is lower than a preset threshold, the current parameters are saved as the 3D reconstructed model.

[0072] During training, the geometric difference loss and surface curvature consistency loss are continuously calculated, and the parameters of the 3D reconstruction network architecture are adjusted based on these loss values. A threshold for the geometric difference loss is preset, for example, 0.05. When the calculated geometric difference loss value falls below this threshold after multiple training cycles, the initial voxel model generated by the network is geometrically very close to the true 3D tree canopy model. At the same time, the surface curvature consistency loss is within an acceptable range, indicating that the network training has achieved good results.

[0073] Therefore, the parameters of the current 3D reconstruction network architecture are saved, and these parameters constitute the final 3D reconstruction model. The 3D reconstruction model can be used for subsequent 3D reconstruction of the target tree crown, and can generate a more accurate and actual tree crown 3D grid model.

[0074] Step S150: calling the three-dimensional reconstruction model to perform three-dimensional reconstruction processing on the real-time collected data of the target tree crown, and outputting a dynamic three-dimensional model of the target tree crown and structural parameter optimization suggestions.

[0075] Step S151: Collect real-time lidar point cloud data, real-time multispectral image data, and real-time ambient light field data of the target tree crown.

[0076] The first step in 3D reconstruction of the target tree canopy is to collect real-time data. Using the same equipment used to collect the data, the target tree canopy is scanned and photographed in real time. The LiDAR scans the target tree canopy in all directions at a rate of 4,000 laser beams per second, acquiring real-time LiDAR point cloud data. For example, during the scan, the point cloud coordinates recorded for a specific location in the target tree canopy are (1.8, 2.2, 2.8). As the scan progresses, a large amount of this point cloud data accumulates, accurately depicting the target tree canopy's outline at the current moment.

[0077] Simultaneously, a multispectral camera captures the target tree canopy in real time, acquiring real-time multispectral image data. The camera automatically adjusts its parameters to adapt to varying lighting conditions, capturing images spanning both visible and near-infrared wavelengths. For example, the reflectance data for the red band is 0.35, the green band is 0.4, the blue band is 0.3, and the near-infrared band is 0.65. These reflectance data across these different bands constitute the real-time multispectral image data.

[0078] In addition, a network of ambient light sensors collects ambient light data in real time. These sensors monitor light intensity, direction, and light scattering. For example, at a given moment, the monitored light intensity is 900 lux, the light is coming from a 45-degree angle from the southwest, and the light scattering coefficient is 0.5. This data forms the real-time ambient light field data.

[0079] Step S152: performing noise filtering and dynamic calibration processing on the real-time lidar point cloud data to generate a calibrated point cloud set, performing radiation correction and atmospheric correction processing on the real-time multispectral image data to generate standardized spectral data, and performing light source intensity normalization processing on the real-time ambient light field data to generate standardized light field distribution data.

[0080] Real-time LiDAR point cloud data is first filtered for noise. Since LiDAR may be subject to external interference during data collection, some noise points are generated. These noise points are identified and removed using a specific filtering algorithm. For example, using a Gaussian filter, each point in the point cloud data is determined to be a noise point based on the distribution and distance relationship of its surrounding points. If a point is significantly out of alignment with most of its surrounding points, it is considered a noise point and removed.

[0081] Next, dynamic calibration is performed. Because LiDAR devices may exhibit certain deviations under different operating conditions, dynamic calibration calibrates the collected point cloud data based on the device's current state and environmental factors. For example, based on information such as the LiDAR's temperature and humidity sensor data and the device's operating time, the coordinate values ​​of the point cloud data are adjusted to more accurately reflect the actual location of the target tree canopy. After noise filtering and dynamic calibration, a calibrated point cloud set is generated.

[0082] Radiometric correction is performed on real-time multispectral image data. Due to factors such as camera sensor characteristics and ambient lighting, image data may exhibit uneven radiometric properties. Using a radiometric correction algorithm, the radiometric value of each pixel in the image is adjusted. For example, based on the camera's calibration parameters and known data from a standard radiometric source, the appropriate adjustment ratio for each pixel is calculated. This adjustment is then applied to all pixels in the image, resulting in a more uniform radiometric image.

[0083] Next, atmospheric correction is performed. Gases and particulate matter in the atmosphere can affect light propagation, causing deviations in the image data. An atmospheric correction algorithm, taking into account factors such as atmospheric scattering and absorption, corrects the image data. For example, using a pre-established atmospheric model and certain image features, the degree of atmospheric influence on light is calculated. Correction is then performed on each pixel in the image to produce standardized spectral data.

[0084] For real-time ambient light field data, light source intensity normalization is performed. Different light sources may have different intensities at different times and in different environmental conditions. To facilitate subsequent processing and comparison, light source intensity normalization is required. For example, if the current light source intensity is 1000 lux and the pre-set normalized standard intensity is 800 lux, then by calculating the proportional relationship, all data related to light source intensity are adjusted to ensure consistency in light source intensity across the entire ambient light field data, generating standardized light field distribution data.

[0085] Step S153: input the calibrated point cloud set, the standardized spectral data and the standardized light field distribution data into the three-dimensional reconstruction model, and output the dynamic three-dimensional model of the target tree crown, the branch and leaf density distribution parameters, and the light shading area marking parameters.

[0086] The processed calibrated point cloud, normalized spectral data, and normalized light field distribution data are fed into the previously trained 3D reconstruction model. Based on this input data, the 3D reconstruction model utilizes its internal network structure and learned parameters to perform a 3D reconstruction of the target tree canopy.

[0087] The model first analyzes the calibrated point cloud collection, combining information from the standardized spectral data and the standardized light field distribution data to determine the outer structure of the target tree canopy. For example, it constructs a 3D mesh model of the target tree canopy based on the distribution of points in the point cloud data, the canopy surface characteristics reflected in the spectral data, and the light propagation in the light field data. This 3D mesh model is dynamic and reflects the actual shape of the target tree canopy at the current moment.

[0088] At the same time, the model also calculates and outputs the branch and leaf density distribution parameters. By analyzing the density of points at different locations in the calibrated point cloud set and combining the information related to branch and leaf growth in the spectral data, the branch and leaf density in different areas of the crown is determined. For example, in a certain area of ​​the crown, the point cloud data shows that the distribution of points is relatively dense, and the spectral data shows that the vegetation index in this area is relatively high, indicating that the branches and leaves in this area are more lush, thereby determining that the branch and leaf density in this area is 80 branch and leaf units per cubic meter (the branch and leaf unit here is a hypothetical unit of measurement). By analyzing the entire crown, the branch and leaf density distribution parameters of different areas are obtained.

[0089] In addition, the model outputs parameters for marking light-blocked areas. Based on the light propagation and occlusion information in the standardized light field distribution data and the structure of the tree canopy in the 3D mesh model, the model identifies areas where light is blocked. For example, by analyzing the direction of light and the geometry of the tree canopy, it is found that certain areas within the canopy have significantly lower light intensity than other areas due to occlusion by the outer branches and leaves. These areas are marked as light-blocked areas, and parameters such as their location and range are recorded as light-blocked area marking parameters.

[0090] Step S154: generating the structural parameter optimization suggestion according to the branch and leaf density distribution parameter and the light shading area marking parameter.

[0091] Step S1541: extracting the branch and leaf density distribution parameters of the dynamic three-dimensional model, comparing them with a preset crown health density threshold, and generating a set of spatial coordinates of the insufficient density area.

[0092] The branch and leaf density distribution parameters are extracted from the dynamic three-dimensional model, and the preset crown health density threshold is determined based on a large amount of tea oil tree research and practical experience. For example, the preset crown health density threshold is 60 branch and leaf units per cubic meter. The extracted branch and leaf density distribution parameters are compared with the threshold. In the dynamic three-dimensional model, it is found that the branch and leaf density in a certain area is 40 branch and leaf units per cubic meter, which is lower than the health density threshold. By traversing all areas of the dynamic three-dimensional model, all areas with branch and leaf density lower than the health density threshold are found, and the spatial coordinates of the above areas are recorded. For example, the spatial coordinates of the above-mentioned insufficient density areas may be (1.0, 1.5, 2.0)-(1.2, 1.8, 2.2), (2.5, 3.0, 2.8)-(2.7, 3.2, 3.0), etc., forming a set of spatial coordinates of insufficient density areas.

[0093] Step S1542: extracting the light-blocking area marking parameters, and calculating the influence coefficient of the insufficient light area in combination with a preset photosynthesis efficiency model.

[0094] The light-blocking area marking parameters are extracted from the dynamic three-dimensional model. The light-blocking area marking parameters include information such as the location and range of the light-blocking area. Combined with the preset photosynthesis efficiency model, the model takes into account the influence of factors such as light intensity, light duration, and photosynthetic characteristics of the crown and leaves on photosynthesis efficiency. For example, in a certain light-blocking area, according to the light-blocking area marking parameters, it can be known that the light intensity in the area is 300 lux, and according to the photosynthesis efficiency model, at this light intensity, the photosynthesis efficiency is reduced by 40% compared with the photosynthesis efficiency under normal light intensity (assuming 800 lux). By performing such calculations for all light-blocking areas, the influence coefficient of each light-deficient area is obtained, which reflects the degree of influence of insufficient light on physiological processes such as crown photosynthesis.

[0095] Step S1543: generating a suggested pruning path according to the spatial coordinate set of the insufficient density area and the influence coefficient of the insufficient illumination area, wherein the suggested pruning path includes the coordinates of the area to be thinned, the coordinates of the area to be light-filled, and an operation priority score.

[0096] Recommended pruning paths are generated based on the spatial coordinates of under-density areas and the influence coefficients of under-lighting areas. For under-density areas, thinning is considered to improve canopy ventilation and lighting conditions, promoting branch and leaf growth. For example, if there are some overly dense branches and leaves near the under-density area (1.0, 1.5, 2.0)-(1.2, 1.8, 2.2), the coordinates of the area containing these branches and leaves are marked as the coordinates of the area to be thinned.

[0097] For areas with insufficient lighting, the influence coefficient determines whether fill light is needed, as well as the intensity and location of the fill light. For example, for an insufficiently lit area with a large influence coefficient, fill light equipment is determined to be deployed near the area, and the coordinates of the area are marked as the coordinates of the area to be filled with light.

[0098] For each area to be thinned and area to be supplemented with light, an operation priority score is assigned based on the degree of its impact on the overall health and growth of the crown. For example, areas to be thinned where the branches and leaves are too dense and seriously affect the surrounding light and ventilation, while also causing insufficient density of nearby branches and leaves, and areas to be supplemented with light where the insufficient light has a large impact coefficient and affects the key growth parts of the crown, are given a higher operation priority score; while areas with less overall impact are given a lower score. In this way, a pruning recommendation path is generated that includes the coordinates of the area to be thinned, the coordinates of the area to be supplemented with light, and the operation priority score. For example, the coordinates of the area to be thinned are (1.3, 1.6, 2.1)-(1.5, 1.8, 2.3), and the operation priority score is 8 points; the coordinates of the area to be supplemented with light are (2.6, 3.1, 2.9)-(2.8, 3.3, 3.1), and the operation priority score is 7 points, etc.

[0099] Step S1544: superimposing the suggested trimming path onto the visual interface of the dynamic three-dimensional model, and generating an operation instruction sequence and sending it to the automated trimming device.

[0100] The generated suggested pruning paths are overlaid on the dynamic 3D model's visualization interface. Within this visualization, different colors and symbols are used to distinguish areas requiring pruning and areas requiring supplemental lighting. For example, areas requiring pruning are marked with a red border and their priority score is displayed; areas requiring supplemental lighting are marked with a blue border and their priority score is also displayed. This allows operators to intuitively identify the areas requiring pruning and supplemental lighting, and their importance.

[0101] At the same time, an operation instruction sequence is generated based on the pruning suggestion path and sent to the automated pruning equipment. The operation instruction sequence specifies the operation steps and parameters of the automated pruning equipment in detail.

[0102] For example, step S1544 includes:

[0103] S1544-1: Convert the coordinates of the area to be pruned into a motion trajectory of the robotic arm of the automated pruning device, and sort the execution order of the robotic arm motion trajectory according to the operation priority score.

[0104] First, the coordinates of the area to be pruned are converted into motion trajectories that can be understood and executed by the robotic arm of the automated pruning equipment. Automated pruning equipment usually has its own specific coordinate system and motion control method, and the coordinates of the area to be pruned in the dynamic three-dimensional model need to be converted into coordinates in the device coordinate system. For example, suppose the coordinates (x, y, z) of the area to be pruned in the dynamic three-dimensional model are based on a coordinate system with the center of the bottom of the crown as the origin, while the coordinate system of the automated pruning equipment is based on a fixed point on the base of the equipment as the origin. Through a series of coordinate transformation calculations (coordinate transformation operations such as translation and rotation are involved here, but the formula editor is not used for detailed explanation, and the calculation logic is described in text), (x, y, z) is converted into coordinates (x', y', z') in the device coordinate system.

[0105] Based on the converted coordinates, the robot's trajectory is planned. For example, if the area to be pruned is an irregular space, the robot needs to reach key locations within the area in a specific order to perform pruning operations. The trajectory may include linear and curved motions to ensure the robot accurately reaches the area to be pruned and effectively prunes branches.

[0106] The robot's motion trajectories are then sorted according to their priority scores. Areas with higher priority scores are prioritized and executed first. For example, if there are three areas to be pruned, Area A has an 8 priority score, Area B has a 6 priority score, and Area C has a 7 priority score. The robot's motion trajectories will be executed first: Area A, then Area C, and finally Area B. This ensures that areas with the greatest impact on the tree crown are prioritized, improving pruning efficiency and effectiveness.

[0107] S1544-2: Based on the sorted motion trajectory of the robotic arm, control the robotic arm of the automated pruning equipment to move to the coordinates of the area to be pruned in sequence, and activate the cutting device to perform the pruning operation.

[0108] Based on the sorted robotic arm motion trajectories, the automated pruning equipment's control system begins operating. The control system precisely controls the robotic arm's movements according to the planned trajectory. For example, for the robotic arm's trajectory for the first area to be pruned, the control system sends instructions to move the robotic arm from its current position along a predetermined path toward the first target coordinate point in the area to be pruned. During this movement, sensors monitor the robotic arm's position and posture in real time to ensure accurate movement along the planned trajectory.

[0109] When the robotic arm reaches the target coordinates of the area to be pruned, the control system activates the cutting mechanism. The cutting mechanism adjusts cutting parameters, such as cutting speed and force, based on the size of the branches and foliage in the area. For example, if the branches and foliage in the area are thicker, the cutting mechanism increases the cutting force; if the branches and foliage are thinner, the cutting speed is adjusted appropriately to ensure efficient and accurate pruning. The cutting mechanism then severs the target branches and foliage, collecting the cut branches and foliage through the machine's collection system to minimize impact on the surrounding environment.

[0110] The robot then moves to other coordinate points in the area to be pruned, following the trajectory, and repeats the above steps until all branches in that area are pruned. Then, following the sorted trajectory, the robot moves to the next area to be pruned, continuing the pruning operation until all branches are pruned.

[0111] S1544-3: Deploy a lighting compensation device at the coordinates of the area to be filled with light, and adjust the fill light angle and radiation intensity of the lighting compensation device according to the influence coefficient.

[0112] Deploy lighting compensation equipment at the determined coordinates of the area to be filled with light. The lighting compensation equipment can be various types of lamps, such as LED lamps. During the deployment process, ensure that the installation position of the lamps is accurate and can effectively provide light to the area to be filled with light.

[0113] The lighting compensation device's fill light angle and radiation intensity are adjusted based on the impact coefficient of the insufficiently illuminated area. Areas with larger impact coefficients indicate that insufficient light in those areas is having a more severe impact on the tree canopy and requires stronger light compensation. For example, for an area with an impact coefficient of 0.6, the pre-set adjustment rules would set the lighting compensation device's radiation intensity to a higher value, such as 1000 lux, and adjust the fill light angle to maximize coverage of the area to ensure adequate lighting.

[0114] For areas with a smaller impact coefficient, the radiation intensity is reduced and the fill light angle is adjusted accordingly. For example, for an area with an impact coefficient of 0.3, the radiation intensity might be set to 600 lux, and the fill light angle adjusted to meet the area's basic lighting needs. By making fine adjustments based on the impact coefficient, we can more accurately provide appropriate lighting conditions for different areas of the canopy, promoting healthy growth.

[0115] S1544-4: Collect the actual execution path of the robot arm's motion trajectory and the fill light coverage area of ​​the illumination compensation device in real time, and calculate the coordinate deviation from the recommended pruning path.

[0116] During the pruning process performed by the automated pruning equipment and the supplemental lighting provided by the illumination compensation device, relevant data is collected in real time. Regarding the robotic arm's motion trajectory, sensors and positioning devices installed on the robotic arm capture the actual position of the robotic arm at each moment in real time, thereby determining the actual execution path of the robotic arm's motion trajectory. For example, during the motion of the robotic arm, the position coordinates are recorded at regular intervals (e.g., 0.1 seconds). These coordinate points are connected to form the actual execution path.

[0117] For light compensation equipment, specialized light monitoring equipment collects real-time information about the fill light coverage area. For example, an array of light sensors, distributed around the tree canopy, detects changes in light intensity at each location. When the light compensation equipment is turned on, the range and boundaries of the fill light coverage area are determined based on changes in light intensity, providing actual information about the fill light coverage area.

[0118] The actual execution path of the collected robot arm motion trajectory and the fill light coverage area information of the illumination compensation device are compared with the coordinate information in the pruning suggestion path. Calculate the deviation between the actual execution path and the coordinates of the corresponding area to be pruned in the pruning suggestion path. For example, a target coordinate of the area to be pruned in the pruning suggestion path is (1.5, 1.8, 2.2), while the coordinates of the robot arm reaching the vicinity of this position in the actual execution path are (1.55, 1.85, 2.25). The coordinate deviation is obtained by calculating the coordinate difference (such as calculating the difference in the x, y, and z directions respectively). For the fill light coverage area, the deviation from the expected fill light area coordinates in the pruning suggestion path is also calculated, which reflects the degree of difference between the actual operation and the expectation.

[0119] S1544-5: When the coordinate deviation exceeds the fault tolerance threshold, the corrected robot arm motion trajectory and fill light parameters are re-planned according to the current actual execution path and the fill light coverage area.

[0120] A tolerance threshold is preset to determine whether the difference between the actual execution and the expected path is within an acceptable range. If the calculated coordinate deviation exceeds the tolerance threshold, it indicates that the actual execution has significantly deviated from the recommended pruning path and replanning is required.

[0121] The robot's trajectory is analyzed based on the current path being executed. For example, if the robot's trajectory in a specific area to be pruned deviates from the expected path and the deviation exceeds the tolerance threshold, the cause of the deviation is analyzed, potentially due to factors such as device positioning errors and external interference. The robot's trajectory is then replanned based on the remaining unpruned portion of the area and the overall pruning goal. The new trajectory takes into account the executed portion and the current situation, ensuring accurate completion of the remaining pruning task.

[0122] For light compensation equipment, adjust the fill light parameters based on the actual fill light coverage area and the lighting requirements of the recommended pruning path. For example, if the fill light coverage area does not meet the expected range and the deviation exceeds the tolerance threshold, the installation angle of the light compensation equipment or the radiation intensity may need to be adjusted to ensure that the fill light meets the needs of the canopy. By replanning the corrected robot arm motion trajectory and fill light parameters, automated pruning and fill light operations are more accurate and efficient.

[0123] S1544-6: Feedback the corrected motion trajectory of the robotic arm and the fill light parameters to the dynamic three-dimensional model, and update the branch and leaf density distribution parameters and the light occlusion area marking parameters.

[0124] The replanned and corrected robot arm motion trajectory and fill light parameters are fed back to the dynamic 3D model. The dynamic 3D model updates the branch and leaf density distribution parameters and the light occlusion area marking parameters based on the feedback information.

[0125] Regarding the branch density distribution parameter, as the robotic arm performs pruning operations, the canopy's branch structure changes. The dynamic 3D model recalculates and updates the branch density in different areas based on the new pruning conditions. For example, if the branch density in a specific area decreases after pruning, the model will adjust the branch density parameter accordingly.

[0126] As the lighting compensation device's fill-light parameters change, the distribution of light within the tree canopy will also change. The dynamic 3D model reanalyzes and re-determines the light-blocked area based on the new fill-light conditions. For example, if the fill-light intensity increases or the fill-light angle is adjusted, and a portion of the previously light-blocked area now receives sufficient light, the model will update the light-blocked area marking parameters, removing that portion from the light-blocked area or adjusting its boundaries. Through such updates, the dynamic 3D model can reflect the latest status of the tree canopy after pruning and fill-lighting operations in real time, providing an accurate information basis for subsequent monitoring and management.

[0127] Figure 2 This diagram illustrates exemplary hardware and software components of a system 100 for 3D tree crown reconstruction based on multi-scale feature fusion, which can implement the concepts of this application, as provided in some embodiments of the present application. For example, processor 120 can be used in system 100 for 3D tree crown reconstruction based on multi-scale feature fusion and perform the functions described in this application.

[0128] The system 100 for 3D tree crown reconstruction based on multi-scale feature fusion can be a general-purpose server or a special-purpose server, both of which can be used to implement the 3D tree crown reconstruction method based on multi-scale feature fusion described herein. Although only one server is shown herein, for convenience, the functions described herein can be implemented in a distributed manner across multiple similar platforms to balance the processing load.

[0129] For example, the tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and storage media 140 in different forms, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion may also include program instructions stored in ROM, RAM, or other types of non-temporary storage media, or any combination thereof. The method of the present application can be implemented according to the above-mentioned program instructions. The tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion also includes an input / output (I / O) interface 150 between the computer and other input and output devices.

[0130] For ease of explanation, only one processor is described in the tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion. However, it should be noted that the tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion in the present application may also include multiple processors, so the steps performed by one processor described in the present application may also be performed jointly or individually by multiple processors. For example, if the processor of the tree crown three-dimensional reconstruction system 100 based on multi-scale feature fusion executes step A and step B, it should be understood that step A and step B may also be executed jointly by two different processors or individually in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor execute steps A and B together.

[0131] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When a processor executes the computer-executable instructions, the above-mentioned tree crown three-dimensional reconstruction method based on multi-scale feature fusion is implemented.

[0132] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, multiple features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A tree crown 3D reconstruction method based on multi-scale feature fusion, characterized in that: The method comprises: Collecting a tree crown data set, wherein the tree crown data set includes laser radar scanning point cloud data, multispectral image data, and ambient light field distribution data; Performing multi-scale feature extraction processing on the tree crown collection data set to obtain local structural features of the tree crown point cloud set, spectral reflectance features of the multispectral image data, and light attenuation features of the ambient light field distribution data; The local structural features, the spectral reflectance features and the light attenuation features are dynamically weighted and fused through a feature fusion network to generate a multi-scale fusion feature of the crown; Training a three-dimensional reconstruction model based on the multi-scale fusion features of the tree crown to generate a three-dimensional grid model of the tree crown; The three-dimensional reconstruction model is called to perform three-dimensional reconstruction processing on the real-time collected data of the target tree crown, and a dynamic three-dimensional model of the target tree crown and structural parameter optimization suggestions are output.

2. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 1, characterized in that: The multi-scale feature extraction processing is performed on the tree crown collection data set to obtain the local structural features of the tree crown point cloud set, the spectral reflectance features of the multispectral image data, and the light attenuation features of the ambient light field distribution data, including: Performing density adaptive clustering processing on the laser radar scanning point cloud data to segment it into multiple point cloud clusters, and extracting the geometric center coordinates, normal vector distribution and curvature change gradient of each point cloud cluster to generate local structural features of the crown point cloud set; Performing band separation processing on the multispectral image data, extracting the reflectance difference between the visible light band and the near-infrared band, and generating spectral reflectance characteristics of the multispectral image data in combination with a vegetation index calculation model; Performing ray tracing simulation processing on the ambient light field distribution data, calculating the light penetration depth based on the spatial occlusion relationship between the light source position and the tree crown point cloud, and generating the light attenuation characteristics of the ambient light field distribution data; The local structural features, the spectral reflectance features, and the light attenuation features are aligned according to spatial coordinates and then stored in a fusion feature cache queue.

3. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 2, characterized in that: The method of dynamically weighting and fusing the local structural features, the spectral reflectance features, and the light attenuation features through a feature fusion network to generate a multi-scale fusion feature of the crown includes: Initialize the local structure feature weight coefficient, the spectral reflectance feature weight coefficient, and the light attenuation feature weight coefficient, wherein the local structure feature weight coefficient is positively correlated with the point cloud density of the crown point cloud set, the spectral reflectance feature weight coefficient is negatively correlated with the vegetation index of the multispectral image data, and the light attenuation feature weight coefficient is exponentially related to the light penetration depth of the ambient light field distribution data; The following operations are performed on each spatial coordinate unit in the fusion feature cache queue: Dynamically adjusting the weight coefficient of the local structural feature according to the curvature change gradient of the local structural feature; Dynamically adjust the spectral reflection feature weight coefficient according to the band reflectivity difference of the spectral reflection feature; Dynamically adjusting the light attenuation feature weight coefficient according to the light penetration depth of the light attenuation feature; Based on the adjusted local structure feature weight coefficient, the spectral reflection feature weight coefficient, and the light attenuation feature weight coefficient, the local structure feature, the spectral reflection feature, and the light attenuation feature are weightedly fused through a feature fusion network to generate a fused feature vector of the current spatial coordinate unit; The fused feature vectors of all spatial coordinate units are connected according to a topological relationship to generate the crown multi-scale fusion feature.

4. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 3, characterized in that: The dynamically adjusting the weight coefficient of the local structural feature according to the curvature change gradient of the local structural feature includes: Calculating a ratio of a curvature change gradient of a current spatial coordinate unit to a global average curvature, and if the ratio is greater than a first threshold, increasing the local structural feature weight coefficient by a first gain amount; If the ratio is less than a second threshold, reducing the local structural feature weight coefficient by a first attenuation amount; The first gain and the first attenuation are dynamically adjusted according to the point cloud density distribution of the crown point cloud set, and satisfy a balance constraint between the total gain and the total attenuation.

5. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 3, characterized in that: The dynamically adjusting the spectral reflection feature weight coefficient according to the band reflectivity difference of the spectral reflection feature includes: Calculating a deviation between a vegetation index of a current spatial coordinate unit and a healthy vegetation index reference value, and if the deviation exceeds a third threshold, increasing the spectral reflectance feature weight coefficient by a second gain; If the vegetation index is lower than the withering warning threshold, reducing the spectral reflectance feature weight coefficient by a second attenuation amount; The second gain and the second attenuation are dynamically modified according to the time series data of the multispectral image data.

6. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 3, characterized in that: The dynamically adjusting the light attenuation feature weight coefficient according to the light penetration depth of the light attenuation feature includes: Calculating a proportion of effective illumination coverage area according to the light penetration depth of the current spatial coordinate unit, and if the proportion is lower than a fourth threshold, increasing the illumination attenuation feature weight coefficient by a third gain amount; If the light penetration depth reaches a saturation threshold, reducing the light attenuation feature weight coefficient by a third attenuation amount; The third gain and the third attenuation are periodically adjusted according to seasonal variation data of the ambient light field distribution data.

7. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 1, characterized in that: The training of the three-dimensional reconstruction model based on the multi-scale fusion features of the tree crown to generate a three-dimensional grid model of the tree crown includes: Constructing a 3D reconstruction network architecture comprising an encoder and a decoder, wherein the encoder is composed of a multi-scale convolution module and a feature pyramid module, and the decoder is composed of a voxel generation module and a surface refinement module; Inputting the crown multi-scale fusion feature into the encoder, extracting three-dimensional spatial features of different levels through the multi-scale convolution module, and performing cross-level feature fusion through the feature pyramid module to generate fused three-dimensional spatial features; Inputting the fused three-dimensional spatial features into the decoder, generating an initial voxel model through the voxel generation module, and smoothing the voxel edges through the surface refinement module; Calculating the geometric difference loss between the initial voxel model and the true tree crown 3D model, and combining it with the surface curvature consistency loss to train the parameters of the 3D reconstruction network architecture; When the geometric difference loss is lower than a preset threshold, the current parameters are saved as the three-dimensional reconstructed model.

8. The method for 3D tree crown reconstruction based on multi-scale feature fusion according to claim 1, characterized in that: The calling of the three-dimensional reconstruction model to perform three-dimensional reconstruction processing on the real-time collected data of the target tree crown, and outputting a dynamic three-dimensional model of the target tree crown and structural parameter optimization suggestions, includes: Collect real-time lidar point cloud data, real-time multispectral image data, and real-time ambient light field data of the target tree crown; Performing noise filtering and dynamic calibration on the real-time lidar point cloud data to generate a calibrated point cloud set, performing radiation correction and atmospheric correction on the real-time multispectral image data to generate standardized spectral data, and performing light source intensity normalization on the real-time ambient light field data to generate standardized light field distribution data; The calibrated point cloud set, the standardized spectral data and the standardized light field distribution data are input into the three-dimensional reconstruction model, and the dynamic three-dimensional model of the target tree crown, the branch and leaf density distribution parameters, and the light shading area marking parameters are output.

Images