Spectral mapping method and device based on multi-source point cloud cooperative registration and storage medium

Abstract

The application discloses a spectrum mapping method and device based on multi-source point cloud cooperative registration and a storage medium, and comprises the following steps: obtaining a coarse registration transformation matrix based on deep geometric semantic feature matching; generating a canopy height model by spatially aligning ground-based laser radar point clouds of a target forest area and a UAV digital surface model based on the coarse registration transformation matrix; performing secondary geometric feature matching on the canopy height model and the UAV digital surface model, identifying canopy high-frequency geometric feature point pairs, and solving a fine registration transformation matrix; performing visibility analysis on the point clouds obtained after registration by the fine registration transformation matrix, and dividing the point clouds into a visible set and an occluded set; and mapping digital orthographic image spectrum information of the target forest area to the visible set, shielding the spectrum mapping result of the target forest area after the spectrum mapping of the occluded set. The application breaks through the cross-modal matching bottleneck by deep feature dimension reduction matching and coarse-to-fine alignment, and realizes high-precision and full-automatic spatial registration of multi-source data in a complex forest area.

Description

Technical Field

[0001] This application relates to the field of forestry information technology, and in particular to a spectral mapping method, device and storage medium based on multi-source point cloud collaborative registration. Background Technology

[0002] In the context of current global climate change, the precise quantification of the carbon sequestration capacity of forest ecosystems, as the largest carbon sink among terrestrial ecosystems, is one of the core scientific issues in global climate change research and the achievement of the "dual carbon" strategic goals. Traditional forest survey methods rely on manual plot measurements, which have inherent drawbacks such as high cost, low efficiency, and difficulty in large-scale repeated monitoring. With the development of "precision forestry," the fusion of spectral texture information from multispectral imagery and three-dimensional geometric information from LiDAR point clouds has become a mainstream trend. These two types of data are naturally complementary in terms of information dimensions and are of great significance for fine forest structure modeling, tree species identification, and carbon storage inversion. However, due to the essential differences in imaging mechanisms, spatial dimensions, and perspective conditions, significant heterogeneity exists among multi-source data.

[0003] Specifically, multispectral imagery consists of a two-dimensional pixel array, while lidar point clouds are discrete three-dimensional point sets. High-precision spatial alignment between the two is a prerequisite obstacle for deep data fusion. Conventional iterative nearest-point algorithms attempt to align motion-reconstructed structure (MRS) point clouds with lidar point clouds. However, MRS point clouds exhibit high noise and distortion in weakly textured regions within the tree canopy, and their density distribution is severely mismatched with that of lidar point clouds, easily getting trapped in local optima and failing to achieve accurate global alignment. Simultaneously, forest trees exhibit non-rigid deformation characteristics due to wind disturbance and seasonal phenological influences. The high self-similarity of forest textures further leads to numerous mismatches in conventional feature descriptors, severely reducing the robustness and accuracy of heterogeneous data registration. These factors collectively make it difficult to establish a strict correspondence between multi-source data under a unified spatial benchmark, directly affecting the accuracy of subsequent quantitative remote sensing analysis and forest parameter inversion.

[0004] On the other hand, in the multi-source data fusion stage after spatial alignment, traditional techniques generally employ the conventional vertical projection method for spectral texture mapping. This method implicitly assumes that the ground surface is continuous and opaque, which cannot effectively handle the complex geometric occlusion relationships in the three-dimensional space of a forest. In vertical projection mode, spectral pixels of the upper canopy are often forcibly or incorrectly mapped onto occluded branches or ground point clouds in the understory, resulting in severe "spectral smearing" and artifacts. The lack of a refined three-dimensional visibility discrimination mechanism leads to severe distortion of the spectral information of the fused point cloud within the canopy and understory structures, making it difficult to meet the engineering requirements for detailed modeling of individual trees and high-fidelity texture analysis.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a spectral mapping method, device and storage medium based on multi-source point cloud collaborative registration, which aims to solve the technical problems of low registration accuracy caused by dimensional differences and non-rigid deformation of existing multi-source heterogeneous forest data, as well as spectral smearing and texture distortion caused by ignoring the three-dimensional spatial occlusion relationship in traditional mapping methods.

[0007] To achieve the above objectives, this application proposes a spectral mapping method based on multi-source point cloud collaborative registration, the method comprising: A deep graph neural network is used to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and match them to obtain a coarse registration transformation matrix. Based on the coarse registration transformation matrix, the ground-based lidar point cloud of the target forest area is spatially aligned with the space where the UAV digital surface model is located to generate a canopy height model; The deep graph neural network is used to perform secondary geometric feature matching between the canopy height model and the UAV digital surface model. High-frequency geometric feature point pairs of the canopy are identified in the matching results, and the fine registration transformation matrix is ​​calculated based on the identification results. In the unified space of the fine registration transformation matrix, the HPR algorithm is used to perform visibility analysis on the point cloud, and the point cloud is divided into a visible set and an occlusion set according to the analysis results. The spectral information of the digital orthophoto of the target forest area is mapped onto the visible set, and the spectral mapping of the occlusion set is masked to obtain the spectral mapping result of the target forest area.

[0008] In one embodiment, before the step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain a coarse registration transformation matrix, the method further includes: Acquire multi-temporal observation data of the target forest area, wherein the multi-temporal observation data is a dataset corresponding to the phenological characteristics of trees in the target forest area and containing canopy spectral information and texture information; Radiometric calibration and motion recovery structure calculations are performed on the canopy spectral and texture information to generate digital orthophotos, UAV digital surface models, and initial point clouds. A ground-based lidar point cloud is used to acquire three-dimensional structural information of the forest understory. The ground-based lidar point cloud is then subjected to a cloth simulation filtering algorithm to decouple the terrain and vegetation features, generating a digital terrain model.

[0009] In one embodiment, the step of using a cloth simulation filtering algorithm to decouple terrain and vegetation features from the ground-based lidar point cloud to generate a digital terrain model includes: The elevation values ​​of the ground-based lidar point cloud are reversed, and the reversed ground-based lidar point cloud is used as the contact surface for cloth simulation. A virtual cloth grid is initialized above the inverted ground-based lidar point cloud, and the virtual cloth grid is controlled to sink downward under the action of gravity to contact the surface of the ground-based lidar point cloud. During the settling process of the virtual cloth grid, the equilibrium position of the grid node is determined based on the collision constraint between the grid node and the reversed surface of the ground-based lidar point cloud, and the distance between the equilibrium grid node and the corresponding point cloud in the corresponding projection neighborhood is calculated. The category of the point cloud is determined according to the preset distance threshold, and the category includes terrain ground points or vegetation non-ground points. The point cloud that is identified as terrain ground points is collected, and vegetation non-ground points are removed to form a discrete set of terrain points; The discrete set of terrain points is subjected to rasterization interpolation to generate the digital terrain model.

[0010] In one embodiment, the step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain a coarse registration transformation matrix includes: The LightGlue depth map neural network is used to perform global semantic analysis on the terrain undulation, gully direction and slope change in the grayscale depth map. The analysis results are used to extract terrain feature point pairs, calculate global translation and rotation parameters based on the terrain feature point pairs, and obtain the coarse registration transformation matrix based on the calculation results.

[0011] In one embodiment, the step of spatially aligning the ground-based lidar point cloud of the target forest area with the space of the UAV digital surface model based on the coarse registration transformation matrix to generate a canopy height model includes: The ground-based lidar point cloud is transformed to the same coordinate system as the UAV digital surface model using a coarse registration transformation matrix. After removing the terrain elevation represented by the digital terrain model from the point cloud in the coordinate system, the vegetation height of each point cloud relative to the ground surface is obtained, forming a normalized point cloud. The normalized point cloud is projected onto a horizontal plane and rasterized, and the maximum vegetation height within each raster cell is extracted to generate the canopy height model.

[0012] In one embodiment, the HPR algorithm is used to perform visibility analysis on the point cloud within the unified space of the fine registration transformation matrix. The step of dividing the point cloud into a visible set and an occlusion set based on the analysis results includes: The point cloud is inverted by HPR transformation to obtain a flipped point set, and the convex hull of the flipped point set is determined. The point cloud located at the vertices of the convex hull is marked as the visible set, and the point cloud located inside the convex hull is marked as the occlusion set. The visible set represents the canopy surface and forest window ground, and the occlusion set represents the branches inside the canopy and the occluded ground.

[0013] In one embodiment, the step of mapping the spectral information of the digital orthophoto of the target forest area to the visible set and masking the spectral mapping of the occlusion set to obtain the spectral mapping result of the target forest area includes: Expand the multispectral and true color attribute fields into the original 3D point cloud data structure; The multispectral reflectance and true-color pixel values ​​in the digital orthophoto are extracted using spatial indexing technology, and the true-color pixel values ​​are written point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance. For the point cloud of the occlusion set, the original attribute structure and original laser intensity field of the point cloud of the occlusion set are kept unchanged, and the mapping result and the masking result are combined into the spectral mapping result.

[0014] In one embodiment, the step of extracting multispectral reflectance and true-color pixel values ​​from the digital orthophoto using spatial indexing technology, and writing the true-color pixel values ​​point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance, includes: Traverse the visible point cloud, detect the original laser echo intensity value of each point, and mark the points whose intensity value is greater than the preset intensity saturation threshold as overexposed points; The true color data of the corresponding pixel positions of each overexposed point in the digital orthophoto are extracted using spatial indexing. The extracted true-color data is written into the attribute field of the overexposed point to repair the overexposure loss, and color consistency smoothing is performed on the overexposed point and its spatial neighborhood point cloud.

[0015] Furthermore, to achieve the above objectives, this application also proposes a spectral mapping device based on multi-source point cloud collaborative registration. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the spectral mapping method based on multi-source point cloud collaborative registration as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the spectral mapping method based on multi-source point cloud collaborative registration as described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: The technical solution of this application employs a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area, and then matches them to obtain a coarse registration transformation matrix. Based on the coarse registration transformation matrix, the ground-based lidar point cloud of the target forest area is spatially aligned with the space of the UAV digital surface model, and a canopy height model is generated through projection rasterization. The deep graph neural network is then used to perform secondary geometric feature matching on the canopy height model and the UAV digital surface model, identifying high-frequency geometric feature point pairs of the canopy in the matching results, and calculating the fine registration transformation matrix based on the identification results. In the unified space of the fine registration transformation matrix, the HPR transformation algorithm is used to perform visibility analysis on the point cloud, and the point cloud is divided into a visible set and an occluded set according to the analysis results. The attribute fields are extended in the point cloud data structure, and the spectral information of the digital orthophoto of the target forest area is written into the extended fields of the visible set, while the writing operation to the occluded set is blocked to retain the original laser intensity information, thus obtaining the spectral mapping result of the target forest area.

[0018] Specifically, the process begins by acquiring a digital terrain model and a UAV digital surface model of the target forest area. These are then rasterized into grayscale depth maps. A depth map neural network is used to extract deep geometric semantic features such as terrain undulations and gully orientations for matching, and a coarse registration transformation matrix is ​​calculated to unify the heterogeneous data to the same geospatial reference. Next, based on the coarse registration transformation matrix, the LiDAR point cloud and the UAV digital surface model are aligned in space to generate a canopy height model. The same depth map neural network is then used to perform secondary geometric feature matching between the canopy height model and the UAV digital surface model, identifying high-frequency geometric feature pairs such as treetop high points and canopy edges. A fine registration transformation matrix is ​​then calculated to correct residual biases. Finally, in the unified space after fine registration, the HPR transform algorithm is used to perform visibility analysis on the point cloud. Convex hull calculation divides the point cloud into a visible set and an occluded set. Based on attribute expansion and differentiated writing strategies, the spectral information of the digital orthophoto is assigned to the visible set, while pixel writing to the occluded set is blocked to preserve the original LiDAR intensity information. Through the aforementioned coarse-to-fine spatial alignment mechanism and differentiated spectral mapping strategy at the underlying data structure level, this application effectively overcomes the core bottleneck of low registration accuracy of multi-source heterogeneous data in forest scenarios, and eliminates artifacts and spectral errors caused by traditional vertical projection from a mechanistic perspective, providing a reliable data fusion technology solution for fine forest modeling, carbon sequestration measurement and ecological monitoring. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the spectral mapping method based on multi-source point cloud collaborative registration in this application; Figure 2 This is a detailed process diagram based on step S10 in the first embodiment; Figure 3 This is a detailed process diagram based on step S20 in the first embodiment; Figure 4 This is a detailed schematic diagram of step S40 based on the first embodiment; Figure 5 This is a detailed process diagram based on step S50 in the first embodiment; Figure 6This is a flowchart illustrating the second embodiment of the spectral mapping method based on multi-source point cloud collaborative registration in this application; Figure 7 This is a schematic diagram illustrating the detailed process of step S80 in the second embodiment; Figure 8 This is a flowchart illustrating the third embodiment of the spectral mapping method based on multi-source point cloud collaborative registration in this application; Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the spectral mapping method based on multi-source point cloud collaborative registration in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] In related technologies, the fusion of structural and spectral information from multi-source remote sensing data in forest scenes faces the core challenge of insufficient cross-modal spatial registration accuracy. On the one hand, conventional methods based on direct registration of 3D point clouds employ iterative nearest-point algorithms or manual feature descriptors to align motion-reconstructed structural point clouds with lidar point clouds. However, in forest scenes, motion-reconstructed structural point clouds exhibit high noise and distortion in weakly textured areas within the canopy. Furthermore, the density distributions of the two types of point clouds are highly mismatched, making the algorithm prone to getting trapped in local optima. Simultaneously, the non-rigid deformation of trees and the self-similarity of texture further lead to frequent feature mismatches, severely limiting the robustness of global registration. On the other hand, traditional vertical projection methods implicitly assume continuous opacity of the ground surface during spectral mapping, failing to handle occlusion relationships in the three-dimensional structure of forests. This results in erroneous contamination of the understory structure by canopy spectral signals, producing a "spectral smearing" phenomenon that affects the accuracy of understory parameter inversion. These two approaches, operating independently, struggle to simultaneously achieve high-precision spatial alignment and high-fidelity spectral attribution in complex forest scenes.

[0025] Based on the aforementioned deficiencies in related technologies, this application proposes a spectral mapping method based on multi-source point cloud collaborative registration. In this method, to address the dual technical obstacles of low registration accuracy due to cross-modal data dimensional heterogeneity and the lack of occlusion discrimination capability in conventional mapping mechanisms, a technical chain of "2.5D projection dimensionality reduction—deep semantic matching—visibility analysis" is constructed. Specifically, the process begins by acquiring a digital terrain model and a UAV digital surface model of the target forest area. These are then rasterized into grayscale depth maps. A depth map neural network is used to extract deep geometric semantic features such as terrain undulations and gully directions for matching, and a coarse registration transformation matrix is ​​calculated to unify the heterogeneous data to the same geospatial reference. Next, based on the coarse registration transformation matrix, the LiDAR point cloud and the UAV digital surface model are aligned in space. A canopy height model is then generated through projection rasterization. The same depth map neural network is used to perform secondary geometric feature matching between the canopy height model and the UAV digital surface model, identifying high-frequency geometric feature pairs such as treetop high points and canopy edges. A fine registration transformation matrix is ​​calculated to correct residual biases. Finally, in the unified space after fine registration, the HPR transform algorithm is used to perform visibility analysis on the point cloud. Convex hull calculation divides the point cloud into a visible set and an occluded set. Based on point cloud attribute field expansion and differentiated writing strategies, the spectral information of the digital orthophoto is written into the expanded field of the visible set, while writing operations to the occluded set are blocked to preserve the original LiDAR intensity information. Through the aforementioned coarse-to-fine spatial alignment mechanism and differentiated spectral mapping strategy at the underlying data structure level, this application effectively overcomes the core bottleneck of low registration accuracy of multi-source heterogeneous data in forest scenarios, and eliminates artifacts and spectral errors caused by traditional vertical projection from a mechanistic perspective, providing a reliable data fusion technology solution for fine forest modeling, carbon sequestration measurement and ecological monitoring.

[0026] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0027] Based on this, embodiments of this application provide a spectral mapping method based on multi-source point cloud collaborative registration, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the spectral mapping method based on multi-source point cloud collaborative registration according to this application. In this embodiment, the spectral mapping method based on multi-source point cloud collaborative registration includes steps S10 to S50: Step S10: Using a deep graph neural network, deep geometric semantic features are extracted from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matched to obtain a coarse registration transformation matrix. In this embodiment, the core objective is to overcome the technical obstacle of directly matching 3D LiDAR point clouds and 2D UAV images in forest scenes due to the heterogeneity of data dimensions. By transforming the 3D spatial alignment problem into a 2D image semantic matching problem, a preliminary spatial correspondence between heterogeneous data is established.

[0028] First, digital terrain model data and UAV digital surface model data for the target forest area were acquired. The digital terrain model is a high-resolution terrain elevation surface generated from the original point cloud of ground-based lidar. After separating ground points from non-ground points using a cloth-based simulation filtering algorithm, the ground point set is rasterized and interpolated. This digital terrain model reflects the vertical undulations of the forest surface, including stable and unchanging terrain structural features such as gullies, slope transitions, and micro-topographic variations. It possesses extremely high spatiotemporal stability and is unaffected by seasonal changes or tree growth. The UAV digital surface model is a high-resolution surface elevation model generated by performing aerial triangulation and dense matching on UAV aerial photographic sequences using a motion reconstruction structure algorithm. This model records the surface morphology of the entire scene, including the canopy top surface, branch surfaces, and bare ground in forest gaps. Although the two data sources differ, they both express the surface or near-surface geometric information of the same geographical area in the form of continuous elevation surfaces, forming the data basis for dimensionality reduction matching.

[0029] Next, 2.5D projection dimensionality reduction processing was performed on the aforementioned digital terrain model and UAV digital surface model. Using a unified spatial resolution as a benchmark, the digital terrain model and UAV digital surface model were divided into grids on a horizontal projection plane, with each grid cell corresponding to a pixel location. For the digital terrain model, the average elevation value of the LiDAR ground points within each grid cell was extracted as the basis for grayscale encoding of that pixel; for the UAV digital surface model, the median or highest elevation value of the densely reconstructed points within each grid cell was extracted as the basis for grayscale encoding of the corresponding pixel. By linearly mapping the elevation values ​​to the grayscale range of 0 to 255, two high-resolution grayscale depth images reflecting the absolute elevation distribution of the terrain surface were generated. In these two grayscale depth images, the level of pixel grayscale value directly corresponds to the absolute elevation of the terrain, and geometric information such as terrain undulations, gully orientations, and slope transitions is losslessly encoded into the texture and edge features of the two-dimensional image.

[0030] Subsequently, a deep graph neural network is used to perform semantic-level feature extraction and matching on the two grayscale depth images. This embodiment selects the LightGlue network as the core operator for feature matching. Unlike traditional handcrafted feature descriptors that rely on local gradient statistics, the LightGlue network possesses global context awareness, enabling it to extract deep geometric semantic feature vectors reflecting the macroscopic trend and local detail changes of the terrain from the entire grayscale depth image. Specifically, during the network's forward inference process, multi-level feature pyramids are first constructed for the two input grayscale depth images, capturing geometric semantic information from large-scale gully systems to small-scale terrain transitions at different scales, from coarse to fine. Next, the network's internal attention mechanism module performs cross-image information interaction and weighting on the feature maps of the two images, automatically focusing on terrain structure regions with high discriminative power and suppressing interference from texture-poor or noisy regions. Finally, the network outputs a set of high-confidence two-dimensional feature point pairs. Each pair of feature points corresponds to the same location of the same terrain entity in the digital terrain model grayscale depth image and the UAV digital surface model grayscale depth image, such as the same gully intersection, the same ridge turning line, or the same slope abrupt change.

[0031] Based on the set of two-dimensional feature point pairs output by the LightGlue network, and combined with the known rasterization spatial resolution and georeferenced information, the two-dimensional image coordinates are inversely calculated back to three-dimensional geospatial coordinates, obtaining corresponding terrain feature point pairs in three-dimensional space. Using these three-dimensional corresponding point pairs, a rigid transformation estimation algorithm is employed to calculate the coarse registration transformation matrix, which includes rotation and translation vectors. This coarse registration transformation matrix quantitatively describes the global spatial transformation relationship between the lidar point cloud coordinate system and the UAV image reconstruction coordinate system, achieving preliminary alignment of the two types of heterogeneous data at a macro-geospatial scale. This lays a unified coordinate benchmark for subsequent fine alignment and canopy height model generation.

[0032] Step S20: Based on the coarse registration transformation matrix, spatially align the ground-based lidar point cloud of the target forest area with the space where the UAV digital surface model is located to generate a canopy height model; This embodiment follows the coarse registration transformation matrix output above. Specifically, it transforms the original ground-based lidar point cloud into a coordinate system unified with the UAV digital surface model, and generates a canopy height model representing the vertical distribution of vegetation by removing the terrain foundation elevation and rasterizing the projection. This canopy height model will serve as the data basis for secondary geometric feature matching with the UAV digital surface model in the subsequent fine registration stage.

[0033] In the specific implementation process, the original point cloud data of the generated ground-based lidar is acquired. This point cloud data records the three-dimensional spatial coordinates of each point and the laser echo intensity value. Its spatial coordinate system is either the independent coordinate system used for ground station measurement or an absolute coordinate system transformed by real-time dynamic positioning technology. Due to the difference in positioning accuracy between the ground-based lidar and the UAV during data acquisition using the Global Navigation Satellite System, and the potential inconsistency in the coordinate transformation parameters used by the two, there is a systematic translation and rotation deviation between the original point cloud and the UAV digital surface model.

[0034] During spatial alignment, each point in the ground-based lidar point cloud undergoes coordinate transformation according to a coarse registration transformation matrix. This transformation matrix is ​​a 4x4 homogeneous matrix, with rotation and translation components in the first three dimensions, and the last row consisting of 0001. The transformed point cloud coordinates are within the same georeferenced frame as the UAV digital surface model. Corresponding features, such as forest edges and protruding rocks, should roughly coincide in spatial location, but sub-pixel residual deviations may still exist locally. These deviations will be corrected in subsequent fine registration steps.

[0035] After obtaining the aligned ground-based lidar point cloud, a canopy height model is further generated. First, the generated time-invariant digital terrain model is invoked, which stores the surface elevation value of each pixel in raster form. For the aligned lidar point cloud, the vegetation height of each point cloud is calculated point-by-point or grid-by-grid. Specifically, for any lidar point, its three-dimensional coordinates are horizontal spatial coordinates and absolute elevation values. The surface elevation values ​​at the corresponding pixels of the digital terrain model for the north and east coordinates are retrieved, and the vertical height of that point relative to the ground is obtained by subtracting the surface elevation value from the elevation value of that point. For point clouds located at the top of tree canopies, this difference represents the tree height; for point clouds located on tree trunks or low vegetation, this difference reflects their distance from the ground. After calculating the height of all point clouds, all point clouds are projected onto a horizontal plane according to their north and east coordinates, and rasterized with a preset grid spacing, such as 20 centimeters. The maximum height value within each grid is taken as the canopy height of that grid, thus generating a rasterized canopy height model. This model represents the pure vegetation vertical structure information after removing the influence of terrain, including high-frequency geometric features such as treetop height, canopy edges, and forest gap distribution. The above description completes the construction of the point cloud alignment and canopy height model, providing a data source with a structure similar to the UAV digital surface model for the next step of fine registration.

[0036] Step S30: The deep graph neural network is used to perform secondary geometric feature matching on the canopy height model and the UAV digital surface model. High-frequency geometric feature point pairs of the canopy are identified in the matching results, and the fine registration transformation matrix is ​​calculated based on the identification results. Based on the coarse registration, a secondary fine matching is performed using the shared canopy geometry information of the canopy height model and the UAV digital surface model, through a depth map neural network. This fine registration transformation matrix is ​​then calculated to correct the local residual biases left over from the coarse registration stage. This step resolves the local misalignment problem caused by terrain smoothing effects and interpolation errors between the digital terrain model and the UAV digital surface model.

[0037] In the specific implementation process, a canopy height model and a UAV digital surface model were acquired. Both models are stored in raster form and have the same spatial resolution and geographical range. The canopy height model records the vertical height distribution of vegetation after removing topography, while the UAV digital surface model records the absolute elevation of the canopy top and forest gap ground. The two models are highly correlated geometrically: at the treetop, both the canopy height model and the UAV digital surface model exhibit local maxima in elevation; at the canopy edges and forest gap boundaries, both show steep elevation changes. These commonalities in their geometric structures form the physical basis for secondary fine-tuning.

[0038] Subsequently, the LightGlue deep map neural network was used to extract and match features between the canopy height model and the UAV digital surface model. The network input consisted of local feature point sets extracted from the two raster images by a convolutional backbone network. In this step, the LightGlue network automatically focused on high-frequency geometric features of the canopy through self-attention and cross-attention mechanisms, including the peak shape of the treetops, the stepped changes at the canopy edges, the break lines at the forest window boundaries, and the gaps between the canopies. These features are represented in the images as local extrema of grayscale values ​​or areas of drastic gradient changes. After multiple attention iterations, the network output a set of sub-pixel-level feature point pairs with high confidence. Each feature point pair is associated with a geometric feature point in the canopy height model and its corresponding geometric feature point in the UAV digital surface model.

[0039] After feature matching is completed, mismatches are removed from the aforementioned set of feature point pairs. Considering that there are only minor residual deviations and no inconsistent structural changes between the canopy height model and the UAV digital surface model, a distance threshold-based random sampling consensus algorithm can effectively filter outlier matches. The selected high-quality feature point pairs are used as control points, and the singular value decomposition method is used to calculate the fine registration transformation matrix between the two images. This matrix is ​​also a 4x4 homogeneous matrix, containing finely adjusted rotation and translation components. The obtained coarse registration transformation matrix is ​​concatenated with the fine registration transformation matrix to form the final joint registration transformation matrix. Applying this joint matrix to the coarsely aligned LiDAR point cloud achieves sub-pixel-level high-precision spatial alignment. The above description completes the entire process from coarse registration to fine registration, ensuring strict consistency between UAV data and LiDAR data in both the horizontal and vertical directions.

[0040] Step S40: In the unified space of the fine registration transformation matrix, the HPR algorithm is used to perform visibility analysis on the point cloud, and the point cloud is divided into a visible set and an occlusion set according to the analysis results. Within a unified spatial framework that has achieved geometric registration, the Hidden Point Removal (HPR) transform algorithm is used to simulate the light propagation and occlusion relationships under orthographic projection. This algorithm performs a binary classification of point cloud visibility, distinguishing between visible canopy surface points and invisible internal branches and occluded ground points from the orthographic perspective. The HPR algorithm, used for point cloud data visibility determination, addresses the core technical problem of traditional vertical projection methods' inability to handle three-dimensional occlusion relationships, leading to spectral smearing.

[0041] In the specific implementation process, the calculated precise registration transformation matrix is ​​applied to all point cloud data, including the original point cloud from the ground-based lidar and the initial point cloud reconstructed from the motion-reconstructed structure of UAV imagery. After the above transformation, all point clouds are located in a unified absolute geographic coordinate system and a precise mapping relationship is established with the pixel space of the UAV digital orthophoto. In this unified space, the orthophoto projection viewpoint is set to look down at the ground surface along the vertical direction, which is consistent with the acquisition viewpoint of the UAV digital orthophoto.

[0042] Subsequently, the HPR algorithm was used to perform visibility analysis on the point cloud. The core operations of this algorithm include spherical inversion and convex hull calculation. Specifically, for each 3D point in the point cloud, the virtual camera position (i.e., the infinity viewpoint of the orthographic projection) is used as a reference point, and the coordinates of all points are transformed to an inversion space with this reference point as the origin. Spherical inversion essentially involves nonlinearly inverting the position vector of each point, mapping points originally located far from the reference point to near points. After this transformation, the 3D convex hull of the point set after spherical inversion is calculated. The convex hull is the smallest convex polyhedron containing all points. According to computational geometry principles, points located at the vertices of the convex hull are visible from the reference point's perspective in the original space because no other points obstruct their view; while points located inside the convex hull are obstructed in the original space, their viewpoints blocked by points at the vertices of the convex hull.

[0043] Based on the above principles, the original point cloud located at the vertices of the convex hull of the point set after spherical inversion is labeled as the visible set, and the original point cloud located inside the convex hull is labeled as the occluded set. These labeling results give the point clouds a clear physical meaning: the visible set corresponds to the leaves on the surface of the canopy and the ground in forest gaps as observed from an orthophoto perspective; these areas should receive spectral mapping from UAV imagery. The occluded set corresponds to the branches inside the canopy, the understory vegetation occluded by the upper canopy, and the completely occluded ground; these areas should not receive spectral information from orthophotos, otherwise, spectral smearing will occur where canopy color is incorrectly attached to the tree trunk or ground. This description completes the visibility classification of the point cloud, providing precise semantic partitioning for the next step of differentiated spectral mapping.

[0044] Step S50: Map the spectral information of the digital orthophoto of the target forest area to the visible set, and after masking the spectral mapping of the occlusion set, obtain the spectral mapping result of the target forest area.

[0045] Based on the defined visible and occluded sets, a differentiated spectral attribution strategy is implemented: for canopy surface and forest gap ground point clouds visible from an orthophoto perspective, multispectral reflectance and true color information from UAV digital orthophotos are accurately written into the extended attribute fields of the point cloud; for occluded canopy branches and ground point clouds, pixel value writing is completely blocked, preserving their original lidar intensity information or ground-based radar side-view texture. This differentiated processing logic fundamentally eliminates spectral smearing artifacts caused by traditional vertical projection methods, achieving high-fidelity fusion of structural and spectral information.

[0046] In the specific implementation process, the first step is to acquire the generated UAV digital orthophoto. This image undergoes radiometric calibration and geometric correction, with each pixel recording the digital quantization values ​​of the three true-color bands (red, green, and blue) and the surface reflectance of multiple multispectral bands such as near-infrared and red edge. The spatial resolution of the digital orthophoto is better than five centimeters, completely consistent with the projection coordinate system of the point cloud, and undergoes coarse and fine registration transformations to ensure a strict correspondence between the image pixels and the spatial positions of ground objects.

[0047] For point clouds labeled as a visual set, spatial indexing techniques are used for spectral mapping. Since the number of point clouds in a visual set typically reaches millions or even tens of millions, to improve mapping efficiency, a spatial index structure is first established for the digital orthophoto image. Common indexing methods include quadtree indexing or grid indexing. For each point in the visual set, its horizontal spatial coordinates are used to quickly locate the corresponding pixel position in the digital orthophoto image via spatial indexing, and the true color value and multispectral reflectance value of that pixel are read. Considering the discreteness of the point cloud and the continuity of the image, for points falling near pixel boundaries, a bilinear interpolation method is used to obtain the spectral value from the weighted average of four adjacent pixels to eliminate the jagged effect. The read spectral values ​​are written point by point into the corresponding extended attribute field of the point cloud to maintain the stability of the original data structure, preventing direct replacement of the original scalar lidar intensity information.

[0048] For point clouds marked as occlusion sets, a write blocking strategy is implemented. Specifically, the spatial index query and spectral value assignment processes are blocked, preventing any pixel information from digital orthophotos from being written to the occlusion set point cloud. This portion of the point cloud retains its original attribute structure, preserving the laser echo intensity values ​​recorded by the original ground-based lidar, or the side-view texture information acquired during ground-based scanning. Since the scanning angle of the ground-based lidar is horizontal or upward, its intensity information can accurately reflect the material properties of tree trunk surfaces, forest floor surfaces, and low-lying vegetation. This information is of significant value in subsequent applications such as tree species identification and biomass inversion.

[0049] After completing the aforementioned differential mapping, the visible point cloud carrying spectral information and the occluded point cloud retaining the original intensity information are merged to form a complete spectral mapping result dataset. In this dataset, canopy surface points possess high-fidelity multispectral reflectance and true-color information, while points inside the canopy and occluded ground points possess accurate lidar intensity information. The two types of point clouds are spatially seamlessly connected without any spectral cross-contamination. The above description completes the final output of the entire spectral mapping method based on multi-source point cloud collaborative registration, providing a high-quality data foundation for applications such as forest fine-grained modeling, tree species identification, and carbon sequestration.

[0050] Furthermore, you can also view Figure 2 , Figure 2 This is a detailed process diagram based on step S10 in the first embodiment. Figure 2 The step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain a coarse registration transformation matrix includes S11~12: Step S11: Use LightGlue depth map neural network to perform global semantic analysis on the terrain undulation, gully direction and slope change in the grayscale depth map; Step S12: Extract terrain feature point pairs from the analysis results, calculate global translation and rotation parameters based on the terrain feature point pairs, and obtain the coarse registration transformation matrix based on the calculation results.

[0051] This embodiment further specifies the concrete implementation method of using a deep graph neural network to extract deep geometric semantic features for matching and calculating the coarse registration transformation matrix. The core of this implementation method lies in selecting the LightGlue deep graph neural network as the feature extraction and matching tool, and performing global semantic analysis on low-frequency geometric features such as terrain undulations, gully directions, and slope changes contained in the grayscale depth maps generated by the digital terrain model and the UAV digital surface model. This extracts robust terrain feature point pairs, and finally calculates the global coarse registration transformation matrix.

[0052] In the specific implementation, the two generated grayscale depth maps are input into the LightGlue depth map neural network. This network consists of two main modules: a feature extraction backbone network and an attention-based graph matching module. The feature extraction backbone network adopts a lightweight convolutional neural network structure, such as the improved SuperPoint architecture, to perform dense feature point detection and descriptor generation on the input image. For the grayscale depth map of the digital terrain model, the network responds to areas in the image where the terrain elevation values ​​change significantly, such as the turning points of valley lines and ridge lines, abrupt slope changes, and the intersections of gullies. For the grayscale depth map of the UAV digital surface model, although its absolute elevation values ​​include canopy height, when trained together with the digital terrain model, the network can automatically learn the common terrain skeleton features of both, ignoring the local high-frequency noise brought by the canopy. The core advantage of the LightGlue network lies in its self-attention and cross-attention mechanisms: In the self-attention stage, feature points within each image communicate with each other through attention weights, enabling the network to understand the global context of terrain features, such as ensuring the continuous direction of a gully should not be interrupted by local noise; in the cross-attention stage, feature points from two images exchange information across images, allowing the network to determine whether a ridge feature in the digital terrain model matches the same ridge feature in the UAV digital surface model. After multiple layers of attention iterations, the network outputs a set of candidate feature point pairs with confidence scores. Because the network directly regresses sub-pixel level coordinates, the localization accuracy of these feature point pairs can reach a fraction of a pixel, far exceeding the integer pixel accuracy of traditional feature descriptors.

[0053] Post-processing is performed on the output candidate feature point pairs. Since there may be a small number of mismatches in terrain features due to canopy occlusion in forest scenes, a random sampling consensus algorithm is used to iteratively remove outlier point pairs. The basic principle of the random sampling consensus algorithm is to randomly extract the minimum sample set, such as three pairs of points, from the candidate point pairs, calculate the assumed transformation model, and then count the number of interior points that satisfy the model. After multiple iterations, the transformation model with the most interior points is retained. After the above screening, the remaining feature point pairs are all high-confidence terrain homonyms, such as the precise correspondence between the bottom of the same gully or the top of the same ridge in two models. Based on the above terrain feature point pairs, the singular value decomposition method is used to solve for the optimal rigid body transformation parameters between the two grayscale depth maps. The specific calculation process is as follows: First, calculate the centroid coordinates of the two sets of point pairs, and subtract the centroid from each point to obtain the decentralized coordinates; then construct the covariance matrix and perform singular value decomposition on the matrix to obtain the rotation matrix; finally, calculate the translation vector based on the difference between the rotation matrix and the centroid. The above rotation matrix and translation vector together constitute the coarse registration transformation matrix. Applying this matrix to the UAV digital surface model and its associated digital orthophoto can achieve preliminary global alignment between UAV data and LiDAR data. The above description completes the entire technical chain from grayscale depth map input to coarse registration transformation matrix output, laying a reliable initial pose foundation for subsequent fine registration.

[0054] Furthermore, you can also view Figure 3 , Figure 3 This is a detailed process diagram based on step S20 in the first embodiment. Figure 3 The step of spatially aligning the ground-based lidar point cloud of the target forest area with the space of the UAV digital surface model based on the coarse registration transformation matrix to generate a canopy height model includes S21~22: Step S21: Use a coarse registration transformation matrix to transform the ground-based lidar point cloud to the same coordinate system as the UAV digital surface model; Step S22: After removing the terrain elevation represented by the digital terrain model from the point cloud in the coordinate system, the vegetation height of each point cloud relative to the ground surface is obtained, forming a normalized point cloud. Step S23: Project the normalized point cloud onto a horizontal plane and rasterize it, and extract the maximum vegetation height in each raster cell to generate the canopy height model.

[0055] This embodiment further defines the specific implementation method for spatially aligning lidar point clouds and generating a canopy height model based on a coarse registration transformation matrix. The core of this implementation method includes two consecutive operations: first, the ground-based lidar point cloud is transformed into a coordinate system identical to the UAV digital surface model using a coarse registration transformation matrix, eliminating systematic translation and rotation deviations between the two; then, under the unified coordinate system, the terrain elevation values ​​represented by the digital terrain model are systematically removed from the aligned point cloud, the height of each point cloud relative to the ground surface is calculated, and finally, a canopy height model representing the vertical distribution of pure vegetation is generated through rasterization.

[0056] In the specific implementation process, the generated ground-based lidar raw point cloud dataset is acquired. This dataset is usually stored in binary format, containing the horizontal spatial coordinates, absolute elevation value, and attributes such as laser echo intensity and echo count for each point. Because the ground-based lidar uses an independent coordinate system or an absolute coordinate system transformed by real-time dynamic positioning technology when it is set up, while the UAV digital surface model uses an absolute geographic coordinate system after motion-reconstructed structure calculation, there is an unknown rigid body transformation between the two.

[0057] The calculated coarse registration transformation matrix is ​​the mathematical expression of this rigid body transformation. During coordinate transformation, for each point in the ground-based lidar point cloud, its three-dimensional coordinates are represented as a homogeneous column vector. This vector is then multiplied by the 4x4 coarse registration transformation matrix to obtain a new homogeneous column vector. The first three components of this new homogeneous column vector are the transformed horizontal spatial coordinates and absolute elevation values. This transformation operation is typically accelerated using parallel computing architectures such as unified computing device architectures to process millions or even tens of millions of point cloud data points. After the transformation, all point clouds and the UAV digital surface model are within the same georeferenced frame. Corresponding features, such as isolated trees and forest edges, roughly overlap in spatial location, but sub-pixel-level residual deviations may still exist locally. These deviations will be further corrected by fine registration.

[0058] Additionally, a time-invariant digital terrain model is obtained. This model is stored in the form of a two-dimensional raster, with each raster pixel recording the surface elevation value at that location. For the converted aligned point cloud, the vertical height of each point relative to the ground needs to be calculated. The specific operation consists of the following sub-steps: First, based on the point's north and east coordinates, locate the corresponding raster pixel position in the digital terrain model. Since the coordinates of the point cloud are continuous values, while the digital terrain model is a discrete raster, a bilinear interpolation method is usually used to calculate the surface elevation interpolation value of the point location from the elevation values ​​of four adjacent pixels. Second, subtract the above surface elevation interpolation value from the point's elevation value to obtain the relative height of the point. If the point is located at the top of a tree canopy, the relative height is a positive value equal to the tree height; if the point is located near the ground, the relative height is close to zero; if the point is located in a pit below the ground surface, the relative height may be negative, and such points are usually discarded as noise. Third, attach the relative height values ​​of all point cloud points as a new attribute field to each point, thereby forming a normalized point cloud.

[0059] After calculating the height of all points, a rasterization process is performed to generate a canopy height model. The spatial resolution of the output raster is set to match the digital terrain model, for example, 20 centimeters, and a blank raster matrix is ​​created. All point clouds are traversed. For each point, the corresponding raster row and column number are calculated based on its north and east coordinates. The relative height of that point is compared with the currently stored maximum height value of that raster, and the larger value is retained. After traversal, the value of each raster pixel represents the maximum vegetation height at that location. For rasters without point cloud data, which typically represent gaps within dense canopies, neighborhood interpolation or null values ​​are used to represent forest gaps. The above description completes the entire generation process from the original lidar to the canopy height model, providing core data characterizing the vertical structure of vegetation for subsequent fine registration.

[0060] Furthermore, you can also view Figure 4 , Figure 4 This is a detailed process diagram based on step S40 in the first embodiment. Figure 4 In the unified space of the fine registration transformation matrix, the HPR algorithm is used to perform visibility analysis on the point cloud, and the point cloud is divided into a visible set and an occlusion set according to the analysis results, including steps S41~42: Step S41: Perform spherical inversion on the point cloud using HPR transformation to obtain a flipped point set, and determine the convex hull of the flipped point set; Step S42: Mark the point cloud located at the vertices of the convex hull as the visible set, and mark the point cloud located inside the convex hull as the occlusion set. The visible set represents the canopy surface and forest window ground, and the occlusion set represents the branches inside the canopy and the occluded ground.

[0061] This embodiment defines a specific implementation method for performing visibility analysis on point clouds and classifying them into visible and occluded sets using the HPR transform algorithm. The core of this implementation method includes two key operations: first, performing spherical inversion on the 3D point cloud using HPR transform and calculating the convex hull of the flipped point set; then, marking points as visible or occluded sets based on whether they are located at vertices or inside the convex hull, and assigning clear physical meanings to these two types of point clouds. This technique accurately simulates the ray occlusion relationship under orthographic projection from a computational geometry perspective, solving the problem that traditional empirical methods cannot accurately determine forest canopy occlusion.

[0062] In the specific implementation process, the first step is to acquire a unified spatial point cloud dataset after precise registration. All point clouds in this dataset are located in the same absolute geographic coordinate system and have established a precise spatial mapping relationship with UAV digital orthophotos. The orthophoto perspective is defined as vertically downwards, equivalent to assuming a virtual camera is positioned at an infinite distance directly above the point cloud and shooting vertically downwards. To achieve visibility analysis from this perspective, the spherical inversion operation in the HPR algorithm is used. The specific steps of this operation are as follows: For each point in the point cloud, the optical center position of the virtual camera is used as the reference point. Since orthophoto projection can be equivalent to parallel light projection, the reference point can be set as a virtual point located sufficiently far above the point cloud bounding box. For example, the reference point coordinates can be set as the minimum horizontal spatial coordinates, the minimum horizontal spatial coordinates, and the maximum elevation value of the entire point cloud, plus an offset greater than the point cloud height.

[0063] Then, the coordinates of each point are transformed into an inversion space with the reference point as the origin. The mathematical essence of the HPR transform is to map the position vector of each point from Cartesian space to spherical angle space. Its core innovation lies in the subsequent spherical inversion operation: taking the reciprocal of the radial distance of each point or performing some form of flipping mapping, so that points that were originally far from the reference point move closer to the reference point, and points that were originally close to the reference point move away from the reference point. After completing the above spherical inversion, a new set of flipped points is obtained. Then, the 3D convex hull of this set of flipped points is calculated. The convex hull is a fundamental concept in computational geometry, defined as the smallest convex polyhedron containing all points, whose boundary is composed of a series of triangular facets. Common algorithms for calculating the 3D convex hull include incremental methods, divide-and-conquer methods, or fast convex hull algorithms. For point clouds with millions of points, fast convex hull algorithms are usually used in combination with parallel computing techniques to improve efficiency. The result of convex hull calculation is a set of point indices located on the vertices of the convex hull, and the topological relationships of the triangles that constitute the surface of the convex hull.

[0064] Furthermore, visibility classification is performed based on the computational convex hull results. According to the properties of the convex hull in computational geometry: from the perspective of the reference point, points located at the vertices of the convex hull are the "outermost" points in the flipped space, corresponding to points visible from the reference point in the original space; points located inside the convex hull are points occluded by these vertices in the original space. Therefore, the original point clouds corresponding to those points in the flipped point set located at the convex hull vertices are labeled as the visible set, and the original point clouds corresponding to those points in the flipped point set located inside the convex hull are labeled as the occluded set. These labeling results are further assigned physical meaning in forestry applications: the visible set point clouds correspond to the canopy surface leaves visible from an orthophoto perspective and the forest floor not obscured by the canopy; these areas should receive spectral information from UAV digital orthophotos. The occluded set point clouds correspond to the branches inside the canopy, the understory vegetation obscured by the upper canopy, and the completely obscured ground; these areas should not receive spectral information from orthophotos, otherwise, a spectral smearing phenomenon will occur where canopy color is incorrectly attached to the tree trunk or ground.

[0065] This embodiment completes the entire visibility analysis process from point cloud input to visible and occlusion sets output, providing accurate semantic partitioning basis for differential spectral mapping. It requires no manual annotation or prior assumptions, automatically calculating occlusion relationships entirely based on the geometric distribution of the point cloud itself. Furthermore, the computational complexity is approximately linearly related to the number of point clouds, making it suitable for processing large-scale forestry point cloud data.

[0066] Furthermore, you can also view Figure 5 , Figure 5 This is a detailed process diagram based on step S50 in the first embodiment. Figure 5 The step of mapping the spectral information of the digital orthophoto of the target forest area to the visible set, and then masking the spectral mapping of the occlusion set to obtain the spectral mapping result of the target forest area includes S51~53: Step S51: Expand the multispectral and true color attribute fields in the original 3D point cloud data structure; Step S52: Use spatial indexing technology to extract the multispectral reflectance and true color pixel values ​​in the digital orthophoto, and write the true color pixel values ​​point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance. Step S53: For the point cloud of the occlusion set, maintain the original attribute structure and original laser intensity field of the point cloud of the occlusion set unchanged, and combine the mapping result and the shielding result into the spectral mapping result.

[0067] This embodiment further defines the specific implementation method for mapping the spectral information of digital orthophotos to the visible set and masking the spectral mapping of the occluded set, as well as the operation of combining the two results into the final spectral mapping result. The core of this implementation method includes two parallel processing branches: the first branch uses spatial indexing technology to efficiently write the multispectral reflectance and true color data of the digital orthophotos into the corresponding extended surface attribute fields of the visible set point cloud, while completely blocking the occluded set point cloud from receiving any spectral attributes; the second branch merges the processing results of the above two branches to form a complete and spectrally uncontaminated output dataset.

[0068] In the specific implementation process, differentiated processing logic is first executed for the visible set and the occluded set. For the visible set point cloud, the spectral information in the UAV digital orthophoto needs to be accurately assigned to each point. Since the number of visible set point clouds typically reaches millions, and the number of pixels in the digital orthophoto is also enormous, if a point-by-point traversal search method is used, the computational complexity is the number of point clouds multiplied by the image width and then multiplied by the image height, which is unacceptable on conventional computing devices. Therefore, spatial indexing technology is used to significantly improve mapping efficiency. In specific implementation, a spatial index structure is first established for the digital orthophoto. A commonly used indexing method is grid indexing: the image is divided into multiple fixed-size grid blocks according to the pixel row and column numbers. Each grid block contains, for example, 64 by 64 pixels, and the geographic spatial range of the grid block is recorded, i.e., minimum horizontal spatial coordinates, maximum horizontal spatial coordinates, minimum horizontal spatial coordinates, and maximum horizontal spatial coordinates. After indexing, for each point in the visual set, its horizontal spatial coordinates are used to quickly locate its corresponding grid block by comparing coordinate ranges. Then, nearest neighbor search or bilinear interpolation is performed only on pixels within that grid block. This grid index reduces the mapping time complexity from linear search to approximately constant levels. After locating the corresponding pixel, the true color value (red, green, and blue bands) and multispectral reflectance values ​​(near-infrared and red-edge bands) recorded for that pixel are read. Considering the sub-pixel level deviation between point cloud coordinates and image pixels, bilinear interpolation is used to obtain the spectral value from the weighted average of four adjacent pixels to eliminate aliasing. The read spectral values ​​are written to the extended attribute field of the point cloud, preventing direct replacement of the original LiDAR intensity information to maintain the stability of the original data structure. For occluded point clouds, a spectral blocking write operation is performed: no spatial index query or pixel reading process is performed, and the spectral assignment step for that point cloud is skipped. Occlusion point clouds retain the laser echo intensity values ​​recorded by the original ground-based lidar, or retain the side-view texture information acquired during ground-based scanning.

[0069] In the specific implementation process, the visible set point cloud, after spectrally mapped, is merged with the occluded set point cloud, after spectrally blocked. The merging operation is not a simple data splicing but requires maintaining the consistency of the spatial index structure and attribute fields of the point clouds. Specifically, first, an empty result point cloud dataset is created, with its coordinate system consistent with the original data. Then, all points in the visible set point cloud are appended to the result dataset one by one, each point carrying its original 3D coordinates and newly assigned multispectral reflectance and true color attributes. Next, all points in the occluded set point cloud are appended to the same result dataset, each point carrying its 3D coordinates and original LiDAR intensity attributes. During the appending process, it is important to note that the point clouds of the visible and occluded sets are spatially complementary, with no overlap or conflict, because the convex hull partitioning has ensured that each point in the point cloud is uniquely labeled as either the visible or occluded set. After merging, each point in the result dataset clearly distinguishes its attribute source: points belonging to the visible set possess high-fidelity spectral information, while points belonging to the occluded set possess true intensity information. The above results represent the final spectral mapping, which can be directly used for subsequent applications such as fine-grained forest modeling, tree species identification, and carbon sequestration. The above description completes the entire process from differential mapping to result merging.

[0070] Regarding the above Figure 5 Step S52 is further refined. The step of extracting the multispectral reflectance and true-color pixel values ​​from the digital orthophoto using spatial indexing technology, and writing the true-color pixel values ​​point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance, includes S52-1 to S52-3: Step S52-1: Traverse the visible point cloud, detect the original laser echo intensity value of each point, and mark the points whose intensity value is greater than the preset intensity saturation threshold as overexposed points; Step S52-2: Use spatial indexing to extract the true color data of the corresponding pixel position of each overexposed point in the digital orthophoto; Step S52-3: Write the extracted true color data into the attribute field of the overexposed point to repair the overexposure loss, and perform color consistency smoothing processing on the overexposed point and its spatial neighborhood point cloud.

[0071] This embodiment further defines an important enhancement in the specific implementation of mapping the spectral information of digital orthophotos onto the surface of a visible set of point clouds using spatial indexing technology: the detection and repair of overexposed points by the lidar sensor. The core of this implementation lies in identifying point clouds in the visible set of point clouds that are intensity-saturated due to the dynamic range limitation of the lidar sensor. It then uses the high dynamic range color information of the corresponding pixels in the digital orthophoto to repair the extended attribute fields of these overexposed points and smooths the surrounding neighborhood, thereby enhancing the integrity and consistency of the canopy surface spectral information.

[0072] In the specific implementation process, the raw intensity value of the lidar is first detected for each point in the visual point cloud. When the lidar sensor receives the echo signal, the response of its photodetector has an upper limit of linear range. When the target surface has extremely high reflectivity, such as smooth tree canopy leaves, under vertical incident conditions or when the detection distance is too close, the echo signal intensity may exceed the linear response threshold of the sensor, causing the recorded intensity value to be truncated to the maximum value. This phenomenon is called intensity overexposure. The intensity value of overexposed points cannot truly reflect the material reflectivity of the target, which will cause errors in subsequent classification and inversion applications. The specific detection method is as follows: Obtain the factory parameters of the lidar sensor and determine its upper limit of linear response range. For example, for an intensity value stored as a 16-bit integer, the upper limit of the linear range may be set to 60,000. Traverse all points in the visual point cloud and compare the intensity value of each point with the above upper limit threshold. If the intensity value of the point is greater than or equal to the threshold, it is marked as an overexposed point; if it is less than the threshold, it is considered a normal point, and its raw intensity value is retained without processing. The marking operation is implemented by adding a flag field to the point cloud attribute table. For example, setting this field to true indicates overexposure, and false indicates normal exposure. The above detection process can be accelerated for large-scale point clouds using a parallel computing architecture, such as allocating a thread on the graphics processor to perform the comparison operation independently for each point.

[0073] In the further implementation process, for each point cloud marked as overexposed, the image color of the corresponding pixel location needs to be obtained from the digital orthophoto as the restoration source. First, based on the horizontal spatial coordinates of the overexposed point, the corresponding pixel location in the digital orthophoto is quickly located using the established spatial index. Since the digital orthophoto has undergone radiometric calibration and geometric correction, the surface reflectance recorded by each pixel is accurate and reliable, and is not affected by lidar intensity saturation. During the positioning process, the bilinear interpolation method is also used to obtain the image color value of the point from the weighted average of four adjacent pixels. The aforementioned image color value includes the true color band (red, green, and blue channels) and the multispectral band (near-infrared, red-edge, etc.). The values ​​of these bands are read out to form a spectral feature vector corresponding to the overexposed point. It is worth noting that the radiometric resolution of digital orthophotos is usually higher than the intensity resolution of lidar. For example, the radiometric resolution of UAV multispectral sensors can reach twelve or sixteen bits, and after radiometric calibration, the reflectance values ​​are continuously distributed between zero and one, thus serving as a high-fidelity alternative data source.

[0074] Furthermore, instead of directly replacing its original laser scalar intensity value, the acquired image color value is written into the extended spectral attribute field corresponding to the overexposed point. This repair operation allows the point to retain its original abnormal intensity record while obtaining multispectral reflectance or true color values ​​extracted from the image in its extended attribute field. However, direct assignment may cause discontinuous jumps in the extended spectral values ​​between the overexposed point and its surrounding normal points. This is because the overexposed point's value is assigned based on the image color, while the spectrum of the surrounding normal points may be affected by the laser echo intensity characteristics, resulting in a numerical discontinuity in visual representation.

[0075] To eliminate the aforementioned discontinuities, color consistency smoothing is performed on the adjacent point clouds of overexposed points. The specific steps are as follows: For each repaired overexposed point, using a spatial indexing structure of the point cloud, such as an octree or KD tree, all neighboring points within a radius of, for example, ten centimeters are searched. These neighboring points typically include both normal points and other overexposed points. The difference between the repaired spectral value of the overexposed point and the original spectral values ​​of the neighboring points is calculated. Then, a weighted average method is used to adjust the spectral values ​​of the neighboring points, making the spectral gradient between the overexposed point and its neighbors smoother. The weighting coefficients are set based on the spatial distance from the neighboring points to the overexposed point; the closer the distance, the greater the weight. This smoothing process can be iterated multiple times until the spectral distribution of the entire canopy surface is continuous and natural. After completing the repair and smoothing of all overexposed points, the spectral information of the visible set point cloud both compensates for the overexposure defects of the lidar intensity and maintains consistency with the original intensity data. The above description completes the entire technical process for intensity overexposure detection and repair, significantly improving the quality of spectral mapping of the canopy surface.

[0076] Furthermore, you can also view Figure 6 , Figure 6 This is a flowchart illustrating the second embodiment of the spectral mapping method based on multi-source point cloud collaborative registration in this application. Before the step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain the coarse registration transformation matrix, steps S60-80 are included: Step S60: Obtain multi-temporal observation data of the target forest area. The multi-temporal observation data is a dataset that corresponds to the phenological characteristics of trees in the target forest area and includes canopy spectral information and texture information. Step S70: Perform radiometric calibration and motion recovery structure calculation on the canopy spectral information and texture information to generate digital orthophotos, UAV digital surface models and initial point clouds; Step S80: Ground-based lidar point cloud is used to acquire three-dimensional structure information of the forest underground. The ground-based lidar point cloud is then subjected to a cloth simulation filtering algorithm to decouple the terrain and vegetation features and generate a digital terrain model.

[0077] This embodiment defines the data acquisition and preprocessing operations performed before obtaining the digital terrain model and UAV digital surface model of the target forest area. The core of this operation comprises three consecutive stages: the first stage acquires multi-temporal canopy spectral and texture information based on forest phenological characteristics; the second stage performs radiometric calibration and motion reconstruction structure calculation on the acquired UAV imagery to generate digital orthophotos, the UAV digital surface model, and an initial point cloud; the third stage uses ground-based lidar to acquire three-dimensional structural information point clouds of the forest understory, and applies cloth simulation filtering to the ground-based lidar point clouds to decouple terrain and vegetation features, ultimately generating the digital terrain model. These three stages together constitute the data source of the entire method, providing standardized and traceable multi-source input data for subsequent coarse registration, fine registration, and spectral mapping.

[0078] In the specific implementation process, the acquired multi-temporal observation datasets need to match the detailed phenological characteristics of the target forest area. There are significant differences in the leaf unfolding, peak leafing, color change, and leaf fall periods among different tree species. For example, deciduous broad-leaved forests experience rapid changes in canopy structure after leaf unfolding in spring, and a sparse canopy after leaf fall in autumn; coniferous forests show relatively gradual canopy changes throughout the year but still exhibit seasonal growth rhythms. Based on these phenological characteristics, the acquired data corresponds to four key nodes: early leaf unfolding, peak leafing, early leaf fall, and complete leaf fall. The time window for each node is controlled within seven to fourteen days to minimize canopy appearance differences caused by phenological changes. When performing the above data collection tasks, a UAV equipped with a five-band multispectral sensor and a high-resolution RGB camera is used. The UAV flight path planning adopts a grid-like track, with a forward overlap rate set at no less than 80% and a lateral overlap rate set at no less than 70%. The flight altitude is adjusted according to the average tree height of the stand. For middle-aged forests with tree heights of 15 to 25 meters, the flight altitude is typically set at 80 to 120 meters to ensure a ground resolution better than 5 centimeters. Before flight, the multispectral sensor underwent dark current and whiteboard calibration, while the RGB camera used a fixed exposure time and white balance setting. During acquisition, real-time positioning data from the Global Navigation Satellite System and attitude data from the Inertial Measurement Unit were recorded simultaneously, providing accurate initial values ​​for exterior orientation elements for subsequent motion reconstruction calculations. The acquired canopy spectral and texture information included reflectance data in five multispectral bands: blue, green, red, red-edge, and near-infrared, as well as high-resolution true-color imagery.

[0079] Radiometric calibration is performed on the acquired UAV imagery. The core of radiometric calibration is converting the raw digital quantization values ​​of the imagery into surface reflectance. Specifically, before flight, gray boards with known reflectance are deployed on the ground. These gray boards typically contain three different gray levels, such as 5%, 20%, and 50% reflectance. After image acquisition, the digital quantization values ​​of the gray board regions are extracted, and a linear regression model between the digital quantization values ​​and reflectance is established. This model is then used to convert all pixels in the entire image into reflectance values. After radiometric calibration, the Structure for Motion Restoration (SRM) algorithm is used to perform 3D reconstruction of the image sequence. This algorithm first extracts scale-invariant feature transformation (SMT) feature points from each image and establishes topological relationships between images through feature point matching. Then, bundle adjustment is used to simultaneously solve for camera intrinsic parameters, including focal length, principal point coordinates, distortion coefficients, and extrinsic parameters for each image, namely position and attitude, generating a sparse point cloud. Finally, a multi-view stereo matching algorithm is used, leveraging epipolar constraints between the sparse point cloud and the images, to densely match pixels on each pair of images, generating a dense point cloud. This dense point cloud is then interpolated to generate a digital orthophoto image with absolute geographic coordinates and a UAV digital surface model. Digital orthophotos are orthorectified images that eliminate projection distortion caused by terrain undulations, with each pixel directly corresponding to the actual ground location; UAV digital surface models are rasterized elevation models, with each grid recording the elevation of the highest point on the ground at that location, including treetops, buildings, and the ground.

[0080] Furthermore, ground-based lidar was used to scan the same forest area, generating ground-based point clouds. Ground-based lidar is typically installed in forest clearings, acquiring detailed 3D structural information of the understory through multi-station scanning, including tree trunk location and diameter at breast height (DBH), low-lying vegetation, litter layer, and surface micro-topography. During scanning, the overlap rate between adjacent stations was no less than 30% to ensure the accuracy of subsequent point cloud registration. The acquired raw point cloud data contains millions to tens of millions of points, each recording its 3D coordinates and laser echo intensity. The point cloud was preprocessed, including outlier removal and noise reduction. Then, a cloth-based simulation filtering algorithm was used to decouple terrain and vegetation features. The core idea of ​​this algorithm is to simulate a virtual cloth covering the point cloud surface after elevation reversal, controlling the cloth's rigidity and gravity parameters so that the cloth adheres to ground points and crosses vegetation points during settlement.

[0081] After filtering, the point clouds of all points identified as terrain ground points are aggregated, and a digital terrain model with a grid spacing of better than 20 centimeters is generated through raster interpolation. This model only represents the true elevation of the forest understory and is not affected by seasonal changes in the canopy above, thus serving as a time-varying reference for subsequent coarse registration.

[0082] Furthermore, you can also view Figure 7 , Figure 7 This is a detailed process diagram based on step S80 in the second embodiment. Figure 7 The step of using a cloth simulation filtering algorithm to decouple terrain and vegetation features from the ground-based lidar point cloud to generate a digital terrain model includes steps S81-85: Step S81: Perform elevation value inversion processing on the ground-based lidar point cloud, and use the inverted ground-based lidar point cloud as the contact surface for cloth simulation. Step S82: Initialize a virtual cloth grid above the inverted ground-based lidar point cloud, and control the virtual cloth grid to sink downwards under the action of gravity to contact the surface of the ground-based lidar point cloud; Step S83: During the settlement process of the virtual cloth grid, the equilibrium position of the grid node is determined based on the collision constraint between the grid node and the reversed ground lidar point cloud surface, and the distance between the equilibrium grid node and the corresponding point cloud in the corresponding projection neighborhood is calculated. The category of the point cloud is determined according to the preset distance threshold. The category includes terrain ground points or vegetation non-ground points. Step S84: Collect the point cloud that is determined to be terrain ground points, and remove vegetation non-ground points to form a discrete terrain point set. Step S85: Perform rasterization interpolation on the discrete set of terrain points to generate the digital terrain model.

[0083] This embodiment further defines the specific implementation method for decoupling terrain and vegetation features and generating a digital terrain model by applying a cloth simulation filtering algorithm to ground-based lidar point clouds. The core of this implementation method includes five consecutive operational steps: First, the original point cloud undergoes elevation inversion, and the inverted point cloud serves as the contact surface for cloth simulation. Then, a virtual cloth grid is initialized above the inverted point cloud, and its downward settlement under gravity is controlled to contact the point cloud surface. Based on collision constraints with the inverted point cloud surface, the equilibrium positions of the grid nodes are determined, and the distance between the balanced grid nodes and the corresponding point cloud is calculated. A preset distance threshold is used to determine whether the point cloud belongs to a terrain ground point or a vegetation non-ground point. Next, all point clouds determined to be terrain ground points are collected, and vegetation non-ground points are removed, forming a discrete terrain point set. Finally, the discrete terrain point set is subjected to rasterization interpolation processing to generate a digital terrain model with continuous elevation information. These five steps constitute a complete physical simulation filtering process from the original point cloud to the digital terrain model.

[0084] In the specific implementation process, a ground-based lidar point cloud is acquired and preprocessed. This point cloud records the three-dimensional coordinates of the forest floor, tree trunks, low vegetation, and part of the canopy's lower surface. Since the original design of the cloth simulation filtering algorithm is based on the upright orientation of the point cloud (i.e., the air above and the ground below), but in actual forest areas, the ground is not strictly level and contains negative terrain such as gullies and pits, an elevation reversal is typically performed on the point cloud to simplify the algorithm. Specifically, all points in the point cloud are iterated through to find the maximum and minimum elevation values. A flip reference plane is then established, for example, by reversing the entire point cloud along the elevation direction; the new elevation value equals the maximum plus the minimum minus the original elevation value. After this reversal, the original ground points become the top of the point cloud, and the original points inside the tree canopy or in the air become the bottom of the point cloud. The inverted point cloud serves as the contact surface for the fabric simulation, and its surface morphology reflects the undulating characteristics of the terrain. Meanwhile, vegetation points such as tree canopies and trunks become raised obstacles after the inversion. This transformation allows the fabric to be naturally guided by the terrain and cross vegetation obstacles as it settles under gravity.

[0085] Furthermore, a virtual cloth mesh is initialized above the inverted point cloud. This cloth mesh consists of a series of regularly arranged nodes, with the node spacing adjusted according to the point cloud density, typically set to two to three times the average spacing of the point cloud, for example, 20 to 50 centimeters. The physical parameters of the cloth include stiffness coefficient, damping coefficient, and gravitational acceleration. The stiffness coefficient controls the cloth's ability to resist bending deformation; a higher stiffness coefficient makes the cloth more inclined to maintain a planar shape when encountering vegetation protrusions, thus crossing the vegetation, while a lower stiffness coefficient makes the cloth fit closely to terrain details. For forest scenes, the stiffness coefficient is usually set in the lower-middle range to balance the ability to follow terrain undulations with the suppression of vegetation noise. During initialization, the cloth mesh is located at a certain height above the inverted point cloud, for example, 10 centimeters above the maximum elevation value of the point cloud. Then, a simulation loop is started: within each time step, the velocity and position of each cloth node are updated according to gravitational acceleration, causing it to sink downwards. When a cloth node comes into contact with the point cloud surface below, the node position is adjusted according to a collision detection algorithm to prevent penetration. The above simulation continues until the position change of the cloth node is less than the convergence threshold or the maximum number of iterations is reached.

[0086] In addition, at each iteration step of the cloth settling process or after final convergence, a type determination is performed on each node of the cloth mesh. Specifically, for each cloth node, a subset of the point cloud corresponding to its horizontal projection position is found, and the distance between the elevation of the node and the elevation of the nearest point cloud point below it is calculated. If this distance is less than a preset distance threshold, the cloth node is considered to have adhered to the contact surface, and the contact point belongs to the terrain ground point. If the distance is greater than the threshold, the cloth node is considered to have been pushed up by vegetation points and has not made contact with the real ground; the point cloud point corresponding to this node is a vegetation non-ground point. The distance threshold is set based on the point cloud density and terrain roughness, and is usually set to one to two times the average spacing of the point clouds. Furthermore, the normal information or neighborhood consistency of the cloth node can be used for auxiliary determination. For example, isolated small vegetation areas may produce local protrusions, which can be filtered out through neighborhood smoothing. The above determination process is executed in parallel to obtain the type label of each point cloud.

[0087] Based on the judgment results, the point clouds of all points marked as terrain ground points are aggregated. Each placement node records the indices of the point cloud points it contacts during the simulation; the point cloud points corresponding to these indices are extracted to form a discrete terrain point set. Simultaneously, all nodes marked as vegetation non-ground points and their corresponding point cloud points are completely removed and not processed further. The aforementioned discrete terrain point set exhibits spatial unevenness: the point density is higher in flat terrain areas and may be lower in steep slopes or densely vegetated areas. This point set only includes points successfully identified as ground by the placement, excluding non-ground elements such as tree trunks and branches.

[0088] Furthermore, the resulting discrete terrain point set is subjected to rasterization interpolation to generate a digital terrain model with a regular grid. First, the spatial resolution of the output raster is set, typically consistent with or slightly finer than the spacing between the grid nodes, for example, 20 centimeters. A blank two-dimensional raster matrix is ​​created, with each raster pixel initialized to null values. For each point in the discrete terrain point set, the corresponding raster row and column number are calculated based on its north and east coordinates, and the elevation value of that point is written into the corresponding raster. Since the discrete point set may contain hollow areas, such as raster cells completely obscured by tree roots, resulting in no ground points, interpolation algorithms are needed to fill these holes. Commonly used interpolation methods include inverse distance weighted interpolation, Kriging interpolation, or nearest neighbor interpolation. Taking inverse distance weighted interpolation as an example, for each hollow raster cell, known elevation points within a certain radius are searched, and a weighted average elevation value is calculated based on the reciprocal of the square of the distance. After interpolation, the raster matrix is ​​smoothed using filtering methods such as median filtering to eliminate noise caused by individual outliers. The final output is a digital terrain model that fully covers the target forest area, where the value of each raster pixel represents the surface elevation at that location. The above description completes the entire cloth simulation filtering process from the original point cloud to the digital terrain model, providing a time-invariant terrain benchmark for subsequent coarse registration.

[0089] Furthermore, you can also view Figure 8 , Figure 8 This is a flowchart illustrating the third embodiment of the spectral mapping method based on multi-source point cloud collaborative registration of this application. The flowchart illustrates the flowchart framework for implementing the spectral mapping method based on multi-source point cloud collaborative registration of this application. Based on the content of the flowchart framework, the implementation of the spectral mapping method based on multi-source point cloud collaborative registration is as follows: The multi-source point cloud collaborative registration and spectral mapping method for forest scenes based on deep feature matching provided in this embodiment strictly follows the data processing logic shown in the flowchart. It constructs a complete technical chain consisting of data acquisition and preprocessing, coarse-to-fine two-level registration based on 2.5D projection, and anti-occlusion spectral mapping based on visibility analysis.

[0090] In the data acquisition and preprocessing stage, based on the phenological characteristics of the entire growth cycle of trees in the study area, a multi-temporal data acquisition operation was planned and implemented through air-ground collaboration. UAVs equipped with multispectral sensors and RGB cameras acquired canopy spectral images and visible light texture images with preset high forward and lateral overlap rates; ground-based lidar simultaneously acquired the three-dimensional point cloud structure of the forest understory. After performing radiometric calibration on the UAV image sequences, the structure-of-motion (SOMO) algorithm was used to calculate the camera's spatial pose, reconstructing sparse and dense point clouds, thereby generating digital orthophotos and digital surface models with absolute geographic coordinates, denoted as DOM and DSM, respectively. Simultaneously, a cloth simulation filtering algorithm was applied to the raw three-dimensional point cloud data acquired by the ground-based lidar. This algorithm uses physical simulation to overlay a virtual cloth mesh onto the elevation-reversed point cloud surface to separate ground points from non-ground points, ultimately interpolating to generate a time-invariant digital terrain model (DTM) within the study area. The aforementioned DOM, DSM, and DTM data products together constitute the unified data base for subsequent registration and mapping stages.

[0091] In the coarse-to-fine registration process, a coarse registration step based on a 2.5D projection dimensionality reduction strategy is first implemented. The digital terrain model derived from the LiDAR point cloud and the digital surface model derived from UAV imagery are rasterized at a fixed spatial resolution, generating two grayscale depth images. The grayscale value of each pixel in the image linearly corresponds to the absolute elevation value of the terrain surface. The LightGlue deep map neural network is used to perform semantic-level feature extraction and matching on the two grayscale depth images. This network can learn deep geometric semantic feature representations from the global context such as terrain undulations, gully directions, and abrupt slope changes, rather than relying on local corner points or gradient changes, thus overcoming the problem of traditional feature extraction failure caused by the lack of texture in forest understory. A set of coarse registration transformation matrices describing the global translation and rotation relationship between the two coordinate systems is obtained through matching calculations. This matrix is ​​used to transform the ground-based LiDAR point cloud of the target forest area into the geospatial reference frame of the UAV digital surface model, achieving first-level spatial alignment of the two types of heterogeneous data. Based on this alignment, the elevation base of the corresponding digital terrain model in the lidar point cloud is removed and the canopy height model, denoted as CHM, is generated by projection rasterization, which only reflects the vertical height of vegetation.

[0092] The fine registration step follows immediately after the coarse registration. Due to the rasterization interpolation and terrain smoothing effects involved in the coarse registration process, slight local offset residuals still exist between the transformed point cloud and the actual canopy surface. To eliminate these biases, the LightGlue depth map neural network is invoked again to perform secondary geometric feature matching between the LiDAR canopy height model (CHM) and the UAV digital surface model (DSM). Unlike coarse registration, which focuses on the macroscopic topographic trend, the matching network in the fine registration stage emphasizes identifying high-frequency geometric features reflecting the individual morphology and spatial distribution of the canopy in both surface models, including local maxima at the treetop, the outer contour edge of the canopy, and the inflection points of the forest window boundaries. By extracting these high-frequency geometric feature points to form sub-pixel precision pairs of corresponding feature points, the final fine registration transformation matrix is ​​calculated. This matrix strictly constrains the residual biases of the point cloud in the horizontal and vertical directions, ensuring high-precision matching between the LiDAR point cloud and the UAV image-reconstructed point cloud within a unified space.

[0093] After spatial registration, the process proceeds to the spectral mapping stage based on visibility analysis. Within the unified 3D space defined by the fine registration transformation matrix, the HPR algorithm is used to classify the visibility of the lidar point cloud point cloud point by point. During algorithm execution, the point cloud coordinates are first transformed to the inversion space and a spherical inversion operation is performed. Then, the convex hull of the inverted point set is calculated. Based on the principle of geometric optics occlusion, 3D points located at the vertices of the obtained convex hull surface are identified as members of the visible set, corresponding to the canopy outer surface, forest gap surface, and exposed branches that can be directly observed under orthographic projection in the real scene. 3D points located inside the convex hull are identified as members of the occlusion set, corresponding to the inner canopy branches completely obscured by upper foliage and the submerged understory vegetation.

[0094] Based on the binary classification results of the visible and occluded sets, a differentiated spectral information assignment strategy is implemented. For the visible set point cloud, the spatial index tree structure is used to quickly retrieve the DOM pixel coordinates corresponding to each 3D point. The multispectral reflectance band values ​​and RGB true color values ​​are directly written into the extended attribute fields of the point cloud. Neighborhood interpolation compensation is performed on areas of overexposure and texture loss caused by LiDAR at specular reflections on leaf surfaces or water surfaces to ensure the continuity and integrity of the canopy surface spectral information. For the occluded set point cloud, the pixel value writing operation is actively blocked, and only the original LiDAR intensity echo information or ground-based radar lateral scan texture is retained, thereby strictly blocking the erroneous mapping path of upper canopy color penetrating downwards and being mistakenly pasted onto the understory structure. Thus, the fusion result obtained in this embodiment is a geometrically accurate and spectrally physically correct multi-source fused point cloud of forest trees, providing a reliable data carrier for subsequent single-tree segmentation, biomass estimation, and tree species classification.

[0095] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the spectral mapping method based on multi-source point cloud collaborative registration in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0096] This application provides a spectral mapping device based on multi-source point cloud collaborative registration. The spectral mapping device based on multi-source point cloud collaborative registration includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the spectral mapping method based on multi-source point cloud collaborative registration in the above embodiment 1.

[0097] The following is for reference. Figure 9This document illustrates a structural schematic of a spectral mapping device suitable for implementing embodiments of this application based on multi-source point cloud collaborative registration. The spectral mapping device based on multi-source point cloud collaborative registration in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The spectral mapping device based on multi-source point cloud collaborative registration shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0098] like Figure 9 As shown, the spectral mapping device based on multi-source point cloud collaborative registration may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the spectral mapping device based on multi-source point cloud collaborative registration. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the spectral mapping device based on multi-source point cloud co-registration to exchange data with other devices wirelessly or via wired communication. Although the figure shows a spectral mapping device based on multi-source point cloud co-registration with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems can be implemented alternatively.

[0099] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0100] The spectral mapping device based on multi-source point cloud collaborative registration provided in this application employs the spectral mapping method based on multi-source point cloud collaborative registration in the above embodiments. This addresses the technical problems of low registration accuracy caused by dimensional differences and non-rigid deformation in existing multi-source heterogeneous forest data, and spectral smearing and texture distortion caused by neglecting three-dimensional spatial occlusion relationships in traditional mapping methods. Compared with the prior art, the beneficial effects of the spectral mapping device based on multi-source point cloud collaborative registration provided in this application are the same as those of the spectral mapping method based on multi-source point cloud collaborative registration provided in the above embodiments. Furthermore, other technical features of this spectral mapping device based on multi-source point cloud collaborative registration are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0101] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0103] This application provides a storage medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the spectral mapping method based on multi-source point cloud collaborative registration in the above embodiments.

[0104] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0105] The aforementioned computer-readable storage medium may be included in a spectral mapping device based on multi-source point cloud co-registration; or it may exist independently and not assembled into a spectral mapping device based on multi-source point cloud co-registration.

[0106] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the spectral mapping device based on multi-source point cloud collaborative registration, the spectral mapping device based on multi-source point cloud collaborative registration implements the technical content of the spectral mapping method embodiment based on multi-source point cloud collaborative registration as shown above.

[0107] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0110] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned spectral mapping method based on multi-source point cloud collaborative registration. This solves the technical problems of low registration accuracy caused by dimensional differences and non-rigid deformation in existing multi-source heterogeneous forest data, and spectral smearing and texture distortion caused by neglecting three-dimensional spatial occlusion relationships in traditional mapping methods. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the spectral mapping method based on multi-source point cloud collaborative registration provided in the above embodiments, and will not be repeated here.

Claims

1. A spectral mapping method based on multi-source point cloud collaborative registration, characterized in that, The spectral mapping method based on multi-source point cloud collaborative registration includes the following steps: A deep graph neural network is used to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and match them to obtain a coarse registration transformation matrix. Based on the coarse registration transformation matrix, the ground-based lidar point cloud of the target forest area is spatially aligned with the space where the UAV digital surface model is located to generate a canopy height model; The deep graph neural network is used to perform secondary geometric feature matching between the canopy height model and the UAV digital surface model. High-frequency geometric feature point pairs of the canopy are identified in the matching results, and the fine registration transformation matrix is ​​calculated based on the identification results. In the unified space of the fine registration transformation matrix, the HPR algorithm is used to perform visibility analysis on the point cloud, and the point cloud is divided into a visible set and an occlusion set according to the analysis results. The spectral information of the digital orthophoto of the target forest area is mapped onto the visible set, and the spectral mapping of the occlusion set is masked to obtain the spectral mapping result of the target forest area.

2. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 1, characterized in that, Before the step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain the coarse registration transformation matrix, the method further includes: Acquire multi-temporal observation data of the target forest area, wherein the multi-temporal observation data is a dataset corresponding to the phenological characteristics of trees in the target forest area and containing canopy spectral information and texture information; Radiometric calibration and motion recovery structure calculations are performed on the canopy spectral and texture information to generate digital orthophotos, UAV digital surface models, and initial point clouds. A ground-based lidar point cloud is used to acquire three-dimensional structural information of the forest understory. The ground-based lidar point cloud is then subjected to a cloth simulation filtering algorithm to decouple the terrain and vegetation features, generating a digital terrain model.

3. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 2, characterized in that, The step of using a cloth simulation filtering algorithm to decouple terrain and vegetation features from the ground-based lidar point cloud to generate a digital terrain model includes: The elevation values ​​of the ground-based lidar point cloud are reversed, and the reversed ground-based lidar point cloud is used as the contact surface for cloth simulation. A virtual cloth grid is initialized above the inverted ground-based lidar point cloud, and the virtual cloth grid is controlled to sink downward under the action of gravity to contact the surface of the ground-based lidar point cloud. During the settling process of the virtual cloth grid, the equilibrium position of the grid node is determined based on the collision constraint between the grid node and the reversed surface of the ground-based lidar point cloud, and the distance between the equilibrium grid node and the corresponding point cloud in the corresponding projection neighborhood is calculated. The category of the point cloud is determined according to the preset distance threshold, and the category includes terrain ground points or vegetation non-ground points. The point cloud that is identified as terrain ground points is collected, and vegetation non-ground points are removed to form a discrete set of terrain points; The discrete set of terrain points is subjected to rasterization interpolation to generate the digital terrain model.

4. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 1, characterized in that, The step of using a deep graph neural network to extract deep geometric semantic features from the grayscale depth maps of the digital terrain model and the UAV digital surface model of the target forest area and matching them to obtain a coarse registration transformation matrix includes: The LightGlue depth map neural network is used to perform global semantic analysis on the terrain undulation, gully direction and slope change in the grayscale depth map. The analysis results are used to extract terrain feature point pairs, calculate global translation and rotation parameters based on the terrain feature point pairs, and obtain the coarse registration transformation matrix based on the calculation results.

5. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 1, characterized in that, The step of spatially aligning the ground-based lidar point cloud of the target forest area with the space of the UAV digital surface model based on the coarse registration transformation matrix to generate a canopy height model includes: The ground-based lidar point cloud is transformed to the same coordinate system as the UAV digital surface model using a coarse registration transformation matrix. After removing the terrain elevation represented by the digital terrain model from the point cloud in the coordinate system, the vegetation height of each point cloud relative to the ground surface is obtained, forming a normalized point cloud. The normalized point cloud is projected onto a horizontal plane and rasterized, and the maximum vegetation height within each raster cell is extracted to generate the canopy height model.

6. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 1, characterized in that, In the unified space of the fine registration transformation matrix, the HPR algorithm is used to perform visibility analysis on the point cloud. Based on the analysis results, the point cloud is divided into a visible set and an occlusion set. This includes the following steps: The point cloud is inverted by HPR transformation to obtain a flipped point set, and the convex hull of the flipped point set is determined. The point cloud located at the vertices of the convex hull is marked as the visible set, and the point cloud located inside the convex hull is marked as the occlusion set. The visible set represents the canopy surface and forest window ground, and the occlusion set represents the branches inside the canopy and the occluded ground.

7. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 1, characterized in that, The step of mapping the spectral information of the digital orthophoto of the target forest area onto the visible set, and then masking the spectral mapping of the occlusion set to obtain the spectral mapping result of the target forest area includes: Expand the multispectral and true color attribute fields into the original 3D point cloud data structure; The multispectral reflectance and true-color pixel values ​​in the digital orthophoto are extracted using spatial indexing technology, and the true-color pixel values ​​are written point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance. For the point cloud of the occlusion set, the original attribute structure and original laser intensity field of the point cloud of the occlusion set are kept unchanged, and the mapping result and the masking result are combined into the spectral mapping result.

8. The spectral mapping method based on multi-source point cloud collaborative registration as described in claim 7, characterized in that, The step of extracting multispectral reflectance and true-color pixel values ​​from the digital orthophoto using spatial indexing technology, and writing the true-color pixel values ​​point by point into the corresponding extended attribute field of the point cloud of the visual set based on the multispectral reflectance, includes: Traverse the visible point cloud, detect the original laser echo intensity value of each point, and mark the points whose intensity value is greater than the preset intensity saturation threshold as overexposed points; The true color data of the corresponding pixel positions of each overexposed point in the digital orthophoto are extracted using spatial indexing. The extracted true-color data is written into the attribute field of the overexposed point to repair the overexposure loss, and color consistency smoothing is performed on the overexposed point and its spatial neighborhood point cloud.

9. A spectral mapping device based on multi-source point cloud collaborative registration, characterized in that, The spectral mapping device based on multi-source point cloud collaborative registration stores a computer program, which, when executed by a processor, implements the spectral mapping method based on multi-source point cloud collaborative registration as described in any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the spectral mapping method based on multi-source point cloud collaborative registration as described in any one of claims 1-8.

Images