Urban digital twin modeling method and system based on multi-source heterogeneous data fusion

Abstract

The application provides a city digital twin modeling method and system based on multi-source heterogeneous data fusion, relates to the technical field of digital twin, and the method acquires registered images and registered point clouds of a target area, respectively extracts image boundary candidates and point cloud boundary candidates, generates boundary uncertainty bands for each boundary candidate, determines uncertainty band width and bias direction identifier; form boundary matching pairs according to spatial intersection relationship, and determine the intersection sections with opposite bias direction identifiers as the set of opposite boundary sections; for each opposite boundary section, determine the image confidence weight and the point cloud confidence weight, and generate a boundary decision result accordingly; perform topological consistency verification on the boundary decision result to determine the unique fused boundary. The method can improve the accuracy and topological reliability of geometric edge fusion and enhance the stability and usability of the city digital twin model in the case of local physical characteristic differences and systematic geometric deviations in multi-source data.

Description

Technical Field

[0001] This application relates to the field of digital twin technology, and more specifically, to a method and system for urban digital twin modeling based on the fusion of multi-source heterogeneous data. Background Technology

[0002] Current urban digital twin modeling typically relies on the texture details of aerial imagery and the spatial geometry of point clouds. Registration, feature extraction, and fusion reconstruction are used to create 3D models suitable for visualization and analysis. However, in high-density built environments, image imaging and laser ranging are significantly affected by acquisition mechanisms and media interactions, often resulting in observational biases due to differences in physical properties in local areas. For example, image edges may exhibit edge expansion or texture artifacts in areas of strong reflection and high brightness, while point cloud edges may show edge contraction, discretization, or depth splitting under grazing incidence, edge blending, or multi-echo scenarios. These differences often manifest as systematic geometric deviations, concentrating near boundaries, making it difficult to maintain consistency between the same structural edges in images and point clouds.

[0003] In existing technologies, methods such as nearest neighbor matching, threshold intersection, fixed weighting, or global optimization are commonly used to determine the fusion boundary for cross-source edge alignment and fusion. Some schemes further improve the appearance continuity after fusion through smoothing and resampling. However, when there are conflicting deviations in local multi-source heterogeneous data, the above methods are prone to topological problems such as mismatch, ghosting, breaks, or self-intersections, leading to instability of the fused edge in subsequent mesh reconstruction, texture mapping, or structured modeling, thus affecting the usability and reliability of the model. Therefore, there is an urgent need for a technical solution that can improve the accuracy and topological reliability of geometric edge fusion in urban digital twin modeling when there are local differences in physical properties and systematic geometric deviations in multi-source heterogeneous data. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application provides a method and system for urban digital twin modeling based on the fusion of multi-source heterogeneous data.

[0005] Firstly, this application provides a method for urban digital twin modeling based on multi-source heterogeneous data fusion, including:

[0006] Acquire the registration image and registration point cloud corresponding to the target area;

[0007] Based on the registered images, image boundary candidates are extracted, image uncertainty bands are generated for each image boundary candidate, and the width of the image uncertainty band and the image offset direction identifier are determined; based on the registered point cloud, point cloud boundary candidates are extracted, point cloud uncertainty bands are generated for each point cloud boundary candidate, and the width of the point cloud uncertainty band and the point cloud offset direction identifier are determined.

[0008] Boundary matching pairs are formed based on the spatial intersection relationship between the image uncertainty zone and the point cloud uncertainty zone, and the intersection segments with opposite offset directions are identified as the set of adversarial boundary segments.

[0009] For each adversarial boundary segment in the set of adversarial boundary segments, the image confidence weight and the point cloud confidence weight are determined based on the corresponding image uncertainty band width and point cloud uncertainty band width.

[0010] Based on the image confidence weight and the point cloud confidence weight, a boundary decision result is generated for the image boundary candidate and the point cloud boundary candidate corresponding to the adversarial boundary segment. A topology consistency check is performed on the boundary decision result. If the preset topology constraint conditions are met, the boundary decision result that passes the topology consistency check is determined as the unique fusion boundary corresponding to the adversarial boundary segment.

[0011] Optionally, the determination of the image uncertainty band width and image offset direction identifier includes:

[0012] Obtain the solar elevation angle and solar azimuth angle corresponding to the registered image, and obtain the camera optical axis direction vector corresponding to the registered image; determine the local normal vector of the candidate surface corresponding to the image boundary based on the registered point cloud;

[0013] Based on the solar elevation angle and solar azimuth angle, the camera optical axis direction vector, and the local normal vector, calculate the geometric relationship parameters between the incident light direction and the observation direction; obtain the specular reflection coefficient associated with the image boundary candidate;

[0014] The geometric relationship parameters and the specular reflection coefficient are input into a preset specular expansion mapping rule to calculate the boundary expansion amount, and the image uncertainty band width is determined based on the boundary expansion amount; the specular reflection direction is determined based on the geometric relationship parameters, and the normal side away from the specular reflection direction is determined as the image offset direction identifier.

[0015] Optionally, determining the image uncertainty band width further includes:

[0016] High-frequency component extraction is performed on the neighboring images of the image boundary candidate to obtain high-frequency texture components; virtual image edge scores are generated based on the energy distribution differences of the high-frequency texture components on both sides of the image boundary candidate in the normal direction.

[0017] If the virtual image edge score meets the preset judgment conditions, the image uncertainty band width is extended and corrected based on the virtual image edge score to obtain the corrected image uncertainty band width; and the direction along the candidate normal of the image boundary, pointing to the side with lower high-frequency texture energy, is determined as the corrected image offset direction identifier.

[0018] Optionally, obtaining the specular reflection coefficient associated with the image boundary candidate includes:

[0019] Obtain a building information model that is spatiotemporally aligned with the target area; parse the semantic information data of the components in the building information model, and establish a mapping relationship between the image boundary candidates and the building components in the building information model through spatial indexing;

[0020] Read the material property information of the building component, and query the preset physical material library based on the material property information to extract the corresponding specular reflection coefficient.

[0021] Optionally, the determination of the uncertain band width and the point cloud offset direction identifier includes:

[0022] The laser acquisition parameters associated with the points in the registration point cloud are obtained. The laser acquisition parameters include the ranging value, the incident beam direction information, the echo identifier, and the divergence angle parameter.

[0023] For each point cloud boundary candidate, the incident geometry relationship is determined based on the incident beam direction information and the local normal of the point cloud boundary candidate, and the laser footprint scale is determined based on the ranging value and the divergence angle parameter.

[0024] The width of the uncertain band of the point cloud is determined based on the incident geometry and the laser footprint scale; and the offset direction marker of the point cloud is determined based on the distribution difference of the echo marker on both sides of the candidate normal of the point cloud boundary.

[0025] Optionally, determining the uncertain band width of the point cloud further includes:

[0026] When the laser acquisition parameters include multi-echo ranging data of the same incident beam, the difference between the first echo ranging value and the last echo ranging value is calculated for the candidate neighborhood of the point cloud boundary.

[0027] When the difference meets the preset difference judgment condition, the width of the uncertain band of the point cloud is increased according to the preset expansion rule; and based on the distribution of the number of points on both sides of the candidate normal of the point cloud boundary between the first echo point set and the last echo point set, the normal direction corresponding to the side with the higher number of points is determined as the point cloud offset direction identifier.

[0028] Optionally, forming boundary matching pairs based on the spatial intersection relationship between the image uncertainty zone and the point cloud uncertainty zone includes:

[0029] In three-dimensional space, an image-side search region corresponding to the image uncertainty zone and a point cloud-side search region corresponding to the point cloud uncertainty zone are constructed respectively.

[0030] When the image-side search region and the point cloud-side search region meet the preset spatial intersection condition, the corresponding image boundary candidate and point cloud boundary candidate are determined as candidate matching pairs.

[0031] The candidate matching pairs are subjected to visibility discrimination to filter out candidate matching pairs that do not meet the preset visibility conditions, and boundary matching pairs are obtained.

[0032] Optionally, the visibility determination includes:

[0033] Based on the camera parameters of the registered image, the pixel positions within the uncertain band of the image are back-projected into a set of line-of-sight rays, and the image-side search region is formed based on the set of line-of-sight rays.

[0034] Using the spatial trajectory of the point cloud boundary candidate as the center line, and determining the cross-sectional scale based on the width of the uncertain zone of the point cloud, the search area on the side of the point cloud is formed;

[0035] For each candidate matching pair, occlusion point cloud located between the camera position and the search area on the side of the point cloud is retrieved in the registration point cloud along the line of sight ray set, and a visibility score is generated based on the occlusion point cloud; wherein, the occlusion point cloud is preferably composed of the first echo point set and / or the echo point set with smaller ranging value;

[0036] When the visibility score meets the preset visibility condition, the candidate matching pair is retained as the boundary matching pair; when there are multiple boundary matching pairs corresponding to the same image boundary candidate, the boundary matching pair with the best visibility score is selected as the final boundary matching pair.

[0037] Optionally, determining the image confidence weight and point cloud confidence weight by determining the bandwidth includes:

[0038] Along the normal direction of the adversarial boundary segment, extract the brightness profile sequence that crosses the image boundary in the registered image, and extract the brightness overshoot and trailing asymmetry from the brightness profile sequence to generate the image artifact index.

[0039] The echo distribution features of the neighborhood of the adversarial boundary segment are extracted from the registered point cloud, and the point cloud artifact index is generated based on the difference between the first echo ranging value and the last echo ranging value and / or the dispersion of the echo point set on both sides of the normal.

[0040] Based on the image artifact index and the point cloud artifact index, the image uncertainty band width and the point cloud uncertainty band width are adjusted respectively through a preset penalty mapping rule to obtain the adjusted image bandwidth and the adjusted point cloud bandwidth.

[0041] The image confidence weight and the point cloud confidence weight are calculated based on the adjusted image bandwidth and the adjusted point cloud bandwidth, and normalization processing is performed on the image confidence weight and the point cloud confidence weight.

[0042] Secondly, this application provides a city digital twin modeling system based on multi-source heterogeneous data fusion, including:

[0043] The acquisition module is used to acquire the registration image and registration point cloud corresponding to the target area;

[0044] The generation module is used to extract image boundary candidates based on the registered image, generate image uncertainty bands for each image boundary candidate, and determine the width of the image uncertainty band and the image offset direction identifier; and to extract point cloud boundary candidates based on the registered point cloud, generate point cloud uncertainty bands for each point cloud boundary candidate, and determine the width of the point cloud uncertainty band and the point cloud offset direction identifier.

[0045] The processing module is used to form boundary matching pairs based on the spatial intersection relationship between the image uncertainty band and the point cloud uncertainty band, and to determine the intersection segments with opposite offset directions as a set of adversarial boundary segments; for each adversarial boundary segment in the set of adversarial boundary segments, the image confidence weight and the point cloud confidence weight are determined based on the corresponding image uncertainty band width and the point cloud uncertainty band width.

[0046] The verification module is used to generate boundary adjudication results for the image boundary candidates and point cloud boundary candidates corresponding to the adversarial boundary segment based on the image confidence weight and point cloud confidence weight, and to perform topology consistency verification on the boundary adjudication results. Under the condition of satisfying the preset topology constraints, the boundary adjudication results that pass the topology consistency verification are determined as the unique fusion boundary corresponding to the adversarial boundary segment.

[0047] Compared with existing technologies, this application constructs boundary candidates and their uncertainty bands on both the image and point cloud sides, and explicitly characterizes the width and offset direction of the uncertainty bands. This transforms cross-source edge differences from "inexplicable errors" into measurable and discriminable descriptions of local uncertainty, thus providing a stable decision basis for subsequent fusion. Furthermore, this application utilizes the spatial intersection relationship of uncertainty bands to form boundary matching and identifies adversarial boundary segments with opposite offset directions. This allows the fusion process to focus on the most significant local conflict areas, avoiding the spread of local adversarial errors into global edge drift. Based on the confidence weights obtained from the image and point cloud uncertainty band widths, this application can adaptively adjust the confidence of the two types of data sources within the adversarial segment, allowing the boundary adjudication results to adjust with changes in local observation reliability, reducing the dominant influence of artifacts from a single data source on the fusion edge. Meanwhile, this application introduces topology consistency verification before outputting the fusion boundary. Under the condition of satisfying the preset topology constraints, a unique fusion boundary is determined, so that the fusion result is more in line with the requirements of subsequent modeling for connectivity, non-self-intersection and structural stability. This improves the accuracy and topology reliability of geometric edge fusion, and enhances the usability and consistency of the city digital twin model as an engineering-level data foundation. Attached Figure Description

[0048] Figure 1 A flowchart illustrating the urban digital twin modeling method based on multi-source heterogeneous data fusion provided in this application embodiment;

[0049] Figure 2 A flowchart illustrating a method for determining the width of an image uncertainty band and an image offset direction identifier, provided in an embodiment of this application;

[0050] Figure 3 A flowchart illustrating a method for determining the width of an uncertain band in a point cloud and identifying the offset direction of the point cloud, provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram of a city digital twin modeling system based on multi-source heterogeneous data fusion provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0053] See Figure 1 The flowchart shown is a process for a city digital twin modeling method based on multi-source heterogeneous data fusion provided in an embodiment of this application, including steps S101 to S105, wherein:

[0054] S101: Obtain the registration image and registration point cloud corresponding to the target area;

[0055] S102: Based on the registered image, extract image boundary candidates, generate image uncertainty bands for each image boundary candidate, and determine the width of the image uncertainty band and the image offset direction identifier; based on the registered point cloud, extract point cloud boundary candidates, generate point cloud uncertainty bands for each point cloud boundary candidate, and determine the width of the point cloud uncertainty band and the point cloud offset direction identifier.

[0056] S103: Based on the spatial intersection relationship between the image uncertainty zone and the point cloud uncertainty zone, a boundary matching pair is formed, and the intersection segments with opposite offset directions are identified as the set of adversarial boundary segments.

[0057] S104: For each adversarial boundary segment in the set of adversarial boundary segments, determine the image confidence weight and the point cloud confidence weight based on the corresponding image uncertainty band width and point cloud uncertainty band width;

[0058] S105: Based on the image confidence weight and the point cloud confidence weight, generate a boundary decision result for the image boundary candidate and the point cloud boundary candidate corresponding to the adversarial boundary segment, and perform a topology consistency check on the boundary decision result. If the preset topology constraint conditions are met, the boundary decision result that passes the topology consistency check is determined as the unique fusion boundary corresponding to the adversarial boundary segment.

[0059] Regarding the above S101:

[0060] In practical implementation, S101 is used in urban digital twin modeling to generate image data and point cloud data for a target area under the same spatiotemporal reference system, so that geometric feature extraction and fusion processing can be carried out on a unified coordinate reference. Registration here can be understood as follows: image frames have their corresponding camera imaging parameters and camera pose parameters, and point cloud data have their corresponding point cloud coordinate system transformation relationship to a unified coordinate system, thereby enabling the image and point cloud to be aligned and mutually indexed within the same three-dimensional spatial reference frame.

[0061] For example, the target area can be an urban block, a densely populated CBD building area, or a road corridor area, and the unified coordinate system can be any one of the geographic coordinate system, engineering coordinate system, or local northeast-sky coordinate system. The data acquisition carrier can be an unmanned aerial vehicle platform, a vehicle-mounted mobile measurement platform, or a ground tripod measurement platform. The acquisition sensors include one of imaging equipment and lidar, and may optionally include an inertial measurement unit (IMU), a satellite positioning unit (GNSS), an odometer, or a gimbal encoder to provide time synchronization and pose prior knowledge.

[0062] For example, acquiring the registered image may include: continuously acquiring multiple frames of images within a preset sampling period, and writing a timestamp and camera parameters to each frame of image; wherein, the camera parameters include at least one of camera intrinsic parameters, distortion parameters, and camera exposure-related parameters. Acquiring the registered point cloud may include: acquiring a point cloud stream or point cloud frames within a time range overlapping with the image acquisition time window, and writing at least a portion of ranging values, incident beam direction information, echo identifiers, divergence angle parameters, intensity values, and timestamps to the points in the point cloud; wherein, the echo identifiers are used to characterize different echo sequences or echo types of the same incident beam.

[0063] In an optional implementation, S101 further includes time synchronization and delay compensation processing to improve the temporal consistency between the image and the point cloud. For example, the timestamps of each sensor can be unified to the same clock reference, and the timestamps of image frames and point clouds can be aligned to the same sampling period. When there is a fixed link delay or exposure readout delay, the image timestamps or point cloud timestamps can be compensated and corrected based on a preset delay amount. As an example, the time alignment error threshold can be set to a range of 5ms to 50ms; image frames or point cloud segments exceeding the threshold can be marked as low-confidence samples to trigger resampling or rejection.

[0064] In an optional implementation, S101 further includes extrinsic parameter calibration and pose calculation processing for the camera and LiDAR to generate a "registration" relationship. For example, rigid body transformation parameters between the camera coordinate system and the LiDAR coordinate system can be pre-calibrated, and during the acquisition process, the pose sequence of the platform in a unified coordinate system is obtained based on GNSS / IMU / odometry fusion calculation; then, the point cloud data is transformed to the unified coordinate system based on the pose sequence, and each frame of image is associated with its camera pose in the unified coordinate system. For motion platform scenarios, motion distortion correction processing can be optionally performed on the point cloud, for example, interpolation compensation is performed on points within the same scanning period based on the IMU attitude sequence and point timestamps to reduce point cloud ghosting and structural distortion.

[0065] In an optional implementation, to improve registration accuracy, S101 may further include registration residual optimization processing based on overlapping observations. For example, the pose sequence may be optimized based on the scanning matching results between point cloud frames, and / or the camera pose may be optimized based on image feature matching and reprojection errors, and / or the overall alignment error in a unified coordinate system may be eliminated based on known geometric constraints of a small number of control points, a known reference map, or a building model. As an example, the point cloud registration residual threshold may be set to 0.02m to 0.20m, and the image reprojection error threshold may be set to 1px to 3px; when the residual exceeds the threshold, rematching, increasing the sampling overlap rate, or reverting to the pose prior may be triggered.

[0066] For example, the output data of S101 can be written into a preset data structure by an automated program for direct use in subsequent steps; the data structure includes: image frame index, image pixel data, image timestamp, image corresponding camera intrinsic parameters and camera pose parameters; point cloud frame index or point cloud block index, point cloud point coordinate data, point cloud point attribute fields and transformation relationship parameters from point cloud to unified coordinate system.

[0067] Regarding S102 above:

[0068] In practical implementation, S102 is used to extract candidate boundaries that can be used to represent geometric contours in both the image domain and the point cloud domain, provided that a unified reference coordinate relationship has been established. For each candidate boundary, an associated uncertainty band, band width, and offset direction identifier are generated. This ensures that subsequent steps can uniformly represent multi-source boundaries in terms of both the possible location range and offset tendency. The candidate boundaries, uncertainty bands, band widths, and offset direction identifiers can all be generated and maintained as structured data objects by an automated program, such as being stored in memory or a file in the form of a record of "candidate number—geometric representation—uncertainty band parameter—offset identifier".

[0069] For example, the extraction of image boundary candidates may include: performing preprocessing such as denoising and scale unification on the registered image, generating an edge response map, and aggregating the connected edge pixels in the edge response map into several ordered boundary curves or line segment sets as image boundary candidates. The edge response map can be generated by any of the gradient operator, structure tensor, or multi-scale edge operator; the candidate aggregation can be implemented by any of the connected component tracking, line segment detection, or curve fitting.

[0070] For example, in a Python environment, interfaces such as `skimage.feature.canny` and `skimage.transform.probabilistic_hough_line` from scikit-image can be used to achieve this. To improve candidate stability, candidates can be filtered and resampled based on constraints such as length, continuity, edge response intensity, or curvature smoothness, so that each image boundary candidate has a clear geometric representation, such as a sequence of pixel coordinates or a parametric curve.

[0071] In generating image uncertainty bands, for each image boundary candidate, the normal direction can be determined based on its local tangential direction, and a band-shaped region can be constructed along this normal direction in the image domain as the image uncertainty band. The image uncertainty band can be expressed in the form of a set of pixels or in the form of parameters, such as a band-shaped region model with the boundary curve as the center line and the band width parameter as the expansion scale. The width of the image uncertainty band can be automatically determined by the program based on the "local uncertainty metric" of the candidate boundary. The uncertainty metric can come from one or a combination of the spatial diffusion degree of the edge response, the local noise level, the degree of blurring, the stability statistics of edge localization, etc. The image offset direction identifier can be used to indicate the offset tendency of the boundary candidate on both sides of its normal direction. The value of the offset direction identifier can be determined by comparing the differences in edge support, texture consistency, or response integral intensity on both sides of the normal direction. The offset direction identifier can be represented by a binary / signed value (e.g., +1 / -1) or an enumerated value for consistency judgment in subsequent steps.

[0072] For example, the extraction of point cloud boundary candidates may include: performing preprocessing such as downsampling and outlier suppression on the registered point cloud, estimating the local normal and curvature information of the point cloud, and identifying boundary point sets in regions with discontinuous normals, abrupt curvature changes, abrupt point density changes, or abrupt changes in fitting residuals; subsequently, performing clustering and trajectory fitting on the boundary point sets to obtain one or more point cloud boundary candidates. Point cloud boundary candidates can be represented by three-dimensional point sequences, polyline / spline parameters, or a "centerline + neighborhood set" approach.

[0073] For example, in a Python / C++ environment, Open3D's `estimate_normals`, `compute_nearest_neighbor_distance`, `cluster_dbscan` and other interfaces can be called to achieve candidate extraction.

[0074] In generating point cloud uncertainty bands, for each point cloud boundary candidate, a three-dimensional strip / tubular region can be constructed along the local normal direction, using its spatial trajectory as the center reference. This uncertainty band can be explicitly represented as a three-dimensional voxel / mesh occupancy set, or parametrically represented as "centerline + cross-sectional scale". The width of the point cloud uncertainty band can be automatically determined by the program based on the local uncertainty metric of the point cloud. The uncertainty metric can originate from one or a combination of point cloud noise level, local sparsity, normal estimation stability, and the discreteness of boundary points. The point cloud offset direction identifier is used to indicate the offset tendency of the point cloud boundary candidate on both sides of the normal direction. The value of the offset direction identifier can be determined by comparing information such as the proportion of points on both sides of the normal direction, the difference in point density, or the difference in fitting residuals.

[0075] For example, the output of S102 can be written into a unified data structure for direct reference in subsequent steps. The data structure includes: a candidate set of image boundaries and its corresponding image uncertainty band representation, image uncertainty band width and image offset direction identifier; a candidate set of point cloud boundaries and its corresponding point cloud uncertainty band representation, point cloud uncertainty band width and point cloud offset direction identifier; and candidate number and index information for tracing.

[0076] Regarding the above S103:

[0077] In specific implementation, S103 is used to establish the spatial correspondence between two types of boundary candidates, based on the "boundary candidate - uncertainty band - bandwidth - offset direction identifier" already generated on both the image and point cloud sides. Its core is: using the uncertainty band as the allowed geometric location range, potential candidate matching pairs are filtered out through spatial intersection; and within the intersection area of ​​the matching pairs, the relative relationship of the offset direction identifiers is further used to identify "adversarial boundary segments," so that subsequent steps can perform differentiated processing on these segments. The input to S103 can include the image boundary candidate set and its image uncertainty band and image offset direction identifier, the point cloud boundary candidate set and its point cloud uncertainty band and point cloud offset direction identifier, and the registration relationship obtained from S101 (used to ensure that the two types of data satisfy the same spatial reference).

[0078] For example, to achieve executable discrimination of spatial intersection relationships, geometric representation and operations can be performed on two types of uncertain bands in a unified discrimination space. The unified discrimination space can be a three-dimensional space, an image plane space, or another equivalent space, depending on the system's data organization and computing power configuration.

[0079] Firstly, when implemented in three-dimensional space, image uncertainty zones can be elevated to three-dimensional image-side search areas. For example, back-projection can be performed on the pixel positions within the image uncertainty zones based on camera parameters to form a set of spatial rays. These rays can then be assigned depth ranges by combining registered point clouds or depth / elevation auxiliary data, thereby constructing three-dimensional strip / wedge-shaped regions. Meanwhile, point cloud uncertainty zones can be directly represented as three-dimensional strip / tubular regions.

[0080] Secondly, when implementing on the image plane, the uncertain zone of the point cloud can be projected onto the corresponding pixel coordinate system of the registered image through the camera, forming a point cloud projection strip region, and overlapping discrimination is performed with the uncertain zone of the image on the image plane; if necessary, a depth consistency tolerance can also be added to suppress pseudo-intersections caused by projection ambiguity.

[0081] In terms of forming boundary matching pairs, candidate retrieval structures can be constructed first to reduce combinatorial explosion.

[0082] For example, a spatial index (e.g., R-tree, octree, or voxel hash table) can be established for the bounding boxes or voxel occupancy sets of uncertain bands on the point cloud side. For each uncertain band on the image side, potential spatially overlapping uncertain bands on the point cloud side are retrieved to obtain candidate combinations. For each candidate combination, a preset spatial intersection criterion is calculated to confirm whether it constitutes a candidate matching pair. The spatial intersection criterion can be any one or a combination of the following: overlap volume / overlap area, whether the minimum distance is less than a threshold, overlap length ratio, overlap point ratio, or overlap score based on the distance field. The threshold can be given by system parameters or adaptively set according to the band width. For example, in a C++ / Python environment, the retrieval structure can be established using Open3D's `VoxelGrid`, `KDTreeFlann`, or PCL's `OctreePointCloudSearch`, `KdTreeFLANN`.

[0083] In determining intersecting segments, for confirmed candidate matching pairs, “segment-level” correspondences can be further extracted from the overlapping areas of uncertain zones on both sides.

[0084] For example, the arc length parameters of the overlapping regions along the boundary candidates can be projected and aggregated to obtain one or more continuous overlapping sub-intervals. Each continuous sub-interval corresponds to an intersecting segment, and its parameter range on the image boundary candidate and the parameter range on the point cloud boundary candidate can be recorded. To ensure the stability of the segments, minimum length constraints, gap merging constraints, or break segmentation constraints can be applied to the overlapping sub-intervals to avoid misidentifying scattered noise overlaps as valid segments. The output of the intersecting segments can be in a structured record format, including fields such as "image boundary candidate identifier, point cloud boundary candidate identifier, segment endpoint parameters / endpoint coordinates, and overlap determination quantity".

[0085] In determining the set of adversarial boundary segments, the associated image offset direction identifier and point cloud offset direction identifier can be read for each intersecting segment, and their relative relationship can be used as the basis for adversarial discrimination. For example, when the offset directions represented by the two are opposite under a unified normal reference, the intersecting segment is marked as an adversarial boundary segment and written into the set of adversarial boundary segments; if the offset direction identifier is symbolic or enumerated, the discrimination can be achieved through symbol reversal, enumeration of conflict tables, or a preset consistency discrimination function.

[0086] Regarding S104 above:

[0087] In practical implementation, S104 is used to convert the comparable error dimension of the uncertainty band width into weight parameters that can be used for subsequent fusion adjudication. Intuitively, the wider the uncertainty band, the more uncertain the boundary position of the data source in that segment, and the smaller its confidence weight should be; conversely, the narrower the uncertainty band, the larger its confidence weight should be. The inputs of S104 include: a set of adversarial boundary segments, and the band width information of the image boundary candidates and point cloud boundary candidates associated with each adversarial boundary segment; the output of S104 is: a set of image confidence weights and point cloud confidence weights for each adversarial boundary segment, which are used to characterize the relative credibility of the two data sources in that adversarial boundary segment.

[0088] For example, the method for determining the segment-level bandwidth can be determined first. Since the uncertain band width can vary along the arc length direction of the boundary candidate, for each adversarial boundary segment, several sampling positions can be selected within the parameter range of that segment. The image uncertain band width and point cloud uncertain band width corresponding to the sampling positions are read respectively, and the sampling results are aggregated to obtain the segment-level bandwidth. The aggregation can be any of the mean, median, truncated mean, or maximum value to enhance robustness to local outliers; the number of sampling points can be given by a preset configuration or adaptively set according to the segment length. The data structure of the adversarial boundary segment can at least include: segment identifier, image boundary candidate identifier, point cloud boundary candidate identifier, segment endpoint parameter range, and segment-level image bandwidth and segment-level point cloud bandwidth fields for the segment, which can be directly called in subsequent steps.

[0089] After obtaining the segment-level image bandwidth and segment-level point cloud bandwidth, the bandwidth can be converted into unnormalized confidence scores using a preset confidence mapping rule, and then confidence weights can be obtained. The confidence mapping rule can satisfy the monotonicity constraint that "the score decreases as the bandwidth increases," and can be implemented using table lookup or function calculation. For example, the unnormalized confidence score can be calculated using an inverse proportional rule, such as generating the score as the reciprocal of the bandwidth and adding a very small positive number to avoid division by zero, or it can be calculated using an exponential decay rule, such as generating the score in an exponential decay form with the bandwidth as the independent variable. Subsequently, normalization processing can be performed on the image score and the point cloud score so that their sum satisfies a preset constraint, such as a sum of 1, thereby obtaining the image confidence weight and the point cloud confidence weight. To avoid the weights becoming extreme and affecting the stability of subsequent solutions, upper and lower limits can be set for the weights and normalized again after truncation. The upper and lower limits can be stored as configurable parameters.

[0090] The sampling, aggregation, mapping, and normalization described above can be performed by automated programs on CPUs or GPUs. For example, in a Python environment, `numpy` can be used to perform aggregation and normalization calculations on the sampled array; when a large number of adversarial boundary segments need to be processed in batches, a parallel computing framework can be used to traverse and update the segments in parallel.

[0091] Regarding the above S105:

[0092] In practical implementation, S105 is used to complete two tasks within the local area of ​​the confrontation boundary section:

[0093] First, using the image confidence weight and point cloud confidence weight output by S104, the image boundary candidates and point cloud boundary candidates within the same segment are adjudicated, generating a boundary adjudication result that can be directly consumed by the subsequent modeling process.

[0094] Secondly, a topological consistency check is performed on the boundary adjudication result to ensure that it meets the preset constraints in terms of connectivity, intersection, and gap with the existing boundary structure, thereby solidifying the adjudication result that passes the check as the unique fusion boundary of the adversarial boundary segment. The inputs of S105 include: the adversarial boundary segment identifier, the corresponding image boundary candidate segment and point cloud boundary candidate segment, the image confidence weight and the point cloud confidence weight, and the preset topological constraints; the outputs of S105 include: the unique fusion boundary (geometric representation and its association identifier with the original candidate boundary), and a pass mark or error type code used to record the check status.

[0095] In generating boundary adjudication results, a segment correspondence can be constructed within each adversarial boundary segment to make the two candidate boundaries comparable in the same parameter domain.

[0096] For example, arc length parameterization can be performed on candidate segments of image boundaries and candidate segments of point cloud boundaries respectively, and sampling can be performed within the segment according to a preset step size or a preset number of sampling points. For each sampling position, corresponding three-dimensional sampling points are obtained on the two candidate boundaries, thus obtaining candidate point pairs. Subsequently, a weighted decision is performed on the candidate point pairs according to the image confidence weight and the point cloud confidence weight to generate a fusion point sequence: for example, the normalized image confidence weight and the point cloud confidence weight are used as weighting coefficients to linearly fuse the image sampling points and the point cloud sampling points to obtain fused sampling points; or the side with higher weight is used as the main boundary and the other side is offset compensated before fusion. The fusion point sequence can be further fitted into a polyline or spline curve to form the boundary decision result of the adversarial boundary segment.

[0097] The above sampling and weighting can be performed numerically using `numpy` / `scipy`; boundary geometry objects can be stored using a lightweight data structure of "point sequence + connection relationship", along with candidate boundary reference information for that segment.

[0098] For topology consistency verification, boundary adjudication results can be treated as geometric objects to be verified, and consistency checks can be performed in conjunction with the confirmed boundary set or segment adjacency relationships. Preset topology constraints can be a set of configurable judgment rules, exemplarily including but not limited to: boundaries must not self-intersect; unexpected intersections must not occur between boundaries and confirmed boundaries; the connection distance between boundary endpoints and their adjacent segment endpoints should be less than a preset tolerance to maintain connectivity; the minimum gap between adjacent boundaries should meet a preset lower limit to avoid adhesion. The above verification can be implemented through line segment / curve intersection detection, closest distance detection, and endpoint connectivity detection, outputting a consistent "pass / fail" result and the corresponding failure reason.

[0099] It can call general computational geometry libraries to complete cross-checking and distance calculation, such as using `shapely` / `pygeos` (2D projection verification) in the Python environment or using CGAL (spatial curve / line segment verification) in the C++ environment; when using 3D boundaries for verification, local projection to the principal plane of the boundary or nearest neighbor distance retrieval based on KD-Tree can be used to improve efficiency.

[0100] When the boundary adjudication result passes the topology consistency check, the boundary adjudication result is solidified as the unique fusion boundary corresponding to the adversarial boundary segment and written into the unique fusion boundary set. At the same time, the source information of the unique fusion boundary (corresponding image boundary candidate identifier, point cloud boundary candidate identifier, weight value, sampling and fitting parameters, etc.) can be recorded for debugging.

[0101] When the validation fails, the process can be handled according to a preset strategy to ensure that the process can be executed: for example, discarding the decision result and falling back to the data source boundary with higher weight as an alternative boundary, or performing partial truncation / splitting on the decision result and then revalidating it.

[0102] Optionally, the unique fusion boundary generated by S105 can serve as a "geometric skeleton constraint" in the urban digital twin modeling process. It provides a reusable structured geometric benchmark for the target area in the form of boundary line elements under a unified coordinate system. For example, the unique fusion boundary set can be used to construct linear elements such as building outlines, road edges, feature dividing lines, or facade boundaries in the target area, and serve as boundary constraints for subsequent mesh generation, patch reconstruction, volumetric modeling, or semantic object instantiation, thereby improving the fusion accuracy and topological consistency of the digital twin model at geometric edges. Furthermore, the unique fusion boundary set can also be used to generate or update the topological relationship graph of the target area (e.g., connectivity, adjacency, and closed loop structures) to support the consistency maintenance of the urban digital twin model in tasks such as spatial querying, collision detection, visualization rendering, and continuous updating of multi-source data.

[0103] Optional, see Figure 2 The flowchart of a method for determining the width of an image uncertainty band and the image offset direction identification provided in this application embodiment includes steps S201~S203, wherein:

[0104] S201: Obtain the solar elevation angle and solar azimuth angle corresponding to the registered image, and obtain the camera optical axis direction vector corresponding to the registered image; determine the local normal vector of the candidate surface corresponding to the image boundary based on the registered point cloud;

[0105] S202: Based on the solar elevation angle and solar azimuth angle, the camera optical axis direction vector, and the local normal vector, calculate the geometric relationship parameters between the incident light direction and the observation direction; obtain the specular reflection coefficient associated with the image boundary candidate;

[0106] S203: Input the geometric relationship parameters and the specular reflection coefficient into a preset specular expansion mapping rule, calculate the boundary expansion amount, and determine the image uncertainty band width based on the boundary expansion amount; and determine the specular reflection direction based on the geometric relationship parameters, and determine the normal side away from the specular reflection direction as the image offset direction identifier.

[0107] In the preceding text, the image uncertainty band width and image offset direction identifier can be obtained from conventional measures such as edge response diffusion and noise level. However, in urban scenarios with a high proportion of highly reflective materials such as glass curtain walls and metal components, specular reflection under specific lighting-observation geometry can easily introduce directional "boundary expansion," causing a systematic deviation in the uncertainty band width and offset direction. To reduce the impact of such systematic deviations on subsequent matching and adjudication, this embodiment introduces the solar altitude angle and azimuth angle, camera optical axis direction vector, local normal vector, and specular reflection coefficient on the image side to construct an executable "geometric parameter-spectral expansion mapping" process to determine the image uncertainty band width and image offset direction identifier.

[0108] In practice, the solar altitude angle and solar azimuth angle are first obtained for each registered image frame, and the camera optical axis direction vector corresponding to that frame is also obtained. The solar altitude angle and solar azimuth angle can be obtained based on the image acquisition time and location using existing solar position calculation algorithms. For example, the system can record the acquisition date, time, and latitude and longitude in the image metadata or acquisition log, and call the astronomical calculation module to output a set of numerical solar altitude angles and azimuth angles. The camera optical axis direction vector can be directly calculated from the camera's attitude parameters in a unified coordinate system. For example, a unit direction vector representing the camera's main viewing direction can be extracted using the orientation information in the camera attitude matrix, and the above parameters are saved as image frame-level metadata.

[0109] Then, the local normal vectors of the corresponding surfaces of each image boundary candidate are determined based on the registration point cloud. Specifically, the system can back-project the sampling points on the image boundary candidates to a unified coordinate system, retrieve a set of spatial points falling near the sampling point in the registration point cloud, perform local plane fitting on the set of spatial points, and select a unit vector from the plane normal direction as the local normal vector at that point.

[0110] For example, for each image boundary sampling point, the system can retrieve several point cloud points within a preset radius, obtain a three-dimensional direction vector representing the orientation of the building surface through principal component analysis or least squares plane fitting, and associate this direction vector with the corresponding image boundary candidate in the form of a numerical field.

[0111] After obtaining the solar direction, camera optical axis direction, and local normal vector, geometric parameters relating the incident light direction to the observation direction are constructed based on the angular relationship between these three factors. These geometric parameters characterize whether the current location is in a geometric configuration that easily produces specular reflection. For example, they may include: whether the angle between the incident ray and the surface normal is close to frontal illumination; whether the angle between the surface normal and the camera's line of sight is conducive to the reception of specular light by the camera; and whether the angle between the predicted specular reflection direction and the camera's line of sight is within a preset threshold.

[0112] For example, the above geometric relationship can be converted into one or more numerical indicators, such as highlight risk level or highlight alignment score, and recorded as geometric relationship parameters in the attributes of image boundary candidates.

[0113] Simultaneously, the specular reflection coefficient associated with the image boundary candidate is obtained to quantify the specular reflection capability of the material at that location. This specular reflection coefficient can be derived from the material attribute field in the building information model or from a preset material classification and parameter lookup table.

[0114] For example, if a boundary candidate is labeled as a glass curtain wall based on a BIM model or semantic segmentation results, the system can retrieve a set of typical specular reflection coefficient values ​​from a pre-set glass material parameter table; if labeled as a metal railing, the corresponding coefficient is obtained from a metal material parameter table; for areas lacking material information, default coefficients can be used as a degradation scheme. The specular reflection coefficient is bound to the image boundary candidate in the form of a single or several floating-point numbers for subsequent mapping calculations.

[0115] When determining the width of the uncertain band in the image, the aforementioned geometric parameters and specular reflection coefficient are input into a preset specular expansion mapping rule to calculate the boundary expansion amount. This mapping rule can be implemented using a lookup table or a simple function. The general principle is: the closer the geometric relationship is to the configuration where specular highlights are likely to occur, and the larger the specular reflection coefficient, the greater the boundary expansion amount; conversely, the expansion amount is smaller.

[0116] For example, the "highlight risk level" can be divided into several levels (such as low, medium, and high), and an additional extended bandwidth value can be assigned to each level to increase the bandwidth based on the baseline image bandwidth obtained in S102. Alternatively, a risk score in the range of 0 to 1 can be calculated based on geometric parameters and the specular reflection coefficient, and this score can be converted into an additional bandwidth value using a preset monotonic mapping curve. Ultimately, the image uncertainty band width is obtained from the baseline bandwidth and the boundary expansion amount, and can be limited to a reasonably reasonable minimum and maximum value range in engineering, such as a range of several centimeters to several tens of centimeters, to avoid extreme values.

[0117] When determining the image offset direction identifier, the specular reflection direction is determined based on the aforementioned geometric parameters, and the normal side that deviates from this specular reflection direction is identified as the image offset direction identifier. Specifically, the system can define a pair of opposite normal sides in the local coordinates of the image boundary, for example, using a positive normal side and a negative normal side to represent the two sides of the region; after estimating which side the specular light mainly diffuses towards by combining the sun direction, surface normal, and camera optical axis direction, the side with more significant specular light diffusion is marked as the "spectral side," and the opposite side is marked as the "structural reality side," and the normal direction corresponding to the "structural reality side" is used as the offset direction identifier.

[0118] For example, if geometric parameters indicate that specular reflections are mainly concentrated on one side of the boundary normal, the system will record the other side as the offset direction to guide subsequent offset compensation between the image and the point cloud in the adversarial boundary segment. The offset direction identifier can be represented by a simple symbolic or enumerated field, such as recorded as facing the outdoor side or facing the building interior side, which is convenient for direct use in subsequent adversarial discrimination and weighting decisions.

[0119] In this way, the uncertainty band width and offset direction identifiers on the image side are expanded from relying solely on edge response and noise level to parameterized results that explicitly consider illumination-observation geometry and material properties, thus better adapting to the digital twin modeling needs of highly reflective urban scenes such as glass curtain walls and metal components.

[0120] Alternatively, the method described above for determining image uncertainty bands using illumination-observation geometry and specular reflection coefficients can effectively reflect directional errors caused by specular clipping. However, in complex urban scenes, especially in areas with numerous glass curtain walls and large reflective facades, another typical problem exists: due to reflections of opposite buildings, the sky, or indoor objects from the glass surface, a large number of "virtual image edges" appear in the image. These edges do not correspond to the actual physical contours in terms of geometric location. If only geometric relationships and material parameters are relied upon, they may still be mistakenly treated as "credible boundaries" in adversarial adjudication, affecting the accuracy of the final fused boundary. Therefore, it is necessary to further analyze the high-frequency information distribution on both sides of the boundary at the local texture level to identify suspected virtual image edges and adjust the width and offset direction of the image uncertainty band accordingly.

[0121] In one optional implementation, after determining the width of the uncertain band and the offset direction of the initial image based on lighting geometry and material parameters, high-frequency component extraction is performed on the neighborhood images of the candidate image boundaries to obtain high-frequency texture components for virtual image discrimination. Specifically, in the image coordinate system, the system can take a strip-shaped region of several pixels on both sides of each candidate image boundary as an analysis window, for example, taking a local neighborhood of 5 to 15 pixels on both sides of the normal direction. For this neighborhood region, the system can apply high-pass filtering or band-pass filtering to highlight the high-frequency components of edges and textures, such as using conventional image processing techniques like Laplacian filtering, Sobel / Scharr gradient filtering, differential Gaussian (DoG) filtering, or wavelet high-frequency subband decomposition. In implementation, the functions `skimage.filters.laplace` and `skimage.filters.sobel` of scikit-image can be called in the Python environment to use the filtering results as high-frequency texture components.

[0122] After obtaining the high-frequency texture components, the system can divide the neighborhood into two sub-regions, "one normal side region" and "the other normal side region," along the normal direction of the candidate image boundary, and statistically analyze the high-frequency energy distribution in each sub-region. For example, the absolute or squared values ​​of the high-frequency response in each sub-region can be accumulated or averaged to obtain two numerical indices, which are used to characterize the strength of the high-frequency texture energy on both sides of the normal direction. To mitigate the influence of isolated noise, the high-frequency response can be truncated or filtered by median before calculation; robust statistics such as quantiles and truncated averages can also be used during the statistical analysis. Based on the high-frequency energy of the two sub-regions, a virtual image edge score can be further constructed to measure whether the high-frequency energy distribution on both sides of the boundary is significantly asymmetrical, and whether it exhibits a characteristic where the texture is mainly concentrated on one side and relatively smooth on the other.

[0123] From a physical perspective, the virtual image edge formed by glass reflection often manifests as follows: one side of the glass surface is superimposed with complex textures from the opposite building or indoor objects, while the other side corresponds to a spatial area outside the real physical outline, such as the sky, distant view, or low-texture area. There is a significant difference between the two in terms of high-frequency texture energy.

[0124] Therefore, high-frequency energy difference or high-frequency energy ratio can be used as the core component of virtual image edge scoring. Combined with auxiliary indicators such as edge response intensity and local contrast, a numerical virtual image edge score can be generated to describe the probability that the boundary candidate is dominated by a virtual image. For example, the score can be normalized to the range of 0 to 1, and an empirical threshold such as 0.7 or 0.8 can be set as the virtual image determination threshold.

[0125] When the virtual image edge score meets the preset judgment conditions, such as being higher than the virtual image judgment threshold, it can be considered that the image boundary candidate is significantly affected by virtual images in the current segment. In order to reduce its dominant role in subsequent adversarial adjudication, it is necessary to perform extended correction on the image uncertainty band width.

[0126] In practice, an additional bandwidth value related to the virtual image edge score can be superimposed on the baseline bandwidth obtained above. For example, the higher the virtual image score, the larger the additional bandwidth. The additional bandwidth can vary linearly or piecewise between preset minimum and maximum expansion amounts to avoid overexpansion. The corrected image uncertainty band width obtained after this expansion correction will cover a larger range of possible errors.

[0127] Meanwhile, the system can also update the image offset direction identifier based on the high-frequency texture energy distribution. Considering that high-frequency textures are often concentrated on the virtual image side or the reflection side, while the texture on the real geometric contour side is relatively simple and the noise is relatively low in many scenarios, in sections with high scores at the virtual image edge, the direction along the candidate normal of the image boundary pointing to the side with lower high-frequency texture energy can be used as the corrected image offset direction identifier to reflect the empirical rule that the real geometric structure is more likely to exist on the side opposite to the virtual image. In implementation, the magnitude of the high-frequency energy statistics on both sides of the normal can be compared, and the normal direction corresponding to the side with lower energy can be selected as the new offset direction identifier, overriding the original offset direction determined based on illumination geometry; if the difference in high-frequency energy on both sides is insufficient to support the direction flip, the original offset direction can be retained unchanged.

[0128] In this way, the uncertainty band width and offset direction identifiers on the image side introduce texture level recognition and correction for virtual image edges on the basis of the original lighting-material driven method. This allows the influence of virtual image edges to be further amplified in the uncertainty band and offset strategy in complex scenes such as glass reflection and multi-layer superimposition, which is beneficial to weaken the interference of virtual image boundaries on the final fusion result in the subsequent adversarial adjudication with point cloud boundaries.

[0129] Optionally, in order to make full use of existing building information model resources in the urban digital twin scenario and provide a clear physical basis for the source of material parameters, this implementation introduces a building information model that is spatiotemporally aligned with the target area. The mapping between image boundary candidates and building components is established through semantic parsing and spatial indexing, and then the specular reflection coefficient is extracted by combining a pre-set physical material library.

[0130] Specifically, a building information model (BIM) aligned with the target area in time and space can be acquired first. The BIM model can be a BIM model, a 3D city model, or other forms of building component-level digital models, such as IFC format BIM files, BIM models exported from Revit, or component-based city models based on CityGML / oblique photogrammetry models. To achieve time and space alignment, it is ensured that there is a known coordinate transformation relationship between the model coordinate system and the unified coordinate system in S101, and that the model version and the image / point cloud acquisition time are at the same construction or change stage, thus avoiding component mismatches caused by building modifications.

[0131] After acquiring the Building Information Model (BIM), the system parses the semantic information data of its components. The parsing includes, but is not limited to: component type (e.g., exterior walls, curtain walls, windows, doors, railings, roofs, etc.), component geometry (e.g., patch sets, blocks, skeleton lines, etc.), component spatial location and orientation, component hierarchical structure (e.g., building-floor-component), and material attribute fields associated with the components (e.g., material name, reflection characteristics, color, roughness, etc.). The parsing process can be implemented using a dedicated BIM parsing library, such as calling IfcOpenShell in a Python environment to read IFC files and extract the geometric and attribute information of corresponding entities such as IfcWall, IfcCurtainWall, and IfcWindow.

[0132] After semantic parsing, the system establishes a mapping relationship between image boundary candidates and building components through spatial indexing. Specifically, using the camera pose and registration relationship obtained in S101, pixels or sampling points on the image boundary candidates are back-projected onto a unified coordinate system to form a set of spatial rays or spatial sampling points. Subsequently, the geometry of the components traversed or adjacent to these rays / sampling points is retrieved in the building information model. For example, a ray-polygon (or ray-mesh) intersection test can be used to calculate the intersection of each ray with the surface of the building component, and the intersecting component closest to the camera position is selected as the building component corresponding to that ray (i.e., the image boundary sampling point). Alternatively, a spatial nearest neighbor search can be used to find the component patch or voxel unit closest to the sampling point within a certain distance threshold.

[0133] After establishing the spatial mapping relationship between image boundary candidates and building components, the material attribute information of the corresponding building components can be read. Material attribute information can include fields such as material name (e.g., Low-E glass, stainless steel plate, polished granite), surface roughness level, gloss, refractive index, and reflectivity. Considering the differences in the naming and meaning of material fields in different BIM models or design software, the system can set up a material attribute parsing adaptation layer to standardize common fields.

[0134] Subsequently, based on the aforementioned material attribute information, a pre-defined physical material library is queried to extract the corresponding specular reflection coefficient. The physical material library can be pre-built by domain experts and typically records a set of physical parameters corresponding to different typical building materials, such as specular reflection coefficient, diffuse reflection coefficient, refractive index, and absorption coefficient, in the form of tables, databases, or configuration files. For example, a set of numerical specular reflection coefficients can be configured for typical materials such as transparent curtain wall glass, coated glass, brushed stainless steel, sprayed aluminum panels, exterior wall coatings, and fair-faced concrete. In implementation, the material category or standardized material name can be used as a search key to find the corresponding entry in the material library, and the obtained specular reflection coefficient is appended as a numerical parameter to the image boundary candidate.

[0135] This is beneficial for improving the reliability of determining the width of the image uncertainty band and the offset direction in complex building material mixed scenarios.

[0136] Optional, see Figure 3 The flowchart of a method for determining the width of an uncertain band and the offset direction of a point cloud, provided in an embodiment of this application, includes steps S301 to S303, wherein:

[0137] S301: Obtain the laser acquisition parameters associated with the points in the registration point cloud, the laser acquisition parameters including the ranging value, incident beam direction information, echo identifier and divergence angle parameter;

[0138] S302: For each point cloud boundary candidate, the incident geometric relationship is determined based on the incident beam direction information and the local normal of the point cloud boundary candidate, and the laser footprint scale is determined based on the ranging value and the divergence angle parameter.

[0139] S303: Determine the width of the uncertain band of the point cloud based on the incident geometry and the laser footprint scale; and determine the offset direction marker of the point cloud based on the distribution difference of the echo marker on both sides of the candidate normal of the point cloud boundary.

[0140] In practical implementation, in real-world urban digital twin modeling scenarios, if the uncertainty band width of the point cloud is set solely based on geometric distribution (such as local point density and curvature changes), it is often difficult to accurately reflect the error characteristics introduced by the lidar acquisition link itself. This is especially true in areas with long-distance measurement, grazing incidence, large beam divergence, or multiple target boundaries, where the spatial "footprint" of the point cloud will be stretched and magnified, and the geometric position error of the boundary points will exhibit obvious anisotropy. Using only a fixed bandwidth or a bandwidth simply related to point density can easily underestimate the uncertainty, thus leading to deviations in adversarial adjudication against image boundaries.

[0141] Therefore, this implementation introduces laser acquisition parameters associated with point cloud points. Starting from physical acquisition factors such as ranging value, incident beam direction information, echo marker, and divergence angle parameters, the width of the uncertain band and the offset direction marker of the point cloud are determined in a way that is more in line with the imaging mechanism.

[0142] In an optional implementation, when generating the registration point cloud data in S101, laser acquisition parameters are recorded for each point cloud point. For example, the laser acquisition parameters may include: a distance value along the laser emission direction, for example, in meters; a direction vector or scanning angle characterizing the laser beam emission direction, such as a horizontal scanning angle or vertical scanning angle; echo identifiers (e.g., the sequence number of the first echo, intermediate echo, and last echo, or a single echo / multiple echo category marker); and laser beam divergence angle parameters, such as half-angle divergence values ​​given in milliradians or angles. In a specific implementation, the raw distance, scanning angle, echo sequence number, and other fields output by the hardware can be directly written into the point cloud structure or point cloud attribute array in the laser driver or point cloud driver; in a C++ environment, PCL or a custom point type can be used to treat these parameters as additional fields in the point type (such as `range`, `azimuth`, `elevation`, `return_id`, `beam_divergence`, etc.).

[0143] Based on the aforementioned data preparation, the system determines the incident geometry relationship for each candidate point cloud boundary based on the incident beam direction information and the local normal of that candidate boundary. Specifically, points on the boundary are sampled along the trajectory of the candidate point cloud boundary at a preset step size, and the corresponding incident beam direction information and local normal vector are read at each sampled point. The local normal vector can be obtained in S102 through neighborhood fitting plane or principal curvature analysis; the incident beam direction information comes from the laser emission direction parameter at that point. The angle between the incident beam and the local surface normal can be calculated to distinguish between near-perpendicular incident and grazing incident cases. Intuitively, when the incident beam is nearly perpendicular to the surface normal, the laser footprint on the component surface is nearly circular, and the positional error is relatively small; while when the incident beam is nearly grazing along the tangential direction of the surface, the footprint is stretched on the local surface, making it easier to cross the foreground / background near the boundary, resulting in a significant increase in the uncertainty of the boundary point in the normal direction. The incident geometry can be recorded as a numerical incident angle index or incident geometry score for use in subsequent bandwidth calculations.

[0144] To determine the scale of laser footprints, a distance-dependent spot size estimate can be constructed using ranging values ​​and divergence angle parameters. Generally, given a fixed laser beam divergence angle, the larger the ranging value, the larger the footprint area on the target surface, and the more significant the blurring effect on the boundary position; at the same distance, the larger the divergence angle, the larger the footprint scale. Therefore, the system can estimate the "lateral scale" or "radius range" of the laser footprint at each boundary sampling point based on the ranging value and divergence angle parameter. For example, a "unit distance diffusion coefficient" can be preset for different types of lidar, and during operation, a linear or piecewise linear amplification can be performed based on the ranging value to obtain an approximate footprint scale; alternatively, the nominal divergence angle provided by the manufacturer can be used as a scaling factor superimposed on the above estimate, so that the larger the divergence angle, the larger the footprint estimate. In implementation, footprint scale ranges can be configured for commonly used ranging intervals using a lookup table, such as using different footprint scale parameters for 0–50 meters, 50–100 meters, and above 100 meters, thus avoiding complex real-time calculations.

[0145] Subsequently, the width of the uncertain band in the point cloud is determined jointly based on the incident geometry and the laser footprint scale. For example, the laser footprint scale can be considered as the basic error scale, and the incident angle index as the amplification factor: when the incident angle is close to the normal, the amplification factor is close to 1, and the width of the uncertain band in the point cloud is close to the basic footprint scale; as the incident angle gradually changes to grazing, the amplification factor increases accordingly, and the width of the uncertain band in the point cloud is widened to reflect the increased positional uncertainty of the boundary points in the normal direction under grazing conditions. In practical engineering, to avoid excessive bandwidth affecting subsequent matching efficiency, upper and lower limits can be set for the width of the uncertain band in the point cloud, for example, truncating it within a preset range to ensure that both physical diffusion effects are considered and computability is maintained. The width of the uncertain band in the point cloud can be calculated point-by-point on the boundary curve according to the sampling points, and aggregated into segment-level bandwidth through averaging or median methods, or it can be directly stored as a point-level attribute for local calculation in S103 to S105.

[0146] In determining the offset direction markers of the point cloud, the physical "solid side" and "empty side" of the boundary are inferred by utilizing the difference in the distribution of echo markers on both sides of the candidate normal of the point cloud boundary. In urban scenarios, a point cloud boundary often corresponds to the boundary between a building facade and the external space, such as the interface between curtain wall glass and outdoor air, or the interface between railings and gaps. When a laser beam passes through this interface, it may generate single or multiple echoes: if it mainly hits the solid surface, it is mostly a single echo or the first echo is concentrated on the solid side; if some energy penetrates or crosses the solid boundary, the last echo or long-distance echo may fall on the empty side or the background surface. Based on this, in this embodiment, the system can divide the local normal of each candidate point cloud boundary into two regions: a positive normal side and a negative normal side, and statistically analyze the distribution of echo markers of point cloud points in each region, such as the proportion and spatial distribution of the first echo, non-first echo, and last echo on both sides.

[0147] For example, a dividing line along the normal direction can be set near the candidate point cloud boundary, dividing the point cloud neighborhood into two sides according to this line. The number of first echo points and last echo points are counted on each side. Generally, the entity side is more likely to have a large number of first echo points, while the empty side or the side behind occlusion is more likely to have last echo points or sparse echoes. Based on this, an "entity side determination rule" can be constructed, for example: comparing the proportion of first echo points on both sides, and identifying the side with a denser first echo as the entity side; to further enhance robustness, information such as point density and ranging value distribution can be combined for weighted judgment. After determining the entity side, the normal direction pointing to the entity side or the normal direction pointing to the opposite side can be used as the point cloud offset direction identifier, depending on whether the system wants to offset towards the structured side or the unstructured side. Optionally, to facilitate adversarial discrimination against the image-side offset direction marker, the point cloud offset direction marker can be defined as the normal direction pointing to the "more believable entity side of the point cloud". Subsequently, in S103, the adversarial boundary segment is identified by comparing whether the image offset direction and the point cloud offset direction are opposite.

[0148] In this way, the width of the uncertain band and the offset direction of the point cloud are no longer determined solely by geometric and statistical characteristics. Instead, key parameters from the physical process of laser ranging (range value, incident direction, divergence angle, and echo marker) are explicitly introduced. Under conditions of long distance, grazing, semi-transparency, or complex occlusion, a more physical description of the uncertainty and offset tendency of the point cloud boundary is provided. This is beneficial for more accurately controlling the weight and directionality of the point cloud side in subsequent adversarial adjudication with image boundaries, thereby improving the overall robustness and accuracy of multi-source boundary fusion in urban digital twin modeling.

[0149] Optionally, when using a lidar sensor that supports multiple echoes, the true geometric uncertainty near the boundary may be underestimated if the multi-echo ranging relationship of the same incident beam is not analyzed. Especially near some transparent curtain walls, sparse vegetation, or grid components, the same incident beam often produces two types of ranging results: first echo and last echo, which correspond to reflection interfaces at different depths.

[0150] Therefore, in one optional embodiment, after acquiring the laser acquisition parameters including echo identifiers, the point cloud can be grouped according to the incident beam identifier within the neighborhood of the point cloud boundary candidate. For each group containing more than one echo of the incident beam, its first echo ranging value and last echo ranging value can be read separately, and the difference between the two can be calculated; the difference can be the absolute difference between the two, and the result can be directly used as the depth separation index of the incident beam. For a certain point cloud boundary candidate or a certain segment, the difference of multiple multi-echo incident beams in the neighborhood of the boundary candidate can be summarized, for example, by taking the average value or taking the value at a preset quantile, to obtain a difference index representing the depth stratification degree of the region.

[0151] Regarding difference determination, the system can pre-configure a difference determination threshold to identify areas with "significant multi-layer reflections". This threshold can be set in conjunction with the scale of common components in typical scenarios. For example, in a building curtain wall scenario, the threshold can be set to a range of 0.5 meters to 2 meters to distinguish between general distance measurement fluctuations and multi-echoes that clearly cross the front and rear components. In scenarios with abundant vegetation on both sides of the road, the threshold can also be adjusted based on empirical parameters such as tree canopy thickness and the depth of roadside structures.

[0152] When the difference index of a candidate neighborhood at a certain boundary reaches or exceeds the aforementioned difference judgment threshold, the point cloud in that region can be considered significantly affected by multiple echoes, making it necessary to expand the width of the point cloud uncertainty band. The expansion rule can be implemented through coefficient amplification or incremental compensation. For example, the original bandwidth can be multiplied by an expansion coefficient greater than 1, which can be set in tiers according to the magnitude of the difference; or a compensation width positively correlated with the difference can be superimposed on the original bandwidth. The expanded and updated bandwidth can be written back into the corresponding point cloud uncertainty band parameters for subsequent weight calculation and adversarial segment adjudication.

[0153] Regarding the updating of point cloud offset direction identifiers, the distribution of the number of points on both sides of the candidate normal direction of the point cloud boundary can be used for judgment. In specific implementation, a local coordinate system can be established locally in the candidate boundary area, with the boundary tangent as one axis and the normal as the other axis. The first and last echo points are projected onto this local coordinate system, and their number on the positive and negative normal sides is counted. For each sampling segment, the proportion of the first echo point on both sides and the proportion of the last echo point on both sides can be calculated separately. When the statistical results show that the proportion of the first echo point in a certain area is significantly higher than that on the other side, the normal direction pointing to that side can be determined as the point cloud offset direction identifier to reflect the more reliable entity interface side of that area. The threshold and the judgment criterion of "significantly higher" can be set based on experience, such as requiring the proportion of points on one side to be higher than 60% or 70%, to improve the stability of the direction determination.

[0154] Through the above multi-echo difference analysis and offset direction update based on the distribution of first and last echoes, on the one hand, the width of the uncertain band of the point cloud was widened in a targeted manner in the multi-layer reflection region to avoid the point cloud side being overly optimistic about the boundary position; on the other hand, the offset direction label was corrected by using the spatial distribution of the first echo, so that the point cloud still has relatively clear directional information "closer to the entity side" in the complex multi-echo region, providing more physically based basic data for the subsequent adversarial weight calculation with the image side.

[0155] Optionally, the above has already given the general idea of ​​forming boundary matching pairs through the spatial intersection relationship between uncertain zones in the image and uncertain zones in the point cloud. However, if we directly perform pairwise distance determination on all uncertain zones in the global space, it will not only lead to excessive computation, but also easily introduce a large number of pseudo-matches that are only geometrically close, but do not correspond in terms of imaging viewpoint and occlusion relationship. To improve the reliability of matching, this application further introduces the construction of search regions on the image side and search regions on the point cloud side, and further filters candidate matching pairs through visibility discrimination.

[0156] In one optional embodiment, the image-side search region can be constructed based on the camera parameters of the registered image and the pixel range of the image uncertainty band. Specifically, pixel positions within the image uncertainty band can be sparsely sampled, for example, by selecting pixels at a fixed step size or randomly selecting a certain proportion of pixels at a fixed number. Then, these pixels are back-projected into line-of-sight rays passing through the camera's optical center using camera intrinsic and extrinsic parameters. In three-dimensional space, depth clipping ranges can be set for the near and far ends of each line-of-sight ray. For example, the near end distance can be set to several meters to exclude the platform's own structure, and the far end distance can be set to tens to hundreds of meters to cover the typical distance range of target urban components. The specific values ​​can be determined based on platform height, building height statistics, and point cloud range design. The set of spatial positions of each line of sight within this depth range constitutes a discrete approximate representation of the image-side search region, which can be recorded in the form of voxel grids or sparse point sets.

[0157] The point cloud-side search region can be constructed by centering on the spatial trajectory of the point cloud boundary candidate and laterally expanding it around it according to the width of the point cloud uncertainty band. Specifically, a strip-shaped or tubular region can be constructed within the normal and tangential neighborhoods of the boundary trajectory, with its cross-sectional radius equal to or slightly larger than the width of the point cloud uncertainty band. For example, in scenarios requiring improved matching tolerance, a certain percentage of safety margin can be added to the bandwidth. The interior of this region can be encoded using a 3D voxel mesh, bounding box set, or other spatially indexed data structures to facilitate rapid intersection testing with the image-side search region.

[0158] In forming candidate matching pairs, the system can first construct a spatial index structure for the point cloud-side search region, such as building a 3D R-tree or octree index based on the bounding boxes of each region. Then, for each bounding box of the image-side search region, a spatial overlap query is performed to obtain a set of point cloud-side search regions that overlap with it. For each pair of image-side and point cloud-side search regions that satisfy a preset spatial intersection condition, their corresponding image boundary candidates and point cloud boundary candidates can be recorded as candidate matching pairs. The preset spatial intersection condition can be that the bounding boxes of the two regions have a non-zero overlap volume, or that the distance is less than a certain threshold. The threshold can be configured according to the typical range of uncertain bandwidth of the point cloud; for example, in scenarios with bandwidths ranging from several centimeters to tens of centimeters, the intersection distance threshold can be set to the same order of magnitude to achieve a balance between matching accuracy and recall.

[0159] After obtaining candidate matching pairs, a visibility discrimination is performed on these pairs to eliminate combinations that are only geometrically close in three dimensions but have significant occlusion in the line-of-sight path. For example, for a candidate matching pair, the system can search for point clouds located between the camera position and the point cloud search area along the corresponding line-of-sight rays in the image-side search area, and count the point cloud density or continuity along the ray path. If there is a large-scale continuous point cloud cluster before the candidate point cloud boundary, and its spatial location corresponds to the area of ​​a known foreground component, the visibility of the candidate matching pair can be determined to be poor; conversely, if there are few point clouds or only a few isolated points in the ray path, the visibility can be considered good.

[0160] Visibility score can be calculated based on factors such as the number of occluded point clouds, their distribution uniformity, and their distance from candidate regions. A visibility threshold can be set; for example, a threshold can be selected based on the overall point cloud density that results in a higher score for normal, unobstructed views and a lower score for views completely blocked by the foreground. When the visibility score of a candidate pair is lower than this threshold, the pair is removed from the candidate set; when the score is not lower than the threshold, it is retained as a valid boundary pair. The visibility threshold can be configured according to different urban scenarios. For example, the threshold can be appropriately lowered in old urban areas with many obstructions, while it can be increased in open street scenes or elevated road scenes to strengthen the suppression of the impact of occlusion.

[0161] By constructing the image-side search region and the point cloud-side search region, and by using visibility discrimination based on occluded point clouds, the boundary combinations participating in adversarial adjudication are further restricted, reducing false matches affected by occlusion, multi-layer structures, and ambiguity of view cone projection. This is beneficial to improving the stability and reliability of subsequent adversarial boundary segment identification and unique fusion boundary generation.

[0162] Optionally, if the uncertain bands on the image side and the uncertain bands on the point cloud side already meet the spatial intersection condition, then further, how to identify the boundary matching pairs that are actually on the camera's visible path, suppress false matching caused by foreground occlusion, glass penetration, or multi-layer structure superposition, thereby improving the physical consistency of the boundary matching pairs and the reliability of subsequent fusion.

[0163] In practical implementation, a set of line-of-sight rays can be constructed based on the camera parameters of the registered image for each image boundary candidate and its corresponding image uncertainty zone. For example, several pixel positions can be selected along the arc length direction of the image boundary candidate at a preset step size, which can be between 2 and 10 pixels. For each selected pixel position, the line-of-sight direction corresponding to that pixel in a unified coordinate system is calculated based on the camera intrinsic parameters, distortion parameters, and camera pose, and the camera optical center is used as the starting point of the line of sight, thus obtaining multiple discrete line-of-sight rays. To limit the subsequent search range, the effective distance upper limit of each ray can be determined by combining the average distance between the candidate matching midpoint cloud boundary candidate associated with the image boundary candidate and the camera. For example, the effective distance upper limit can be set to a range of several meters before and after the average distance, with a forward reserved distance of 0.5 to 3 meters and a backward margin of 0.2 to 1 meter.

[0164] For the point cloud side, the search area constructed above can be directly used as the target area for visibility discrimination. That is, the spatial trajectory of the point cloud boundary candidates is used as the center line, and the cross-sectional radius is determined based on the width of the uncertain band in the point cloud, forming a three-dimensional tubular region. The value of the cross-sectional radius can be 1 to 2 times the width of the uncertain band in the point cloud. When the point cloud measurement noise is high or the points near the boundary are sparse, the cross-sectional radius should be appropriately increased to avoid missed detections. When the point cloud density is high and the noise is low, a smaller radius close to the width of the uncertain band can be selected to reduce the probability of introducing irrelevant points. To improve the spatial query efficiency of rays and point clouds, the three-dimensional tubular region and the retrieval space in front of it can be organized using a conventional spatial index structure. For example, the point cloud can be divided into fixed grid or voxel units, and the point cloud index included in each unit can be maintained.

[0165] During the visibility assessment process, for each candidate matching pair, occlusion point retrieval is performed in the registered point cloud along its corresponding line-of-sight ray set. Specifically, for each line-of-sight ray, point cloud points within a certain neighborhood distance of the ray are retrieved within the spatial range between the camera position and the search area on the point cloud side of the candidate matching pair. Points with ranging values ​​less than the average distance of the candidate matching pair's point cloud boundary are selected as occlusion candidate points. In the case of multi-echo ranging, occlusion candidate points are preferentially selected from the first echo point set and the echo point set with smaller ranging values ​​to more closely approximate the actual visible surface. The neighborhood distance between the ray and the point cloud points can be set according to the average point spacing of the point cloud, for example, set to 0.03 meters to 0.15 meters. It can be appropriately increased when the point cloud is sparse and appropriately decreased when the point cloud is dense.

[0166] After obtaining the ray-level occlusion results, the visibility score can be defined as the proportion of effective, unoccluded lines of sight in the candidate matching pair. For example, for all lines of sight belonging to a candidate matching pair, the number of lines that do not hit occluded candidate points and the total number of lines of sight are counted, and the ratio of the two is calculated as the visibility score of the candidate matching pair. To reduce interference from occasional occlusion, rays that hit only a single isolated point can be ignored, and rays that hit multiple points consecutively can be counted as strong occlusion rays. The visibility discrimination threshold can be preset according to the actual scene and the system's tolerance for mismatches. For example, for street scenes with complex occlusion environments, the threshold can be set between 0.6 and 0.7; for open roads or square scenes with less occlusion, the threshold can be set between 0.7 and 0.9. The specific value of the threshold can be obtained through offline labeled sample statistics.

[0167] When the visibility score of a candidate matching pair is not lower than a preset visibility threshold, the candidate matching pair is retained as a boundary matching pair. When the visibility score is lower than the threshold, the matching pair is determined to lack sufficient visibility from the current viewpoint and is discarded to avoid using false correspondences formed by occlusion or penetration in subsequent fusion stages. When multiple boundary matching pairs correspond to the same image boundary candidate, the pair with the highest score among those with visibility scores not lower than the threshold can be selected as the final boundary matching pair. When the scores are close, auxiliary indicators such as the length of the intersection segment or the degree of overlap can be used to make decisions. For example, matching pairs with a longer spatial overlap length or a larger number of overlapping point clouds are preferred, thereby further improving the robustness of the matching results while ensuring visibility.

[0168] This significantly reduces the false matching rate caused by foreground occlusion, glass perspective, and multi-layer structure superposition, providing a more reliable set of input boundaries for subsequent adversarial boundary adjudication and topology consistency verification.

[0169] In an optional embodiment, regarding S104 above, before mapping the uncertain band width of the segment-level image and the uncertain band width of the point cloud to confidence weights, a signal morphology-based artifact index correction step is further introduced. The main technical problem solved by this embodiment is that, under extreme imaging conditions such as light percolation tailing and multi-echo splitting, when weight allocation is based solely on the macroscopic dimension of bandwidth, segments with small bandwidth but low physical reliability are prone to appear, affecting the adjudication accuracy of adversarial boundary segments.

[0170] Specifically, on the image side, a brightness profile sequence can be extracted along the normal direction of the image boundary candidate near each adversarial boundary segment. For example, multiple sampling positions can be selected along the arc length direction of the image boundary candidate with a step size of 1 to 3 pixels. For each sampling position, a brightness profile spanning the image boundary candidate is extracted from the image according to a preset normal sampling length. The normal sampling length can cover a range of several pixels on both sides of the image uncertainty band width, for example, extending 1 to 3 times the image uncertainty band width on both sides, or no less than 5 pixels. For each brightness profile, smoothing and normalization processing can be performed, and the peak region near the high-brightness side and the transition regions from high to low on both sides can be identified on the profile. Brightness overshoot can be understood as the excess of the peak brightness relative to the average brightness of the adjacent stable region, and tail asymmetry can be understood as the difference in the length of the brightness transition regions on both sides of the normal direction or the difference in attenuation rate. Subsequently, the brightness overshoot and trailing asymmetry at all sampling locations within an adversarial boundary segment can be statistically analyzed. For example, the mean, maximum, or high quantile can be calculated. The statistical results are then scaled to a range of 0-1 using a preset segmented lookup table rule to obtain the image artifact index. Specifically, when the brightness profile is close to an ideal step shape and overshoot and trailing are limited, the image artifact index is close to 0; when there are significant overshoot peaks and long trailing, the image artifact index is close to 1. In implementation, existing edge detection and one-dimensional signal analysis tools can be called in the image processing module. For example, OpenCV and NumPy can be used in a Python environment to sample, smooth, and detect extreme values ​​in the brightness profile.

[0171] On the point cloud side, echo distribution features can be extracted within the candidate neighborhood of the point cloud boundary corresponding to each adversarial boundary segment. Specifically, all point cloud points with first echo and last echo identifiers can be collected within the spatial range of this segment, and the point cloud points are assigned to the corresponding incident beam trajectories according to the incident beam number or ray index. For each incident beam, the difference between the first echo ranging value and the last echo ranging value can be calculated, and the difference can be compared with a preset difference threshold. The difference threshold can be set according to the ranging accuracy of the lidar and the scene scale, for example, it can be selected in the range of 0.05 to 0.30m. At the same time, the number, point density, or height distribution of echo points on both sides of the boundary normal can be counted to characterize the difference in dispersion of the point clouds on both sides. If the difference between the first and last echo ranging on the same incident beam is significant and the point cloud on one side of the normal is sparse while the point cloud on the other side is concentrated, it indicates that the multi-echo splitting or transmission effect is strong; conversely, when the difference between the first and last echo ranging is small and the point cloud distribution on both sides is balanced, the boundary echo morphology can be considered to be relatively ideal. Based on the aforementioned multi-echo difference and dispersion on both sides of the normal axis, these can be scaled to a range of 0 to 1 through normalization and weighted statistics to form a point cloud artifact index. A value close to 0 indicates a clean point cloud signal with clear boundaries, while a value close to 1 indicates strong multi-echo or flying point interference. In implementation, the neighborhood retrieval and clustering interfaces provided by PCL or Open3D can be called in a C++ or Python environment to complete the construction and statistics of point cloud neighborhoods.

[0172] After obtaining the image artifact index and the point cloud artifact index, a penalty adjustment is performed on the original segment-level image uncertainty band width and the point cloud uncertainty band width. For example, a magnification factor range associated with the image artifact index can be configured for the image side, such as 1.0 to 3.0: when the image artifact index is below the first threshold (e.g., 0.3), the magnification factor is close to 1.0, maintaining the original image uncertainty band width unchanged or slightly reduced; when the image artifact index is above the second threshold (e.g., 0.7), the magnification factor is close to 3.0, significantly increasing the image uncertainty band width, with the intermediate range smoothly transitioning using linear interpolation or piecewise interpolation. A similar magnification or reduction factor range can be configured for the point cloud side, such as 0.7 to 1.5: when the point cloud artifact index is low, the point cloud uncertainty band width can be moderately reduced to reflect its superior accuracy; when the point cloud artifact index is high, the bandwidth can be moderately increased to reflect the instability of the ranging pattern. After coefficient adjustment, upper and lower limits can be imposed on the bandwidth of the image side and the point cloud side respectively. For example, the minimum bandwidth should not be less than one pixel or one point cloud sampling interval, and the maximum bandwidth should not exceed a certain proportion of the typical target size in the scene, so as to avoid bandwidth distortion caused by extreme artifact index.

[0173] After completing the bandwidth adjustment based on the artifact index, the mapping and normalization rules in S104 can be used to convert the adjusted image bandwidth and the adjusted point cloud bandwidth into image confidence weights and point cloud confidence weights.

[0174] For example, unnormalized confidence scores can be calculated for both sides of the bandwidth first, for example, using a lookup function that is the inverse of the bandwidth or decreases by the bandwidth. Then, the scores are normalized so that the sum of the image confidence weight and the point cloud confidence weight is 1. For data sources with a large artifact index, their confidence scores will be significantly reduced because their bandwidth is amplified, thus automatically reducing their influence in the adversarial boundary section. For data sources with a small artifact index and signal morphology close to the ideal boundary, the bandwidth adjustment is smaller, and the confidence weight can be maintained at a relatively high level. The above brightness profile statistics, echo feature statistics, and weight calculation processes can be automated on a CPU or GPU, and array operations can be performed using NumPy in a Python environment.

[0175] Based on the same inventive concept, this application also provides an urban digital twin modeling system based on multi-source heterogeneous data fusion, which corresponds to the urban digital twin modeling method based on multi-source heterogeneous data fusion. Since the principle of the system in this application is similar to the urban digital twin modeling method based on multi-source heterogeneous data fusion described above in this application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0176] Reference Figure 4 The diagram shown is a schematic of a city digital twin modeling system based on multi-source heterogeneous data fusion provided in an embodiment of this application. The system includes:

[0177] The acquisition module 10 is used to acquire the registration image and registration point cloud corresponding to the target area;

[0178] The generation module 20 is used to extract image boundary candidates based on the registered image, generate image uncertainty bands for each image boundary candidate, and determine the width of the image uncertainty band and the image offset direction identifier; and to extract point cloud boundary candidates based on the registered point cloud, generate point cloud uncertainty bands for each point cloud boundary candidate, and determine the width of the point cloud uncertainty band and the point cloud offset direction identifier.

[0179] Processing module 30 is used to form boundary matching pairs based on the spatial intersection relationship between image uncertainty bands and point cloud uncertainty bands, and to determine the intersection segments with opposite offset directions as a set of adversarial boundary segments; for each adversarial boundary segment in the set of adversarial boundary segments, to determine the image confidence weight and the point cloud confidence weight based on the corresponding image uncertainty band width and point cloud uncertainty band width.

[0180] The verification module 40 is used to generate a boundary decision result for the image boundary candidate and point cloud boundary candidate corresponding to the adversarial boundary segment based on the image confidence weight and point cloud confidence weight, and to perform a topology consistency verification on the boundary decision result. Under the condition of satisfying the preset topology constraint conditions, the boundary decision result that passes the topology consistency verification is determined as the unique fusion boundary corresponding to the adversarial boundary segment.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for urban digital twin modeling based on multi-source heterogeneous data fusion, characterized in that, include: Acquire the registration image and registration point cloud corresponding to the target area; Based on the registered image, image boundary candidates are extracted, image uncertainty bands are generated for each image boundary candidate, and the width of the image uncertainty band and the image offset direction identifier are determined; the generation of the image uncertainty band includes: determining the corresponding normal direction according to the local tangential direction of the image boundary candidate, and constructing a strip-shaped region along the normal in the image domain as the image uncertainty band; The process of determining the image uncertainty band width and image offset direction identifier includes: obtaining the solar elevation angle and solar azimuth angle corresponding to the registered image, and obtaining the camera optical axis direction vector corresponding to the registered image; determining the local normal vector of the surface corresponding to the candidate image boundary based on the registered point cloud; calculating the geometric relationship parameters between the incident light direction and the observation direction based on the solar elevation angle and solar azimuth angle, the camera optical axis direction vector, and the local normal vector; obtaining the specular reflection coefficient associated with the candidate image boundary; inputting the geometric relationship parameters and the specular reflection coefficient into a preset specular expansion mapping rule to calculate the boundary expansion amount, and determining the image uncertainty band width based on the boundary expansion amount; and determining the specular reflection direction based on the geometric relationship parameters, and identifying the normal side away from the specular reflection direction as the image offset direction identifier; Based on the registered point cloud, point cloud boundary candidates are extracted, and point cloud uncertainty bands are generated for each point cloud boundary candidate. The width of the point cloud uncertainty band and the point cloud offset direction identifier are determined. The generation of point cloud uncertainty bands includes: taking the spatial trajectory of the point cloud boundary candidate as the center reference and constructing a three-dimensional strip or tubular region along the local normal direction of the point cloud boundary candidate as the point cloud uncertainty band. The determination of the uncertain band width and the point cloud offset direction identifier includes: acquiring laser acquisition parameters associated with point cloud points in the registered point cloud, the laser acquisition parameters including ranging values, incident beam direction information, echo identifiers, and divergence angle parameters; for each point cloud boundary candidate, determining the incident geometry relationship based on the incident beam direction information and the local normal of the point cloud boundary candidate, and determining the laser footprint scale based on the ranging values ​​and the divergence angle parameters; determining the point cloud uncertain band width based on the incident geometry relationship and the laser footprint scale; and determining the point cloud offset direction identifier based on the distribution difference of the echo identifiers on both sides of the normal of the point cloud boundary candidate. Boundary matching pairs are formed based on the spatial intersection relationship between the image uncertainty zone and the point cloud uncertainty zone, and the intersection segments with opposite offset directions are identified as the set of adversarial boundary segments. For each adversarial boundary segment in the set of adversarial boundary segments, the image confidence weight and the point cloud confidence weight are determined based on the corresponding image uncertainty band width and point cloud uncertainty band width. Based on the image confidence weight and the point cloud confidence weight, a boundary decision result is generated for the image boundary candidate and the point cloud boundary candidate corresponding to the adversarial boundary segment. A topology consistency check is performed on the boundary decision result. If the preset topology constraint conditions are met, the boundary decision result that passes the topology consistency check is determined as the unique fusion boundary corresponding to the adversarial boundary segment.

2. The method according to claim 1, characterized in that, The determination of the image uncertainty band width also includes: High-frequency component extraction is performed on the neighboring images of the image boundary candidate to obtain high-frequency texture components; virtual image edge scores are generated based on the energy distribution differences of the high-frequency texture components on both sides of the image boundary candidate in the normal direction. If the virtual image edge score meets the preset judgment conditions, the image uncertainty band width is extended and corrected based on the virtual image edge score to obtain the corrected image uncertainty band width; and the direction along the candidate normal of the image boundary, pointing to the side with lower high-frequency texture energy, is determined as the corrected image offset direction identifier.

3. The method according to claim 1, characterized in that, The step of obtaining the specular reflection coefficient associated with the image boundary candidate includes: Obtain a building information model that is spatiotemporally aligned with the target area; parse the semantic information data of the components in the building information model, and establish a mapping relationship between the image boundary candidates and the building components in the building information model through spatial indexing; Read the material property information of the building component, and query the preset physical material library based on the material property information to extract the corresponding specular reflection coefficient.

4. The method according to claim 1, characterized in that, The determination of the uncertain band width of the point cloud also includes: When the laser acquisition parameters include multi-echo ranging data of the same incident beam, the difference between the first echo ranging value and the last echo ranging value is calculated for the candidate neighborhood of the point cloud boundary. When the difference meets the preset difference judgment condition, the width of the uncertain band of the point cloud is increased according to the preset expansion rule; and based on the distribution of the number of points on both sides of the candidate normal of the point cloud boundary between the first echo point set and the last echo point set, the normal direction corresponding to the side with the higher number of points is determined as the point cloud offset direction identifier.

5. The method according to claim 4, characterized in that, The step of forming boundary matching pairs based on the spatial intersection relationship between image uncertainty bands and point cloud uncertainty bands includes: In three-dimensional space, an image-side search region corresponding to the image uncertainty zone and a point cloud-side search region corresponding to the point cloud uncertainty zone are constructed respectively. When the image-side search region and the point cloud-side search region meet the preset spatial intersection condition, the corresponding image boundary candidate and point cloud boundary candidate are determined as candidate matching pairs; The candidate matching pairs are subjected to visibility discrimination to filter out candidate matching pairs that do not meet the preset visibility conditions, and boundary matching pairs are obtained.

6. The method according to claim 5, characterized in that, The execution visibility determination includes: Based on the camera parameters of the registered image, the pixel positions within the uncertain band of the image are back-projected into a set of line-of-sight rays, and the image-side search region is formed based on the set of line-of-sight rays. Using the spatial trajectory of the point cloud boundary candidate as the center line, and determining the cross-sectional scale based on the width of the uncertain zone of the point cloud, the point cloud side search area is formed; For each candidate matching pair, occlusion point cloud located between the camera position and the search area on the side of the point cloud is retrieved in the registration point cloud along the line of sight ray set, and a visibility score is generated based on the occlusion point cloud; wherein, the occlusion point cloud is preferably composed of the first echo point set and / or the echo point set with smaller ranging value; When the visibility score meets the preset visibility condition, the candidate matching pair is retained as the boundary matching pair; when there are multiple boundary matching pairs corresponding to the same image boundary candidate, the boundary matching pair with the best visibility score is selected as the final boundary matching pair.

7. The method according to claim 4, characterized in that, The determination of image confidence weights and point cloud confidence weights by determining the bandwidth includes: Along the normal direction of the adversarial boundary segment, extract the brightness profile sequence that crosses the image boundary in the registered image, and extract the brightness overshoot and trailing asymmetry from the brightness profile sequence to generate the image artifact index. The echo distribution features of the neighborhood of the adversarial boundary segment are extracted from the registered point cloud, and the point cloud artifact index is generated based on the difference between the first echo ranging value and the last echo ranging value and / or the dispersion of the echo point set on both sides of the normal. Based on the image artifact index and the point cloud artifact index, the image uncertainty band width and the point cloud uncertainty band width are adjusted respectively through a preset penalty mapping rule to obtain the adjusted image bandwidth and the adjusted point cloud bandwidth. The image confidence weight and the point cloud confidence weight are calculated based on the adjusted image bandwidth and the adjusted point cloud bandwidth, and normalization processing is performed on the image confidence weight and the point cloud confidence weight.

8. A city digital twin modeling system based on multi-source heterogeneous data fusion, characterized in that, include: The acquisition module is used to acquire the registration image and registration point cloud corresponding to the target area; The generation module is used to extract image boundary candidates based on the registered image, generate image uncertainty bands for each image boundary candidate, and determine the width of the image uncertainty band and the image offset direction identifier; and to extract point cloud boundary candidates based on the registered point cloud, generate point cloud uncertainty bands for each point cloud boundary candidate, and determine the width of the point cloud uncertainty band and the point cloud offset direction identifier. The generation of the image uncertainty band includes: determining the corresponding normal direction based on the local tangential direction of the image boundary candidate, and constructing a band-shaped region along the normal in the image domain as the image uncertainty band; The process of determining the image uncertainty band width and image offset direction identifier includes: obtaining the solar elevation angle and solar azimuth angle corresponding to the registered image, and obtaining the camera optical axis direction vector corresponding to the registered image; determining the local normal vector of the surface corresponding to the candidate image boundary based on the registered point cloud; calculating the geometric relationship parameters between the incident light direction and the observation direction based on the solar elevation angle and solar azimuth angle, the camera optical axis direction vector, and the local normal vector; obtaining the specular reflection coefficient associated with the candidate image boundary; inputting the geometric relationship parameters and the specular reflection coefficient into a preset specular expansion mapping rule to calculate the boundary expansion amount, and determining the image uncertainty band width based on the boundary expansion amount; and determining the specular reflection direction based on the geometric relationship parameters, and identifying the normal side away from the specular reflection direction as the image offset direction identifier; The processing module is used to form boundary matching pairs based on the spatial intersection relationship between the image uncertainty band and the point cloud uncertainty band, and to determine the intersection segments with opposite offset directions as a set of adversarial boundary segments; for each adversarial boundary segment in the set of adversarial boundary segments, the image confidence weight and the point cloud confidence weight are determined based on the corresponding image uncertainty band width and the point cloud uncertainty band width. The generation of the point cloud uncertainty zone includes: taking the spatial trajectory of the point cloud boundary candidate as the center reference, and constructing a three-dimensional strip-shaped or tubular region along the local normal direction of the point cloud boundary candidate as the point cloud uncertainty zone; The determination of the uncertain band width and the point cloud offset direction identifier includes: acquiring laser acquisition parameters associated with point cloud points in the registered point cloud, the laser acquisition parameters including ranging values, incident beam direction information, echo identifiers, and divergence angle parameters; for each point cloud boundary candidate, determining the incident geometry relationship based on the incident beam direction information and the local normal of the point cloud boundary candidate, and determining the laser footprint scale based on the ranging values ​​and the divergence angle parameters; determining the point cloud uncertain band width based on the incident geometry relationship and the laser footprint scale; and determining the point cloud offset direction identifier based on the distribution difference of the echo identifiers on both sides of the normal of the point cloud boundary candidate. The verification module is used to generate boundary adjudication results for the image boundary candidates and point cloud boundary candidates corresponding to the adversarial boundary segment based on the image confidence weight and point cloud confidence weight, and to perform topology consistency verification on the boundary adjudication results. Under the condition of satisfying the preset topology constraints, the boundary adjudication results that pass the topology consistency verification are determined as the unique fusion boundary corresponding to the adversarial boundary segment.

Images