Real scene three-dimensional modeling method based on oblique photography and laser point cloud multi-source aerial triangulation fusion

Abstract

The invention relates to the technical field of three-dimensional modeling, and discloses an oblique photography and laser point cloud multi-source aerial triangulation fused live-action three-dimensional modeling method, which comprises the following steps: step S101, constructing a local elevation field and calculating a slope direction; step S102, extracting a vertex cracking chain; step S103, fitting a drift driving function; step S104, updating the pose and geometric coordinates of the camera; step S105, re-parameterizing the local texture parameter coordinates; and step S106, outputting the live-action three-dimensional model. The method adapts to canyon height difference and corridor weak geometric scenes, and the complementary advantages of oblique photography images and laser point clouds are coordinated through the coherent process of multi-source data space-time unification, structural feature extraction, geometric observation fitting, joint optimization and topological repair; and staggered layers in the slope direction caused by cross-modal drift are inhibited in a targeted manner, and grid topology cracks and vertex cracking are eliminated, so that the suitability of texture mapping and a geometric structure is improved.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and more specifically, to a real-scene 3D modeling method based on the fusion of oblique photogrammetry and multi-source aerial triangulation of laser point clouds. Background Technology

[0002] In the field of realistic 3D modeling, the fusion technology of oblique photogrammetry and laser point cloud has become a core means of obtaining high-precision scene models. Oblique photogrammetry, with its multi-view shooting advantage, can provide rich high-resolution texture information and clearly restore the details of the scene surface; while laser point cloud has strong geometric constraint capabilities, and can still accurately capture 3D structures in areas with weak texture, shadows, and repetitive textures. The complementarity of the two can significantly improve the integrity and expressiveness of the model, and is widely used in modeling tasks of complex terrains such as canyons and corridors.

[0003] However, canyon and corridor scenes possess significant unique characteristics: large elevation differences, clear spatial orientation, and elongated shapes, easily creating environments with weak geometric constraints. In these environments, scene feature points are sparsely distributed and difficult to match, resulting in insufficient time-bundle adjustment constraints for camera pose calculation and a tendency for system drift along the corridor direction. Existing fusion methods have obvious limitations, often using laser point clouds only as registration references or geometric hole-filling tools, without designing specific technical processes for the weak geometric characteristics of these scenes, nor establishing a correlation mechanism between structural anomalies and geometric constraints.

[0004] During cross-modal data fusion, the system drift of camera pose is further amplified, directly triggering a chain of problems. In the texture unfolding stage, drift leads to inconsistencies between mesh topology and texture mapping, resulting in dense seam bands, which in turn causes vertex fragmentation—that is, the same geometric vertex is copied into multiple instances in the texture parameter space. Fragmented vertices and dense seam edges are interconnected, forming geometric misalignments along the slope direction. Simultaneously, topological gaps are generated in the mesh, and the texture shows significant fragmentation due to vertex fragmentation. Furthermore, existing methods fail to transform these structural anomalies into quantifiable geometric observations, and the joint optimization process lacks targeted constraints, failing to effectively suppress drift solidification. The resulting models exhibit poor geometric continuity and insufficient texture consistency, making it difficult to meet the accuracy requirements of practical applications such as engineering measurement and scene visualization. Summary of the Invention

[0005] This invention provides a real-scene 3D modeling method that integrates oblique photography and multi-source aerial triangulation of laser point clouds, solving the technical problems mentioned in the background.

[0006] This invention provides a method for real-scene 3D modeling by fusing oblique photogrammetry and multi-source aerial triangulation of laser point clouds, comprising the following steps: Step S101: Acquire oblique photographic images, camera intrinsic parameters and laser point clouds, calculate the initial camera pose, unify the laser point clouds to the world coordinate system and construct a local elevation field, and calculate the slope aspect. Step S102: Generate a triangular mesh with texture parameter coordinates based on the initial camera pose. Based on the degree of cracking of the vertices in the texture parameter space, and combined with the orthogonality of the seam edge and the slope direction, extract the vertex cracking chain with the largest cumulative cracking energy. Step S103: Calculate the projection of the difference in centroids of the triangular facets on both sides of the joint edge in the vertex cracking chain onto the slope direction, obtain the slope shear jump observation, and establish a one-dimensional position sequence along the corridor direction, and fit a drift driving function containing second-order smooth regularity. Step S104: Construct a joint objective function that includes image reprojection error, laser point-to-surface residual and smoothing term. Superimpose the drift driving function along the slope onto the camera center to correct the pose. Perform joint solution to update the camera pose and geometric coordinates. Step S105: Determine the adaptive scale based on the median nearest neighbor distance of the laser point cloud, merge the spatially overlapping split vertex instances, and reparameterize the local texture parameter coordinates based on the consistency of the ratio between the 3D side length and the texture side length. Step S106: Based on the repaired mesh topology and local texture parameter coordinates, calculate the weighted weights of the image from each viewpoint on the face using the updated camera pose, perform multi-view texture fusion, and output a real-world 3D model.

[0007] The beneficial effects of this invention are as follows: This invention is suitable for canyon elevation differences and corridors with weak geometry. Through a coherent process of multi-source data spatiotemporal unification, structural feature extraction, geometric observation fitting, joint optimization, and topology repair, it coordinates the complementary advantages of oblique photogrammetric images and laser point clouds. It specifically suppresses slope-side layering caused by cross-modal drift, eliminates mesh topological gaps and vertex fragmentation, and improves the adaptability of texture mapping to geometric structures. The adaptive scale and proportion consistency design adapts to different data densities and local area requirements, avoiding unnecessary computational redundancy. The final output realistic 3D model has good geometric continuity and texture consistency, meeting the basic requirements for model quality in practical applications such as engineering measurement and scene visualization, while maintaining the coherence and reliability of the modeling process. Attached Figure Description

[0008] Figure 1 This is a flowchart of the real-scene 3D modeling method of multi-source aerial triangulation of oblique photography and laser point cloud according to the present invention; Figure 2 This is a schematic diagram of the computational scenario of the present invention. Detailed Implementation

[0009] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0010] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0011] like Figures 1-2 As shown, the method for real-world 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud includes the following steps: Step S101: Acquire oblique photographic images, camera intrinsic parameters and laser point clouds, calculate the initial camera pose, unify the laser point clouds to the world coordinate system and construct a local elevation field, and calculate the slope aspect. Step S102: Generate a triangular mesh with texture parameter coordinates based on the initial camera pose. Based on the degree of cracking of the vertices in the texture parameter space, and combined with the orthogonality of the seam edge and the slope direction, extract the vertex cracking chain with the largest cumulative cracking energy. Step S103: Calculate the projection of the difference in centroids of the triangular facets on both sides of the joint edge in the vertex cracking chain onto the slope direction, obtain the slope shear jump observation, and establish a one-dimensional position sequence along the corridor direction, and fit a drift driving function containing second-order smooth regularity. Step S104: Construct a joint objective function that includes image reprojection error, laser point-to-surface residual and smoothing term. Superimpose the drift driving function along the slope onto the camera center to correct the pose. Perform joint solution to update the camera pose and geometric coordinates. Step S105: Determine the adaptive scale based on the median nearest neighbor distance of the laser point cloud, merge the spatially overlapping split vertex instances, and reparameterize the local texture parameter coordinates based on the consistency of the ratio between the 3D side length and the texture side length. Step S106: Based on the repaired mesh topology and local texture parameter coordinates, calculate the weighted weights of the image from each viewpoint on the face using the updated camera pose, perform multi-view texture fusion, and output a real-world 3D model.

[0012] In one embodiment of the present invention, the process includes acquiring oblique photographic images, camera intrinsic parameters, and laser point clouds; calculating the initial camera pose; unifying the laser point clouds to the world coordinate system and constructing a local elevation field; and calculating the slope aspect. Acquire multiple oblique photographic images For each tilted photographic image Set the camera intrinsic parameter matrix ,in Number the images; acquire laser point clouds and record the three-dimensional coordinates of each laser point in the laser sensor coordinate system. ,in Number the laser points; For each oblique photographic image Perform bundle adjustment to establish the three-dimensional coordinates of any spatial point. Rather than this tilted photographic image Image point homogeneous image plane coordinates Projection relationship between them: in As a proportional scalar, For rotation matrix, The translation vector is obtained by minimizing the projection residuals of all image points. and , by set To establish the initial camera pose; Record the sampling time for each laser point. Determine the carrier attitude rotation matrix in the carrier coordinate system at this sampling moment. Translation vector of the carrier Determine the fixed rotation matrix of the laser sensor relative to the carrier. With a fixed translation vector By transforming each laser point to the world coordinate system according to the rigid body transformation relationship, the three-dimensional coordinates of the laser point in the world coordinate system are obtained. : In the world coordinate system, a local subset of the laser point cloud is selected, and the coordinates of the points within the local subset are denoted as follows: Representing the local elevation field using plane functions : Solving for the coefficients of a plane function using least squares constraints , , : According to the local elevation field Determine the horizontal gradient vector : The horizontal gradient vector Normalization yields the unit vector of slope aspect on the horizontal plane. : The slope is the unit vector on the horizontal plane. Extending to three-dimensional space, we obtain the slope aspect. : in and Slope direction In the world coordinate system direction and Components in direction.

[0013] It should be noted that oblique photogrammetry images are ground scene images acquired by shooting at an oblique angle, reflecting the texture and geometric appearance of ground objects; they can be collected by drones, manned aircraft, or ground-based mobile devices equipped with oblique photogrammetry cameras. The camera intrinsic parameter matrix is ​​a matrix describing the optical characteristics and imaging geometry of the camera, reflecting core parameters such as focal length and principal point coordinates; for example, a focal length of 5mm to 50mm and principal point coordinates corresponding to the center position of the image resolution can be calculated using camera calibration software combined with data captured by a calibration board, which will not be elaborated here. Laser point clouds are massive sets of three-dimensional points acquired after a laser sensor emits a laser beam to scan a target; they can be collected by LiDAR sensors mounted on drones, vehicles, or fixed platforms. The three-dimensional coordinates of a laser point in the laser sensor coordinate system are the spatial position coordinates of the laser point relative to the origin of the laser sensor, reflecting the relative positional relationship between the laser point and the sensor. The three-dimensional coordinates of a spatial point are the position coordinates of any point in space in the world coordinate system, reflecting the absolute spatial position of that point. The homogeneous image plane coordinates of an image point are the homogeneous coordinate representation of the spatial point after it is imaged on the oblique photogrammetry image, reflecting the projected position of the spatial point on the image. The projection residual is the difference between the coordinates of a point projected onto the image and its actual image coordinates, reflecting the fitting accuracy of the projection relationship. The rotation matrix is ​​an orthogonal matrix describing the rotation relationship between coordinate systems, reflecting the attitude orientation of the camera or platform. The translation vector is a vector describing the translation relationship between coordinate systems, reflecting the positional offset of the camera or platform. The initial camera pose is the position and attitude information of the camera during imaging, composed of the rotation matrix and the translation vector, reflecting the spatial state during camera capture.

[0014] It should be noted that the laser point sampling time is the specific time when the laser sensor records the laser point, reflecting the acquisition sequence of the laser point; it can be recorded through the time synchronization module of the lidar sensor. The carrier attitude rotation matrix in the carrier coordinate system is the rotation matrix of the carrier coordinate system relative to the reference coordinate system at the sampling time, reflecting the carrier's attitude state; it can be obtained through the fusion calculation of the inertial measurement unit (IMU) and the global positioning system (GPS) on the carrier. The carrier translation vector in the carrier coordinate system is the translation vector of the origin of the carrier coordinate system in the reference coordinate system at the sampling time, reflecting the carrier's position state. The fixed rotation matrix of the laser sensor relative to the carrier is the fixed rotation relationship matrix between the laser sensor coordinate system and the carrier coordinate system, reflecting the installation attitude of the sensor and the carrier; the preferred value is an orthogonal matrix determined based on the sensor installation calibration data; it can be obtained through calibration experiments after sensor installation. The fixed translation vector of the laser sensor relative to the carrier is the fixed translation relationship vector between the origin of the laser sensor coordinate system and the origin of the carrier coordinate system, reflecting the installation position of the sensor and the carrier; the preferred value is a vector determined based on the sensor installation dimensions; it can be obtained by measuring the relative position of the sensor and the carrier origin using a ruler or laser rangefinder. The three-dimensional coordinates of a laser point in the world coordinate system represent its absolute position within the unified world coordinate system, reflecting its global spatial location. A plane function is a linear function used to approximate local terrain elevation, reflecting the local terrain's tilt trend. The coefficients of the plane function are the coefficients of x and y, along with a constant term, reflecting the plane's tilt and elevation datum. The local elevation field is the elevation distribution of a local area represented by a plane function, reflecting the local terrain's elevation variation patterns. The horizontal gradient vector reflects the terrain's tilt direction and steepness on the horizontal plane. The unit vector of slope aspect on the horizontal plane is the normalized vector of the horizontal gradient vector, reflecting the projection direction of the terrain's slope aspect onto the horizontal plane. Slope aspect is the direction vector of terrain tilt in three-dimensional space, reflecting the actual tilt orientation of the terrain.

[0015] It should be noted that the selection criteria for the local subset of the laser point cloud are as follows: all laser points within a 5-meter spatial distance from the target laser point are selected; if there are fewer than 15 points within 5 meters, the 15 closest laser points are selected; if there are more than 20, the 20 closest laser points are selected. This ensures that the local subset reflects the local terrain features while avoiding computational redundancy due to an excessive number of points. The least squares constraint solution for the plane function coefficients can use a weighted least squares method. The weight is the reciprocal of the distance from the laser point to the center of the local subset; points closer to the center have larger weights, reducing the influence of edge noise points. During the solution process, a normal equation system is constructed, and the coefficients a, b, and c are obtained using matrix inversion or iterative methods. For example, for 15 laser points, a 3×3 normal equation matrix is ​​constructed, and the coefficient values ​​are obtained after solving. This will not be elaborated further here.

[0016] It should be noted that this invention provides a unified spatiotemporal reference and basic geometric parameters for the fusion modeling of multi-source oblique photogrammetry and laser point clouds. By collecting raw data from multiple sources, the initial camera pose is calculated to establish image imaging relationships. The laser point clouds are unified to the world coordinate system to achieve spatiotemporal data alignment. A local elevation field is constructed and the slope aspect is calculated to extract core geometric features of the terrain. This allows oblique photogrammetry images and laser point clouds to be calculated collaboratively within the same coordinate framework, avoiding fusion errors caused by inconsistent multi-source data references. The construction of the local elevation field provides a quantifiable geometric basis for subsequent extraction of slope-related structural features, while the slope aspect calculation accurately reflects the terrain's tilt direction, laying the foundation for solving the problem of slope-related layering in canyon areas. This ensures that subsequent modeling operations can be carried out based on consistent geometric and terrain information, improving the coherence and reliability of the overall modeling process.

[0017] In one embodiment of the present invention, a triangular mesh with texture parameter coordinates is generated based on the initial camera pose. Based on the degree of fission of the vertices in the texture parameter space, and combined with the orthogonality of the seam edges and slope direction, the vertex fission chain with the largest cumulative fission energy is extracted, including: Using the initial camera pose set With oblique photography images A dense 3D point set is obtained through multi-view geometric reconstruction, and a triangular mesh is constructed on the dense 3D point set to record the vertex set. With triangular facet set For the vertex set Each vertex in Calculate three-dimensional coordinates It is expanded into each vertex through projection and texture unfolding. Assign coordinates to texture parameters This yields a triangular mesh with texture parameter coordinates; For each vertex Count the number of instances of this vertex in the texture parameter space. Count the number of adjacent triangles of this vertex in the geometric mesh. Define the degree of fission of vertices in the texture parameter space. : in For the number of instances, This represents the number of adjacent triangular faces. Add the edges located on the boundaries of different tiles in the texture parameter map set to the seam edge set. For any seam edge Define a three-dimensional direction vector : in and Vertices With vertex The three-dimensional coordinates; using the three-dimensional direction vector With slope vector Calculate the orthogonality coefficient between the joint edge and the slope direction. : in Represents the vector dot product. Indicates the magnitude of the vector; Butt joint edge assembly Each seam edge The cracking energy is defined by multiplying the mean cracking degree at both ends of this joint edge with the orthogonality coefficient. : in and The degree of fission at both vertices; set at the seam edge. Select the set of connected seam edges Calculate the set of connected seam edges The cumulative cracking energy is obtained by summing the cracking energies of all joint edges. : Determine the cumulative cracking energy from the set of all connected joint edges. Largest set The set with the largest cumulative cracking energy This forms the vertex cracking chain with the highest cumulative cracking energy.

[0018] It should be noted that a dense 3D point set is a high-density spatial point set obtained through multi-view geometry reconstruction. A triangular mesh is a network of triangular facets formed by connecting dense 3D point sets. Texture parameter coordinates are the coordinates that map 3D vertices to a 2D texture plane, reflecting the correspondence between vertices and texture images. A triangular mesh with texture parameter coordinates is a triangular mesh with a 2D texture mapping relationship, reflecting the association between 3D geometry and 2D texture. The number of vertex instances in texture parameter space is the number of times the same geometric vertex is copied in texture parameter space, reflecting the degree of splitting of the vertex in texture mapping. The number of adjacent triangular facets of a vertex in the geometric mesh is the number of triangular facets directly connected to that vertex, reflecting the connection strength of the vertex in the geometric topology. Vertex splitting is the ratio of the number of instances in texture parameter space to the number of adjacent triangular facets in geometry, reflecting the degree of inconsistency between the vertex in texture and geometric topology. A texture parameter atlas is a 2D texture set integrating multiple texture tiles, reflecting the overall arrangement of textures in a 3D scene; preferably, it is a set of texture images with a size of 2048×2048 or 4096×4096. A tile is an independent texture region in the texture parameter set, reflecting a texture fragment of a local surface in a 3D scene; it is preferably a regularly shaped rectangular region with a size of not less than 256×256 pixels. A seam edge set is the collective term for the boundary edges of different tiles in the texture parameter set, reflecting the connection boundaries between texture tiles. A seam edge is an edge on the boundary of a texture tile, reflecting the connection position between two texture tiles. A 3D direction vector is a vector describing the extension direction of a seam edge in 3D space, reflecting the spatial orientation of the seam edge. The orthogonality coefficient is an index measuring the degree to which the 3D direction of a seam edge is perpendicular to the slope vector, reflecting the degree of conformity between the seam edge and the contour line direction. Fission energy is the product of the average fission degree at both vertices of a seam edge and the orthogonality coefficient, reflecting the structural anomaly intensity of the seam edge. A connected seam edge set is a set of topologically connected seam edges, reflecting continuous structural anomaly regions. Cumulative fission energy is the sum of the fission energies of all seam edges in the connected seam edge set, reflecting the overall structural anomaly degree of the set. The vertex fission chain with the highest cumulative fission energy is a chain-like structure composed of the set of connected seam edges with the highest cumulative fission energy, reflecting the most significant structural anomaly region in the scene.

[0019] It should be noted that the more instances there are in the texture parameter space, the more severe the splitting of vertices due to texture seams. In contrast, the number of adjacent triangles in the geometric mesh is an inherent topological property of the vertex and is not affected by texture mapping. The ratio of the two can objectively quantify the inconsistency of vertices at the texture and geometric levels, avoiding the problem that a single indicator cannot accurately reflect structural anomalies. For example, if a geometric vertex is adjacent to 4 triangles and is split into 2 instances in the texture space, the splitting degree is 0.5, indicating a slight splitting. If it is split into 4 instances, the splitting degree is 1, indicating a severe splitting. This design can accurately distinguish different degrees of structural anomalies. Vertex fission directly reflects the degree of local structural splitting, while the orthogonality coefficient reflects the degree of fit between the joint edge and the contour lines of the canyon terrain. The product of the two can highlight joint edges that are severely split and related to the main direction of the terrain. This correlation is not a simple superposition, but rather strengthens the core features through multiplication, making joint edges with high fission energy the key carriers reflecting cross-modal drift. For example, the average fission degree of the two vertices of a joint edge is 0.8, the orthogonality coefficient is 0.9, and the fission energy is 0.72, which is much higher than that of a joint edge with a fission degree of 0.8 but an orthogonality coefficient of 0.3 (energy 0.24), effectively screening out core joint edges related to the terrain.

[0020] It should be noted that the multi-view geometric reconstruction adopts the Feature Matching-based Structure of Motion (SFM) algorithm. First, feature points of each image are extracted using Scale Invariant Feature Transform (SIFT). Then, mismatched feature points are removed using the Random Sample Consensus (RANSAC) algorithm. Finally, a dense 3D point set is obtained through bundle adjustment. The point cloud density threshold is set to 5 to 10 points per square meter to ensure that the point cloud reflects detailed geometry without generating redundant computation. The projection and texture unfolding operation process specifically includes: First, the 3D vertices are projected onto multiple oblique photographic images using camera intrinsics and initial pose, and the pixel coordinates of each vertex on each image are recorded. Second, a greedy parameterization algorithm is used to divide the 3D mesh surface into multiple continuous local regions. Third, the vertices of each local region are projected onto the corresponding local image region to obtain 2D texture parameter coordinates. Fourth, the coordinates of overlapping regions are smoothed to ensure that the texture mapping has no significant distortion.

[0021] It should be noted that the instance determination criterion is that if the difference in texture parameter coordinates of vertices is greater than 1 pixel, they are considered different instances. During the statistical analysis, minor coordinate differences caused by repeated projections are excluded; if the difference between two coordinates is less than 1 pixel, they are considered the same instance. Tile division uses a grid curvature-based algorithm, grouping regions with similar curvature in the 3D grid into a single tile. The tile shape is rectangular, with dimensions between 256×256 pixels and 1024×1024 pixels. During division, it is ensured that the 3D region corresponding to each tile is continuous, and the overlapping area between tiles does not exceed 5 pixels to avoid texture seams. The seam edge determination criterion is that if the texture parameter coordinates of the vertices on both sides of an edge belong to different tile numbers, and the difference in grayscale values ​​between the two textures is greater than 30 (0 to 255 grayscale range), then it is considered a seam edge. The boundary gap threshold is set to 1 pixel; if the gap between the texture coordinates of the vertices on both sides of an edge is greater than 1 pixel, it is also considered a seam edge. In addition, connectivity is defined as edges sharing a vertex; the minimum number of edges is set to 5. If the number of seam edges in a connected set is less than 5, it is considered an invalid set and will not participate in the cumulative fission energy calculation to avoid interference from short-distance isolated anomalies.

[0022] It should be noted that this invention transforms the geometric misalignment problem caused by cross-modal drift into structural anomaly features of texture and geometry. By generating triangular meshes with texture parameter coordinates, defining vertex fission degree to quantify the degree of structural splitting, and combining the orthogonality of seam edges and slope aspect to screen core anomaly edges, the invention ultimately extracts the vertex fission chain with the largest cumulative fission energy. This transforms the originally difficult-to-locate geometric misalignment into quantifiable structural features. The combination of fission degree and orthogonality can accurately screen core anomaly regions related to canyon topography, avoiding interference from irrelevant structural noise. The calculation of cumulative fission energy highlights the most significant anomaly chain overall, thus providing a precise carrier for subsequently transforming structural anomalies into geometric observations.

[0023] In one embodiment of the present invention, the projection of the centroid difference between the two triangular facets on both sides of the joint edge in the vertex splitting chain onto the slope direction is calculated to obtain slope shear jump observations, and a one-dimensional position sequence along the corridor direction is established. A drift driving function containing second-order smooth regularization is fitted, including: For the set of vertex fission chains Every seam edge In the set of triangular facets The two triangular facets adjacent to this seam edge are determined in the middle. and triangular pieces The three-dimensional coordinates of the three vertices are denoted as , , Define the centroid : triangular pieces The three-dimensional coordinates of the three vertices are denoted as , , Define the centroid : Using slope vector Calculate the seam edge Slope shear jump observation : in Represents the vector dot product. Represents the magnitude of the aspect vector; Using laser point clouds in world coordinate system Calculate the centroid of laser point cloud : in The number of laser points in the laser point cloud; constructing the covariance matrix using the laser point cloud. : For covariance matrix Perform eigenvalue decomposition and select the eigenvector corresponding to the largest eigenvalue. Calculate the unit vector of the corridor direction. : For the set of vertex fission chains Every seam edge Calculate the midpoint : Define seam edge One-dimensional position of the corridor direction : In one-dimensional position and corresponding slope aspect shear jump observations Constructing the target functional : in For drift driving function, For the drift driving function with respect to one-dimensional position The second derivative, The weights are smooth and regularized; the objective functional is minimized. Determine the drift driving function .

[0024] It should be noted that the slope aspect shear jump observation is the projection of the difference in centroids of the triangular facets on both sides of the joint edge onto the slope aspect, reflecting the degree of discrete layering along the slope aspect. The centroid of the laser point cloud is the arithmetic mean of the three-dimensional coordinates of all laser points, reflecting the overall spatial center position of the laser point cloud. The covariance matrix is ​​a symmetric matrix describing the correlation between the coordinate dimensions of the laser point cloud, reflecting the spatial distribution characteristics of the point cloud. The eigenvector is an orthogonal vector obtained after eigenvalue decomposition of the covariance matrix, reflecting the principal direction of the point cloud distribution. The unit vector of the corridor direction is the normalized vector of the maximum principal direction of the laser point cloud distribution, reflecting the extension direction of the canyon corridor. The difference vector is the coordinate difference vector between the midpoint of the joint edge and the centroid of the laser point cloud, reflecting the positional offset of the joint edge relative to the centroid of the point cloud. The one-dimensional position of the joint edge in the corridor direction is the inner product of the difference vector and the unit vector of the corridor direction, reflecting the linear position of the joint edge along the corridor. The objective functional is a functional expression that includes data fitting and smoothness constraints, reflecting the fitting accuracy and smoothness requirements of the drift driving function. The data fitting term is the sum of the squared differences between the slope shear jump observations and the values ​​of the drift driving function, reflecting the degree to which the function fits the observed data. The second-order smoothing regularization term is the product of the smoothing regularization weight coefficient and the square integral of the second derivative of the drift driving function, reflecting the strength of the curvature constraint on the function. The smoothing regularization weight coefficient is a parameter that adjusts the strength of the smoothing constraint, reflecting the balance between data fitting and function smoothness; its preferred value is between 0.001 and 0.01. The drift driving function is a continuous function describing the change in drift along the corridor direction, reflecting the spatial distribution of cross-modal drift. The second derivative of the drift driving function with respect to one-dimensional position reflects the rate of change of drift.

[0025] It should be noted that the vertex fission chain is the carrier of structural anomalies. The difference in centroids of the triangular facets on both sides of its seam edge reflects local geometric misalignment. Projecting this difference onto the slope aspect allows for the precise extraction of the misalignment component along the slope aspect, eliminating interference from misalignments in other directions, thus transforming the originally abstract misalignment phenomenon into clear observational data. The main distribution direction of the laser point cloud is consistent with the direction of the canyon corridor. By calculating the centroid of the point cloud, constructing the covariance matrix, and performing eigenvalue decomposition, the eigenvector corresponding to the largest eigenvalue is the corridor direction. This method does not require manual definition of the corridor direction and is entirely based on data extraction. Projecting the midpoint of the seam edge onto this direction yields a one-dimensional position, ensuring that the seam edges in three-dimensional space are arranged in an orderly manner according to the corridor extension direction, providing a linear coordinate basis for fitting the continuous drift function. The objective functional includes both a data fitting term and a second-order smoothing regularization term. The data fitting term ensures that the function fits the observation data, while the second-order smoothing regularization term constrains the curvature of the function, preventing drastic fluctuations caused by observation noise. By minimizing this functional, the resulting drift-driving function reflects the spatial variation of the actual drift and possesses good continuity and rationality, which can be directly used for subsequent orientation correction of camera pose. For example, if only data fitting is considered, the function may fluctuate frequently and fail to reflect the true drift pattern. After adding the second-order smoothing regularization, the function becomes smooth and fits the observation data, ensuring the effectiveness of drift correction.

[0026] It should be noted that the covariance matrix can be constructed using an equal-weighting method, without assigning additional weights to the laser points. If there is significant noise in the laser points (e.g., a single point is too far from other points), a distance threshold can be used to remove noisy points. The threshold is set to three times the median nearest neighbor distance of the laser point cloud to ensure that the matrix reflects the true distribution characteristics of the point cloud. Singular Value Decomposition (SVD) can be used for eigenvalue decomposition. All non-zero eigenvalues ​​are retained during eigenvalue calculation, and the eigenvector corresponding to the largest eigenvalue is selected as the corridor direction reference to ensure the accuracy of the main direction extraction. The trapezoidal integral method in numerical integration can be used, with the integration step size set to the minimum interval of the one-dimensional position sequence of the corridor. The integration range covers the maximum and minimum values ​​of the one-dimensional positions of all seam edges, ensuring that the integration result accurately reflects the curvature constraint of the function. Furthermore, the value of the smoothing regularization weight coefficient needs to be considered in conjunction with the noise level of the slope shear jump observation. If the observed data has high noise, the weight coefficient is set to 0.01 to enhance the smoothing constraint; if the observed data has high precision, the weight coefficient is set to 0.001 to prioritize data fitting. In practical applications, the optimal value can be determined through cross-validation, which will not be elaborated here.

[0027] It should be noted that this invention transforms the structural anomalies reflected by the vertex fission chain into quantitative geometric observations, establishes a one-dimensional position sequence along the corridor, and fits a continuous drift driving function using a target functional with smooth constraints, providing an accurate drift model for subsequent pose correction. Core layering information is extracted through slope shear jump observations, eliminating irrelevant interference and ensuring the relevance of the observation data; the corridor orientation is autonomously extracted based on laser point clouds, achieving objective ordering of the position sequence; a second-order smooth regularization term ensures the rationality of the drift function and avoids noise influence. Thus, cross-modal drift is transformed from discrete observations into a continuous function, clarifying the spatial distribution law of the drift and providing a reliable basis for subsequent directional correction of camera pose.

[0028] In one embodiment of the present invention, a joint objective function is constructed, comprising image reprojection error, laser point-to-surface residual, and smoothing term. The drift driving function is superimposed along the slope direction onto the camera center to correct the pose. Joint solution is then performed to update the camera pose and geometric coordinates, including: For the initial camera pose set For each camera in the dataset, calculate the camera center in the world coordinate system. : in For the first Rotation matrix of each camera, Let this be the translation vector of the camera; use the unit vector of the corridor direction. With the centroid of laser point cloud Calculate the one-dimensional position of the camera center along the corridor direction. : Using drift drive function With slope vector Calculate the corrected camera center : in Let the magnitude of the slope vector be used; using the corrected camera center. Update translation vector : Depend on and This results in the updated camera pose. Using camera intrinsic matrix With the updated camera pose, the geometric coordinates of the 3D points Calculate projected image points : in This involves operations to normalize 3D camera coordinates to the image plane; and calculating image reprojection error. : in Let these be the homogeneous coordinates of the image point. It is a two-dimensional Euclidean norm; Using laser point clouds in world coordinate system For each laser point Associated local plane parameter vector ,in For unit normal vector, As a constant term, calculate the residual from the laser point to the surface. : Using drift drive function The second derivative Constructing smooth items : in The smoothing regularization weighting coefficients are used to calculate the image reprojection error. Laser point to surface residual With smooth terms Combine to form a joint objective function : in These are the weighting coefficients; through the joint objective function Minimize the value to obtain the updated set of camera poses. With the updated set of 3D point geometric coordinates .

[0029] It should be noted that the updated camera translation vector is the translation vector obtained by back-calculating after correcting the camera center. The updated camera pose is the combination of the camera rotation matrix and the updated translation vector. The one-dimensional position of the camera center in the corridor direction is the linear coordinate of the camera center projected onto the unit vector of the corridor direction, reflecting the ordered position of the camera along the corridor. The corrected camera center is the coordinate of the original camera center superimposed with the contribution of the drift driving function along the slope. The projected image point is the pixel of the 3D point projected onto the image through the camera intrinsic parameters and the updated pose. The image reprojection error is the sum of squares of the differences between the homogeneous coordinates of the image point and the projected image point. The local plane parameter vector is the combination of the unit normal vector describing the local plane and a constant term, reflecting the geometric characteristics of the local plane. The smoothing term is the product of the smoothing regularization weight coefficient and the square integral of the second derivative of the drift driving function, reflecting the curvature constraint strength of the drift function. The joint objective function is a weighted combination of the image reprojection error, the laser point-to-surface residual, and the smoothing term, reflecting the overall optimization objective under multi-source constraints. The weighting coefficient is a parameter that balances the contributions of multi-source residuals and reflects the importance of different modal data; the preferred value is 0.1 to 1.0.

[0030] It should be noted that the drift driving function quantifies the drift distribution along the slope direction. Superimposing it onto the camera center along the slope direction can directly and specifically correct the camera's positional deviation in the layering direction. The camera center is a core feature of the pose; after correction, the translation vector is derived in reverse, ensuring the self-consistency of pose rotation and translation and avoiding conflicts between the two. Image reprojection error ensures the consistency of texture and geometry, laser point-to-surface residuals strengthen geometric constraints, and the smoothing term ensures the rationality of the drift function. The weighted combination of these three achieves collaborative constraints on multi-source data, enabling optimization to simultaneously meet the accuracy requirements of image texture and laser point geometry. In addition, synchronously updating the camera pose and 3D point coordinates can avoid error propagation caused by single optimization. The camera pose affects the accuracy of 3D point projection, and the 3D point coordinates determine the effectiveness of geometric constraints. The coordinated optimization of the two allows the constraints of the image and laser points to be mutually adapted, ultimately resulting in a globally consistent model.

[0031] It should be noted that the local plane parameter vector can be obtained using the k-nearest neighbor algorithm. For each laser point, 15 to 20 neighboring laser points are selected, and the unit normal vector and constant term of the local plane are obtained through least squares fitting. The 3D point geometric coordinates are first converted to coordinates in the camera coordinate system through the updated camera rotation matrix, and then added to the updated translation vector to obtain the camera coordinates. The camera coordinates are then converted to homogeneous coordinates of the image plane through the camera intrinsic parameter matrix, and finally divided by the homogeneous term to obtain the 2D projected image point. The weight coefficients are determined based on the relative accuracy of the laser point cloud. When the laser point cloud density is greater than 5 points per square meter and the plane fitting error is less than 0.05 meters, the weight coefficients are taken as 0.5 to 1.0; when the laser point cloud is sparse or the accuracy is low, they are taken as 0.1 to 0.5. In practical applications, the values ​​can be determined through cross-validation to select the values ​​that best match the consistency between the model texture and geometry, which will not be elaborated here.

[0032] In one embodiment of the present invention, an adaptive scale is determined based on the median nearest neighbor distance of the laser point cloud, spatially overlapping fragmented vertex instances are merged, and local texture parameter coordinates are reparameterized based on the consistency of the ratio between the 3D side length and the texture side length, including: laser point cloud in world coordinate system Each laser point in Calculate its nearest neighbor distance to other laser points. : in For three-dimensional Euclidean distance; for all The median of the nearest neighbor distance in the laser point cloud is obtained by calculating the median. : Median nearest neighbor distance using laser point cloud Determine the adaptive scale : in It is a fixed proportionality coefficient; In a triangular mesh with texture parameter coordinates, for each geometric vertex Fission instance set Any pair of instances Calculate 3D distance : in For example The three-dimensional coordinates; when At that time, the instance With examples Merged into a single geometric vertex Calculate the single geometric vertex after merging. 3D coordinates : and originally belong to the instance With examples The local triangular facets are all associated with a single geometric vertex after merging in the geometry. ; Selecting a single geometric vertex that includes the merged vertices Local triangular facet set Local triangular facet set The set of edges in the set is denoted as For the edge set Each edge in Calculate the local scaling factor : in and As vertices With vertex Local texture parameter coordinates, and As vertices With vertex 3D coordinates; constructing a local texture parameter coordinate reparameterization objective function : The objective function is reparameterized by minimizing the local texture parameter coordinates. Solving for local texture parameter coordinates .

[0033] It should be noted that the nearest neighbor distance of the laser point cloud is the minimum spatial distance from a single laser point to other laser points, reflecting the local density of the laser point cloud. The median nearest neighbor distance of the laser point cloud is the median value of the nearest neighbor distances of all laser points, reflecting the overall distribution density characteristics of the laser point cloud. The fixed scaling factor is an adjustment parameter used to determine the adaptive scale, reflecting the spatial tolerance of splitting vertex merging; a value of 2.0 is preferred. The adaptive scale is the product of the median nearest neighbor distance of the laser point cloud and the fixed scaling factor, reflecting the merging threshold for splitting vertex instances. A splitting instance pair is a combination of any two instances in the set of splitting instances of geometric vertices. The 3D distance is the spatial straight-line distance between splitting instance pairs, reflecting the spatial overlap of instances. The merged single geometric vertex is the unified vertex after merging spatially overlapping splitting instance pairs. The 3D coordinates of the merged single geometric vertex are the average of the 3D coordinates of the splitting instance pairs. The edges in the set of local triangular patches are the boundary edges of the triangular patches in that local region, reflecting the connectivity of the local topology. The texture edge length is the distance between the two vertices of an edge in the texture parameter space, reflecting the local scale of the texture mapping. The 3D edge length is the distance between the two vertices of an edge in 3D space, reflecting the local scale of the geometry. The local scale factor is the ratio of the sum of texture edge lengths to the sum of 3D edge lengths within a local region, reflecting the scale matching relationship between texture and geometry. The objective function for reparameterizing local texture parameter coordinates measures the difference between the texture edge length and the scaled 3D edge length, reflecting the optimization objective of reparameterization; its preferred value is the sum of the squares of the difference between the texture edge length and the local scale factor multiplied by the 3D edge length. The reparameterized local texture parameter coordinates are the optimized texture parameter coordinates, reflecting the mapping relationship between texture and geometric scale matching.

[0034] It's important to note that maintaining the consistency between the texture edge length and the 3D edge length is crucial for avoiding texture distortion. By calculating a local scaling factor, the reparameterized texture scale can be adapted to the geometric scale. The objective function isn't simply about achieving equal texture edge lengths; rather, it establishes a relationship between the two through a scaling factor, ensuring uniform scaling of the texture mapping within local regions. Furthermore, merging spatially overlapping fractured vertices first eliminates structural gaps at the topological level, laying a unified geometric foundation for subsequent texture reparameterization. Then, performing texture reparameterization on local regions avoids error propagation caused by global reparameterization, while focusing on affected areas to improve efficiency. This ensures that the repair process addresses both geometric structural issues and texture mapping quality.

[0035] It should be noted that the preferred value range for the fixed scaling factor is 1.5 to 2.5. When point cloud noise is low, a value of 1.5 to 2.0 is used to avoid excessive merging; when point cloud noise is high or fragmented instances are scattered, a value of 2.0 to 2.5 is used to ensure effective merging of overlapping instances. The division rule for fragmented instance sets is that the criteria for determining fragmented instances of the same geometric vertex are that the difference in 3D coordinates is less than 0.5 times the adaptive scale, and they share at least two identical triangular faces. During division, the initial index of the geometric vertices is used for grouping, and then the coordinate difference and topological connectivity are used for filtering to ensure that instances within the same set are all fragmentation products of that vertex. The selection range of the local triangular facet set is centered on the merged single geometric vertex, selecting directly connected triangular faces. If the number of connected faces is less than 10, it is expanded to second-order adjacent faces to ensure that the selected local area can cover the influence range of the merged vertex and avoid texture discontinuities caused by reparameterization being limited to only a small number of faces. This invention determines an adaptive scale based on the resolution of the laser point cloud itself, merges spatially overlapping fractured vertices to repair the mesh topology, and then reparameterizes the local texture parameter coordinates based on the scale consistency between texture and geometry to solve the problems of topological gaps and texture distortion.

[0036] In one embodiment of the present invention, based on the repaired mesh topology and local texture parameter coordinates, the weighted weights of the image from each viewpoint on the surface are calculated using the updated camera pose, multi-view texture fusion is performed, and a realistic 3D model is output, including: Using the repaired mesh topology, let the vertex set be... The set of triangular facets is Each vertex The three-dimensional coordinates are For the set of triangular facets any facet in Let its three vertices be denoted as Calculate the center point of the patch : Calculate the normal vector of the facet : in For vector cross product, The vector magnitude; Using the updated camera pose set , for the Each camera calculates the camera center. : Construct a vector from the camera center to the center point of the patch. : Define the unit vector of the line of sight. Distance from camera to plane : Calculate the cosine of the incident angle : in It is a vector dot product; calculate the weighted weights of the image from each viewpoint on the face. : Using local texture parameter coordinates and the updated camera pose, the patch Mapped to each oblique photographic image Upsampling yields the color vector Using weighted weights Calculate the facet The final color vector : each piece Geometric coordinates, topological connections, and final color vectors The elements are combined to form a textured 3D mesh, which is then used to output a realistic 3D model.

[0037] It should be noted that the repaired mesh topology is a mesh structure resulting from cracked vertex merging and local UV reparameterization, reflecting a regular geometric connection relationship without topological gaps. The center point of a facet is the arithmetic mean of the three-dimensional coordinates of the three vertices of the triangular facet, reflecting the central spatial position of the facet. The edge vector is the coordinate difference vector between any two vertices of the triangular facet. The facet normal vector is the normalized vector obtained after the cross product of the edge vectors, reflecting the spatial orientation of the facet. The viewing direction vector is the vector from the camera center to the center point of the facet, reflecting the camera's observation direction of the facet. The distance from the camera to the facet is the magnitude of the viewing direction vector, reflecting the spatial distance between the camera and the facet. The viewing direction unit vector is the normalized vector of the viewing direction vector. The cosine of the incident angle is the inner product (taking non-negative values) of the facet normal vector and the viewing direction unit vector, reflecting the degree of the camera's frontal view of the facet. The weighted weight is the ratio of the cosine of the incident angle to the square of the distance from the camera to the facet, reflecting the contribution of images from different viewing angles to the facet texture. The color vector is the color data obtained by mapping the facets onto the image and sampling them, reflecting the color information of the facets viewed from the image. The final color vector is the result of averaging the color vectors from each viewpoint according to weights, reflecting the final texture color of the facets. The textured 3D mesh is the model carrier that integrates geometric coordinates, topological relationships, and the final color vector, reflecting a complete 3D representation of the real scene.

[0038] It should be noted that the larger the cosine of the incident angle, the higher the degree of orthographic representation of the image on the viewing surface, and the clearer the texture; the closer the distance from the camera to the surface, the richer the image details. By balancing the influence of these two factors through a ratio, the texture contribution of the superior viewpoint is maximized. The output format of the 3D mesh is preferably OBJ format; textures are stored in PNG format with a resolution of 2048×2048 or 4096×4096, and texture coordinates ranging from 0 to 1, corresponding one-to-one with the model vertices to ensure correct texture mapping. In addition, a depth detection method can be used to calculate the difference between the depth value of each sampling point in the image and the distance from the camera to the surface; if the absolute value of the difference is greater than 0.5 meters, it is determined to be occlusion, and the color vector of that viewpoint is discarded; if the difference is less than or equal to 0.5 meters, it is determined to be unoccluded, and the color vector is retained for fusion. This invention, based on a repaired mesh topology and precise texture parameter coordinates, filters high-quality image information by calculating weighted weights for each viewpoint, performs multi-view texture fusion, and ultimately outputs a textured realistic 3D model. The weighted weights accurately highlight the contribution of effective viewpoints, avoiding the impact of inferior viewpoints on texture quality. Multi-view fusion fully utilizes the color information of multiple images, compensating for the limitations of a single image's viewpoint. The output model possesses both regular geometric structure and clear, consistent texture, adapting to the application requirements of realistic 3D scenes. This ensures that the model's visual effect and geometric accuracy are well-matched, with no obvious texture distortion or fragmentation, realistically reproducing the scene's appearance and providing reliable model support for subsequent applications.

[0039] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0040] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A real-scene 3D modeling method based on multi-source aerial triangulation of oblique photogrammetry and laser point cloud, characterized in that, Includes the following steps: Step S101: Acquire oblique photographic images, camera intrinsic parameters and laser point clouds, calculate the initial camera pose, unify the laser point clouds to the world coordinate system and construct a local elevation field, and calculate the slope aspect. Step S102: Generate a triangular mesh with texture parameter coordinates based on the initial camera pose. Based on the degree of cracking of the vertices in the texture parameter space, and combined with the orthogonality of the seam edge and the slope direction, extract the vertex cracking chain with the largest cumulative cracking energy. Step S103: Calculate the projection of the difference in centroids of the triangular facets on both sides of the joint edge in the vertex cracking chain onto the slope direction, obtain the slope shear jump observation, and establish a one-dimensional position sequence along the corridor direction, and fit a drift driving function containing second-order smooth regularity. Step S104: Construct a joint objective function that includes image reprojection error, laser point-to-surface residual and smoothing term. Superimpose the drift driving function along the slope onto the camera center to correct the pose. Perform joint solution to update the camera pose and geometric coordinates. Step S105: Determine the adaptive scale based on the median nearest neighbor distance of the laser point cloud, merge the spatially overlapping split vertex instances, and reparameterize the local texture parameter coordinates based on the consistency of the ratio between the 3D side length and the texture side length. Step S106: Based on the repaired mesh topology and local texture parameter coordinates, calculate the weighted weights of the image from each viewpoint on the face using the updated camera pose, perform multi-view texture fusion, and output a real-world 3D model.

2. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 1, characterized in that, Multiple oblique photographic images were acquired, a camera intrinsic parameter matrix was set for each oblique photographic image, laser point clouds were acquired, and the three-dimensional coordinates of each laser point in the laser sensor coordinate system were recorded. For each oblique photographic image, perform bundle adjustment to establish the projection relationship between the three-dimensional coordinates of any spatial point and the homogeneous image plane coordinates of its image point on this oblique photographic image. Solve the rotation matrix and translation vector by minimizing the projection residuals of all image points, and form the initial camera pose from the rotation matrix and translation vector. For each laser point, the sampling time is recorded. The carrier attitude rotation matrix and carrier translation vector in the carrier coordinate system at this sampling time are determined. The fixed rotation matrix and fixed translation vector of the laser sensor relative to the carrier are determined. Each laser point is transformed to the world coordinate system according to the rigid body transformation relationship to obtain the three-dimensional coordinates of the laser point in the world coordinate system. A local subset of the laser point cloud is selected in the world coordinate system, the local elevation field is represented by a plane function, and the coefficients of the plane function are solved by least squares constraints to construct the local elevation field. The horizontal gradient vector is determined based on the local elevation field. The horizontal gradient vector is normalized to obtain the unit vector of the slope aspect on the horizontal plane. The unit vector of the slope aspect on the horizontal plane is extended to three-dimensional space to obtain the slope aspect.

3. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 1, characterized in that, Using the initial camera pose set and oblique photographic images, a dense 3D point set is obtained through multi-view geometric reconstruction, and a triangular mesh is constructed on the dense 3D point set. The vertex set and the triangular face set are recorded. The 3D coordinates of each vertex in the vertex set are calculated. The texture parameter coordinates are assigned to each vertex through projection and texture unfolding to obtain a triangular mesh with texture parameter coordinates. For each vertex, count the number of instances of that vertex in the texture parameter space, and count the number of adjacent triangles of that vertex in the geometric mesh. Use the ratio of the number of instances to the number of adjacent triangles to define the degree of fragmentation of the vertex in the texture parameter space.

4. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photography and laser point cloud as described in claim 3, is characterized in that, Add the edges located on the boundaries of different tiles in the texture parameter map set to the seam edge set. Define a three-dimensional direction vector for any seam edge. Calculate the orthogonality coefficient between the seam edge and the slope using the three-dimensional direction vector and the slope vector. For each seam edge in the seam edge set, the cracking energy is defined by the product of the mean cracking degree of the two vertices of this seam edge and the orthogonality coefficient. A connected seam edge set is selected from the seam edge set, and the sum of the cracking energies of all seam edges in the connected seam edge set is calculated to obtain the cumulative cracking energy. The set with the largest cumulative cracking energy is determined from all connected seam edge sets, and the set with the largest cumulative cracking energy constitutes the vertex cracking chain with the largest cumulative cracking energy.

5. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photography and laser point cloud as described in claim 1, characterized in that, For each seam edge in the vertex splitting chain, identify the two triangular facets adjacent to this seam edge in the triangular facet set. Calculate the average of the three-dimensional coordinates of the three vertices of the two triangular facets to obtain their centroids. Calculate the difference between the centroids of the two triangular facets, calculate the inner product of the difference between the centroids and the aspect vector, and divide the inner product by the magnitude of the aspect vector to obtain the aspect shear jump observation of this seam edge. Calculate the centroid of the laser point cloud in the world coordinate system. Construct a covariance matrix using the three-dimensional coordinates of each laser point in the laser point cloud and the centroid of the laser point cloud. Perform eigenvalue decomposition on the covariance matrix, select the eigenvector corresponding to the largest eigenvalue and normalize it to obtain the unit vector of the corridor direction. Calculate the midpoint of each seam edge, calculate the difference vector between the midpoint and the centroid of the laser point cloud, and calculate the inner product of the difference vector and the unit vector of the corridor direction to obtain the one-dimensional position of the seam edge in the corridor direction. A target functional is constructed, which includes a data fitting term and a second-order smoothing regularization term. The data fitting term is the sum of the squared differences between the slope shear jump observations and the values ​​of the drift driving function at the corresponding one-dimensional positions. The second-order smoothing regularization term is the product of the smoothing regularization weight coefficient and the integral of the square of the second derivative of the drift driving function with respect to the one-dimensional position. The drift driving function is determined by minimizing the target functional.

6. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 1, characterized in that, For each camera in the initial camera pose set, the camera center in the world coordinate system is calculated using the camera's rotation matrix and translation vector. The one-dimensional position of the camera center in the corridor direction is calculated using the unit vector of the corridor direction and the centroid of the laser point cloud. The corrected camera center is calculated using the value of the drift driving function in the one-dimensional position and the slope vector. The corrected camera center is written back to the translation term to obtain the updated camera translation vector. The updated camera pose is composed of the camera's rotation matrix and the updated camera translation vector.

7. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 6, is characterized in that, Using the camera intrinsic parameter matrix and the updated camera pose, the projection relationship between the geometric coordinates of the three-dimensional points and the projected image points on the oblique photographic image is established, and the sum of squares of the differences between the homogeneous coordinates of the image points and the projected image points is calculated as the image reprojection error. Using the laser point cloud in the world coordinate system, a local plane parameter vector is associated with each laser point, and the sum of the squares of the distances between the laser point and the local plane is calculated as the laser point-to-plane residual; a smoothing term is constructed using the second derivative of the drift driving function. The image reprojection error, laser point-to-surface residual, and smoothing term are weighted and combined to obtain the joint objective function. By minimizing the joint objective function, the updated camera pose set and the updated 3D point geometric coordinate set are obtained.

8. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 1, characterized in that, For each laser point in the laser point cloud in the world coordinate system, calculate the nearest neighbor distance from each laser point to other laser points, and calculate the median of the nearest neighbor distances of all laser points to obtain the median of the nearest neighbor distance of the laser point cloud; use the product of the median of the nearest neighbor distance of the laser point cloud and a fixed scaling factor to determine the adaptive scale; In a triangular mesh with texture parameter coordinates, calculate the 3D distance for any pair of instances in the set of fragmented instances of each geometric vertex. When the 3D distance is less than or equal to the adaptive scale, merge the pair of instances into a single geometric vertex. Calculate the average of the 3D coordinates of the pair of instances as the 3D coordinates of the merged single geometric vertex, and associate all the local triangular faces that originally belonged to the pair of instances with the merged single geometric vertex in the geometric structure. A set of local triangular faces containing the merged single geometric vertex is selected. For each edge in the set of local triangular faces, the ratio of the sum of the texture edge lengths of all edges to the sum of the 3D edge lengths of all edges in the set is calculated to obtain the local scaling factor. A local texture parameter coordinate reparameterization objective function is constructed, which is the sum of squares of the differences between the texture edge length and the product of the local scaling factor and the 3D edge length. The local texture parameter coordinates are solved by minimizing the local texture parameter coordinate reparameterization objective function.

9. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud as described in claim 1, characterized in that, Using the repaired mesh topology, for each triangular facet in the triangular facet set, the average of the three-dimensional coordinates of the three vertices of the triangular facet is calculated to obtain the facet center point; the cross product of the two edge vectors of the triangular facet is calculated and normalized to obtain the facet normal vector. For each camera in the updated camera pose set, calculate the camera center in the world coordinate system; construct the gaze direction vector from the camera center to the center of the patch; calculate the magnitude of the gaze direction vector to obtain the distance from the camera to the patch; and normalize the gaze direction vector to obtain the gaze direction unit vector.

10. The method for real-scene 3D modeling by multi-source aerial triangulation of oblique photogrammetry and laser point cloud according to claim 9, characterized in that, Calculate the dot product between the normal vector of the patch and the unit vector of the viewing direction, and take the larger value between the dot product and zero as the cosine of the incident angle; use the ratio of the cosine of the incident angle to the square of the distance from the camera to the patch to calculate the weighted weight of the image of each viewpoint on the patch. Using local texture parameter coordinates and the updated camera pose, triangular facets are mapped onto images from various viewpoints to obtain color vectors. Weighted averages are then applied to the color vectors from each viewpoint to obtain the final color vector. Finally, a textured 3D mesh is constructed using the geometric coordinates, topological connectivity, and final color vector of each triangular facet, and a realistic 3D model is output.

Images