Multi-level fusion optimization method and system for oblique photography aerial triangulation processing

Abstract

The invention provides a multi-level fusion optimization method and system for oblique photography aerial triangulation processing, and relates to the technical field of photogrammetry, and the method comprises the steps: obtaining multi-view image data, constructing an image connection graph, carrying out the dynamic partitioning through the combination of weighted spectral clustering and multi-constraint spectral clustering, and forming a subnet with an internal connection and overlapping relation. And based on the image quantity, the point cloud scale and the matching complexity, evaluating the calculation load, distributing the subnets to distributed calculation nodes, and optimizing subnet parameters in parallel. A target overlapping area is determined through multi-level subnet fusion, a common point and a camera pose are extracted to construct a pose graph model, and a Gaussian-Newton method is adopted to solve global consistency transformation parameters. Under the constraint of a ground control point, overall optimization is carried out through weighted least square adjustment, finally, a global model is constructed, lightweight beam adjustment is executed, and an aerial triangulation processing result is generated. According to the invention, the automation degree of oblique photography data processing and the three-dimensional reconstruction precision are improved.

Description

Technical Field

[0001] This application relates to the field of photogrammetry technology, and in particular to a multi-level fusion optimization method and system for oblique photogrammetry aerial triangulation. Background Technology

[0002] With the widespread application of oblique photogrammetry in smart cities, geological disaster monitoring, and other fields, higher demands are being placed on the efficient processing and accurate 3D reconstruction of large-scale oblique photogrammetry data. Current processing workflows need to handle data volumes of tens of thousands of high-resolution images while ensuring the accuracy and computational efficiency of aerial triangulation, which urgently requires distributed processing and automated fusion technologies.

[0003] Existing technologies employ a distributed aerial triangulation method based on geographic grid partitioning. This method divides the survey area into multiple sub-regions according to a regular grid, performs local aerial triangulation calculations independently on each computing node, and finally unifies the results of each sub-region to the global coordinate system through coordinate transformation. This method mainly relies on the geographic coordinate range during partitioning and maintains continuous calculation within each sub-region.

[0004] However, when dealing with complex terrain and densely built-up areas, the geographic grid boundaries of this method tend to cut off strongly connected regions, resulting in an insufficient number of connection points at the boundaries of sub-regions, which affects the accuracy of subsequent fusion. At the same time, the fixed grid division method has limited adaptability to uneven data distribution, which may lead to uneven computational load, with some nodes under heavy computational pressure while others are idle, affecting the overall processing efficiency. Summary of the Invention

[0005] This application provides a multi-level fusion optimization method and system for oblique photogrammetry aerial triangulation to solve the problems of low automation and low 3D reconstruction accuracy in existing technologies.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a multi-level fusion optimization method for oblique photogrammetry aerial triangulation, comprising: Obtain multi-view image data using oblique photogrammetry equipment; Based on the feature matching relationship between images in the multi-view image data, an image connection graph is constructed, and a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm is used to dynamically divide the image connection graph into blocks to form multiple subnets with internal connections and overlapping relationships. The computational load of the subnet is evaluated based on the number of images, point cloud size and matching complexity. Based on the evaluation results, a work-stealing algorithm is used to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet in order to optimize the subnet parameters in parallel. Multi-level subnet fusion is performed on all subnets to determine the target overlapping region. Common points and camera pose information are extracted from the target overlapping region corresponding to the subnet with optimized parameters. Based on the common points and camera pose information, a pose graph optimization model is constructed. Based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method. In the presence of ground control points, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized as a whole using the weighted least squares adjustment algorithm. Based on the optimization results, a global model is constructed, and a lightweight beam adjustment process is performed on the global model to generate the aerial triangulation results.

[0007] Optionally, constructing an image connectivity map based on feature matching relationships between images in the multi-view image data includes: A feature extraction algorithm based on deep learning is used to extract feature points that represent image content from the multi-view image data. The feature points are matched using the nearest neighbor search and geometric verification algorithm to establish feature matching relationships between images. The nearest neighbor search and geometric verification algorithm is constructed based on KD-Tree FLANN matching and RANSAC fundamental matrix estimation. Based on the feature matching relationship, an image connection graph is constructed, wherein the nodes in the image connection graph represent images, the edges represent feature matching relationships between images, and the weight of the edge is a comprehensive metric determined based on the number of matching feature points between images, the reprojection error of matching feature point pairs, and the co-view geometric relationship.

[0008] Optionally, the step of using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to dynamically segment the image connectivity map into multiple subnets with internal connectivity and overlap relationships includes: Based on the edge weights of the image connectivity graph, a weighted adjacency matrix and a degree matrix are constructed using a weighted spectral clustering algorithm, and the Laplacian matrix is ​​calculated based on the weighted adjacency matrix and the degree matrix. The Laplacian matrix is ​​subjected to eigenvalue decomposition to obtain multiple eigenvalues; Select the k smallest eigenvalues ​​from the plurality of eigenvalues, and construct a new feature space based on the eigenvectors corresponding to the k smallest eigenvalues, where k is greater than or equal to 2; Using a multi-constraint spectral clustering algorithm, the image set is divided into k clusters that satisfy preset constraints in the new feature space through iterative optimization. Based on the k clusters, multiple subnets are constructed, wherein the constraints include subnet size balance constraints and minimum common point number constraints between subnets.

[0009] Optionally, the step of performing multi-level subnet fusion on all subnets to determine the target overlapping area includes: Based on the common images and common points of adjacent subnets, identify the common coverage areas between adjacent subnets; By analyzing the three-dimensional spatial points and camera pose information contained in each subnet, the initial overlapping area between adjacent subnets is located in the common coverage area; Within the initial overlapping region, extract the shared three-dimensional spatial points and corresponding camera observation information of adjacent subnets; Based on the shared three-dimensional spatial points and the corresponding camera observation information, the coordinate transformation relationship between adjacent subnets is determined; Based on the coordinate transformation relationship and the camera pose information, the initial overlapping region is precisely defined to form the target overlapping region.

[0010] Optionally, the computational load assessment of the number of images, point cloud size, and matching complexity of the subnet is performed. Based on the assessment results, a work-stealing algorithm is used to allocate each subnet to a corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet to optimize the subnet parameters in parallel, including: The point cloud scale is obtained by counting the number of images contained in each subnet and calculating the total number of 3D spatial points in each subnet. Analyze the complexity of feature matching relationships in each subnet to obtain the corresponding matching complexity; Using a pre-built subnet computational complexity evaluation model, the computational load of the number of images, the point cloud size, and the matching complexity is evaluated to obtain the evaluation results; Based on the evaluation results, each subnet is initially allocated to the corresponding distributed computing nodes through dynamic load balancing scheduling based on the work-stealing algorithm, wherein each distributed computing node maintains a corresponding local task queue. After a distributed computing node completes the computation of a subnet in its local task queue, it randomly obtains the corresponding subnet to be processed from the task queues of other distributed computing nodes whose load is greater than its local load. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the assigned subnets, and the parameters of the adjusted subnets are optimized asynchronously and in parallel to obtain parameter-optimized subnets. The assigned subnets include subnets in the local task queue and subnets to be processed.

[0011] Optionally, the step of using the Levenberg-Marquardt algorithm to perform local bundle adjustment on the assigned subnet includes: During the local bundle adjustment process, the sparse structure of the Hessian matrix is ​​used in conjunction with a preset cost function to reduce the dimensionality of the parameters of the assigned subnet. In this process, Cauchy and robust kernel functions are introduced into the cost function to reduce the impact of mismatched feature points on the parameters of the assigned subnet.

[0012] Optionally, the parameters corresponding to the asynchronously parallel optimized subnet are used to obtain the parameter-optimized subnet, including: The parameters of the subnet are optimized using an asynchronous parallel mode, and the optimized parameters are transmitted to the main distributed computing node. At the same time, the optimization progress data is periodically saved to the storage system through a checkpoint mechanism to obtain the subnet with optimized parameters. When the distributed computing node fails, the local bundle adjustment process is restored from the last successfully saved valid state based on the optimization progress data saved by the checkpoint mechanism.

[0013] Optionally, the step of constructing a pose graph optimization model based on the common points and the camera pose information, and using the Gauss-Newton method to solve for the globally consistent transformation parameters based on the pose graph optimization model, includes: Based on the common points, calculate the relative transformation matrix between adjacent subnets; Based on the relative transformation matrix and the camera pose information, a pose graph optimization model is constructed, wherein the nodes of the pose graph optimization model represent the camera pose of the subnet, and the edges represent the relative transformation matrix between adjacent subnets. Based on the nodes and edges of the pose graph optimization model, an error function is established, which characterizes the difference between the camera poses and the relative transformation matrix of adjacent subnets. Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct the optimization problem. The optimization problem is solved iteratively using the Gauss-Newton method to obtain the global consistency transformation parameters between subnets.

[0014] Optionally, when ground control points exist, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized using a weighted least squares adjustment algorithm, including: Based on the aforementioned global consistency transformation parameters, an adjustment optimization framework is established; The coordinates corresponding to the ground control points are incorporated into the adjustment optimization framework as fixed constraints. In the adjustment optimization framework, the sliding window optimization method is used to adjust the parameters of the surrounding area of ​​the ground control points. Based on the adjusted parameters of the surrounding area, and combined with the variance component estimation method, the weight ratios of various observations are adaptively optimized. Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized using the weighted least squares adjustment algorithm to obtain the optimization results.

[0015] Secondly, this application provides a multi-level fusion optimization system for oblique photogrammetry aerial triangulation, comprising: The acquisition module is used to acquire multi-view image data using oblique photogrammetry equipment; The construction module is used to construct an image connectivity graph based on the feature matching relationship between images in the multi-view image data, and to perform dynamic block processing on the image connectivity graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form multiple subnets with internal connections and overlapping relationships. The evaluation module is used to evaluate the computational load of the number of images, point cloud size and matching complexity of the subnet. Based on the evaluation results, the work-stealing algorithm is used to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet in order to optimize the subnet parameters in parallel. The fusion module is used to perform multi-level subnet fusion on all subnets to determine the target overlapping region. It extracts common points and camera pose information from the target overlapping region corresponding to the subnet with optimized parameters, and constructs a pose graph optimization model based on the common points and camera pose information. Based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method. The optimization module is used to optimize the global consistency transformation parameters as a whole by using the coordinates of the ground control points as fixed constraints and employing the weighted least squares adjustment algorithm when ground control points exist. The adjustment module is used to construct a global model based on the optimization results, and to perform lightweight bundle adjustment processing on the global model to generate aerial triangulation processing results.

[0016] This application provides a multi-level fusion optimization method for oblique photogrammetry aerial triangulation. The method includes: acquiring multi-view image data using an oblique photogrammetry device; constructing an image connectivity graph based on the feature matching relationships between images in the multi-view image data; dynamically dividing the image connectivity graph into multiple subnets with internal connectivity and overlap using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm; evaluating the computational load of the subnets based on the number of images, point cloud size, and matching complexity; allocating each subnet to a corresponding distributed computing node using a work-stealing algorithm based on the evaluation results; and performing local beamforming on each distributed computing node using the Levenberg-Marquardt algorithm. The system performs overall processing to optimize subnet parameters in parallel; multi-level subnet fusion is performed on all subnets to determine the target overlap region; common points and camera pose information are extracted from the target overlap region corresponding to the subnets with optimized parameters; based on the common points and camera pose information, a pose graph optimization model is constructed; based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method; in the case of ground control points, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized overall using a weighted least squares adjustment algorithm; based on the optimization results, a global model is constructed, and the global model is subjected to lightweight bundle adjustment processing to generate aerial triangulation results.

[0017] The technical solution provided in this application has the following beneficial effects: This application provides a complete and reliable raw data foundation for subsequent processing, ensuring the comprehensiveness and accuracy of the data source. Intelligent segmentation based on the actual connectivity strength between images ensures both tight connectivity within subnets and sufficient overlapping areas for subsequent fusion. It achieves rational allocation and efficient utilization of computing resources, avoiding uneven node load and improving overall computational efficiency. Pose graph optimization ensures high-precision alignment between subnets, providing accurate transformation parameters for global model construction. Combining ground control points improves the absolute accuracy of the model, ensuring the geographic reference accuracy of the results. Further optimization of internal model consistency eliminates systematic errors and improves the quality of the final results.

[0018] Furthermore, this application also obtains image feature points through deep learning feature extraction algorithms, and uses KD-Tree-based FLANN matching and RANSAC fundamental matrix estimation for feature point matching and geometric verification to establish feature matching relationships between images, and constructs an image connectivity graph based on this. This connectivity graph uses images as nodes and matching relationships as edges, with edge weights comprehensively considering the number of matching points, reprojection error, and co-view geometric relationships to form a graph structure that accurately reflects the image connectivity strength.

[0019] Furthermore, this scheme can establish a graph model that accurately reflects the actual connection relationship between images, providing a reliable basis for subsequent intelligent segmentation, effectively improving the accuracy and robustness of feature matching, and laying a solid foundation for the overall processing flow.

[0020] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a multi-level fusion optimization method for oblique photogrammetry aerial triangulation provided in this application embodiment; Figure 2 A schematic diagram illustrating a specific implementation of a multi-level fusion optimization method for oblique photogrammetry aerial triangulation provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a multi-level fusion optimization system for oblique photogrammetry aerial triangulation provided in an embodiment of this application. Detailed Implementation

[0023] In large-scale aerial triangulation processing of oblique photogrammetry, existing distributed processing methods based on geographic grids face challenges. These methods use fixed grids to divide the survey area. When dealing with complex urban terrain, grid boundaries often cut off strongly connected building clusters and terrain features, resulting in weak connection constraints at sub-region boundaries. Simultaneously, the fixed grid division pattern struggles to adapt to uneven data distribution, easily causing load imbalances between computing nodes and affecting overall processing efficiency. These factors collectively limit the accuracy and effectiveness of large-scale oblique photogrammetry data processing.

[0024] To address the aforementioned issues, this application proposes a multi-level fusion optimization method for oblique photogrammetry aerial triangulation. This method first constructs a connectivity graph based on the actual matching relationships between images, and then uses intelligent clustering to divide the survey area into subnets with tight internal connections and sufficient overlap. Subsequently, a computational load assessment mechanism is established to achieve dynamic allocation and parallel optimization of subnet tasks. Finally, multi-level fusion and overall adjustment ensure the consistency of the global model. This method overcomes the limitations of fixed grid division, ensuring effective connectivity between subnets through data-driven intelligent partitioning, and improving computational resource utilization through dynamic load balancing. It achieves high-precision and high-efficiency processing of ultra-large-scale oblique photogrammetry data, effectively solving the problems of accuracy loss and low efficiency caused by unreasonable partitioning in existing technologies.

[0025] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The core of this application is to provide a multi-level fusion optimization method for oblique photogrammetry aerial triangulation, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: Step 101: Acquire multi-view image data using oblique photogrammetry equipment.

[0027] In step 101, the oblique photogrammetry device is a specialized camera system capable of simultaneously acquiring images from both vertical and oblique perspectives, typically mounted on a drone or aerial platform. Multi-view image data refers to a collection of images taken from different angles of the same area using the oblique photogrammetry device. The image data includes pixel arrays, geographic location information, and shooting parameters, forming the fundamental data source for subsequent processing.

[0028] In this embodiment, an oblique photogrammetry device equipped with a multi-lens camera is used to take aerial photos of the target area. During the flight, the device automatically acquires images according to a preset flight path and overlap. Each shooting point simultaneously acquires images at vertical and multiple oblique angles. All image data, along with the corresponding position and attitude information, are uniformly stored as a raw dataset, providing complete input data for subsequent processing.

[0029] For example, an oblique photogrammetry survey was conducted on a certain urban area A. A five-lens oblique photogrammetry camera was mounted on a UAV platform and flew along a preset flight path. The forward overlap was set to 80%, the lateral overlap was set to 70%, and the flight altitude was 300 meters. A total of 5,000 multi-angle images were acquired. Each image was accompanied by POS data, including latitude, longitude, elevation, and attitude angle. All data was stored in a common image format for subsequent processing.

[0030] Step 102: Based on the feature matching relationship between images in the multi-view image data, construct an image connection graph, and use a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to dynamically divide the image connection graph into multiple sub-networks with internal connections and overlapping relationships.

[0031] In step 102, the image connectivity graph is a graph structure where nodes represent individual images, edges represent feature matching relationships between images, and edge weights comprehensively reflect the number of matching points, geometric accuracy, and co-view strength. Dynamic block partitioning is a process of adaptively dividing the dataset based on actual connectivity relationships. Subnets are subsets of images obtained through clustering analysis, with tight internal connections and overlapping areas retained with other subnets.

[0032] In this embodiment, feature points are first extracted from multi-view images and matched to establish connections between images. Then, a weighted image connection graph is constructed, and a weighted spectral clustering algorithm is used to divide the image based on the connection graph. During the clustering process, the subnet size balance and minimum common point constraints are considered. The image set is divided into multiple subnets through feature decomposition and cluster analysis. Each subnet is densely connected and has sufficient overlap with adjacent subnets.

[0033] For example, feature extraction and matching are performed on the 5000 images obtained in step 101 to construct an image connectivity graph containing 5000 nodes, where the edge weights are calculated based on the number of matching points, reprojection error, and common-view geometric relationships. A weighted spectral clustering algorithm is used to divide the image set into 50 subnets, each containing 80-120 images. Adjacent subnets are guaranteed at least 20% overlap to ensure tight internal connections and sufficient common points between subnets.

[0034] Step 103: Evaluate the computational load of the number of images, point cloud size and matching complexity of the subnet. Based on the evaluation results, use the work-stealing algorithm to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, use the Levenberg-Marquardt algorithm to perform local bundle adjustment processing on the subnet to optimize the subnet parameters in parallel.

[0035] In step 103, the computational load assessment is a quantitative analysis of the subnet processing complexity, including the number of images, point cloud size, and matching complexity. Distributed computing nodes in this application represent independent processing units participating in parallel computing. These nodes are connected through a computer network to form a distributed computing environment. Each node is responsible for processing the allocated subnet data, possessing independent computing power and storage space, and achieving efficient processing of large-scale oblique photogrammetric data through collaborative work. The work-stealing algorithm is a dynamic task allocation mechanism, and local bundle adjustment is the process of solving for the camera pose and 3D point coordinates within the subnet using an optimization algorithm.

[0036] In this embodiment, the number of images, point cloud size and matching complexity of each subnet are first evaluated to establish a computational load model. Based on the evaluation results, a work-stealing algorithm is used to allocate the subnets to distributed computing nodes. Each node uses the Levenberg-Marquardt algorithm to perform local bundle adjustment on the allocated subnets. During the optimization process, matrix sparsity is used to accelerate the calculation, and a robust kernel function is introduced to suppress the influence of mismatches. Asynchronous communication and checkpointing mechanisms are used to ensure computational reliability and efficiency.

[0037] For example, the 50 subnets obtained in step 102 are evaluated. Subnet A contains 100 images and 50,000 3D points. Its matching complexity is rated as high and it is assigned to computing node 1. The task allocation is dynamically adjusted using a work-stealing algorithm. Node 1 uses the Levenberg-Marquardt algorithm to perform local bundle adjustment on subnet A. After 15 iterations, the camera pose and 3D point coordinates converge. The optimization results are transmitted back to the master node through asynchronous communication. A checkpoint is saved every 10% of the progress is completed.

[0038] Step 104: Perform multi-level subnet fusion on all subnets to determine the target overlapping region. Extract common points and camera pose information from the target overlapping region corresponding to the subnet with optimized parameters. Based on the common points and camera pose information, construct a pose graph optimization model. Based on the pose graph optimization model, use the Gauss-Newton method to solve for the global consistency transformation parameters.

[0039] In step 104, the target overlapping region is the spatial range shared by adjacent subnets, the common point is the three-dimensional spatial point shared by multiple subnets, the camera pose information includes camera position and attitude parameters, the pose graph optimization model is a graph structure describing the spatial relationship between subnets, and the global consistency transformation parameter is a transformation matrix that unifies the coordinate systems of each subnet.

[0040] In this embodiment, overlapping regions are determined based on subnetting, and common points and camera poses are extracted from the optimized subnets. Based on this information, a pose graph optimization model is constructed, where nodes represent subnet poses and edges represent relative transformation relationships. An error function is defined to describe the difference between actual observations and model predictions. A nonlinear optimization method with a robust kernel function is used to iteratively solve for the optimal global consistency transformation parameters using the Gauss-Newton method.

[0041] For example, the overlapping region between subnet A and subnet B is identified, 2000 common points and corresponding camera poses are extracted; a pose graph model is constructed, an error function is defined and a robust kernel function is added; the Gauss-Newton method is used for optimization, and after 20 iterations, the transformation parameters between subnet A and subnet B are obtained, including rotation matrix and translation vector, to achieve high-precision alignment of the two subnets.

[0042] Step 105: If ground control points exist, use the coordinates corresponding to the ground control points as fixed constraints, and use the weighted least squares adjustment algorithm to optimize the global consistency transformation parameters.

[0043] In step 105, the existence of ground control points refers to ground markers with known precise coordinates established within the survey area. The coordinate data of these control points can be introduced into the aerial triangulation process as absolute constraints to improve the absolute accuracy and geographic reference accuracy of the model. Ground control points are ground markers with known precise coordinates. Fixed constraints treat the coordinates of the control points as unadjustable baseline conditions. The weighted least squares adjustment algorithm is a method of solving for optimal parameters by minimizing the sum of squared weighted residuals.

[0044] In this embodiment, the coordinates of ground control points are incorporated into the optimization framework as fixed constraints. The sliding window method is used to optimize the camera pose and 3D point parameters of the area surrounding the control points. The weight ratios of various observations are adaptively adjusted through variance component estimation. Based on the adjusted weight configuration, the weighted least squares adjustment algorithm is used to optimize and solve the global consistency transformation parameters and associated parameters as a whole.

[0045] For example, 10 ground control points were set up in area A as fixed constraints. The sliding window method was used to optimize the parameters within a 50-meter radius around the control points. The weight ratio of image observations and control point observations was adjusted by variance component estimation. Weighted least squares adjustment was used to optimize the global transformation parameters. After adjustment, the mean square error of the control point residuals was 0.02 meters, which met the accuracy requirements.

[0046] Step 106: Based on the optimization results, construct a global model and perform lightweight beam adjustment on the global model to generate aerial triangulation results.

[0047] In step 106, the global model is the 3D reconstruction result that integrates all subnet data, the lightweight bundle adjustment is the process of quickly and finely optimizing the global model, and the aerial triangulation result is the final result that includes accurate camera pose and 3D point coordinates.

[0048] In this embodiment, a unified global 3D model is constructed based on the overall optimization results. Lightweight beam adjustment is performed on the poses of all cameras and the coordinates of 3D points. Residual errors are eliminated through a finite number of iterations. The global transformation parameters remain unchanged, and only other parameters are fine-tuned. The final output is an aerial triangulation result containing accurate poses and 3D point clouds.

[0049] For example, a global model is constructed by integrating data from 50 subnets, containing 5,000 camera poses and 2 million 3D points; lightweight bundle adjustment is performed, and after 5 iterations, the model accuracy is improved, with the maximum reprojection error reduced to 1.5 pixels; the final aerial triangulation results are output, including optimized camera parameters, 3D point clouds, and an accuracy assessment report.

[0050] This method achieves efficient processing of ultra-large-scale oblique photogrammetry data through intelligent segmentation, load-balanced distributed computing, and multi-level fusion optimization. It improves the automation level and quality of aerial triangulation processing, ensures the high accuracy and global consistency of 3D models, and provides reliable technical support for real-scene 3D modeling.

[0051] To address the issues of accuracy and efficiency in establishing image connectivity relationships during oblique photogrammetry data processing, some embodiments include step 102: constructing an image connectivity graph based on feature matching relationships between images in the multi-view image data, such as... Figure 2 The above includes: Step 201: Based on a deep learning-based feature extraction algorithm, extract feature points from the multi-view image data to represent the image content.

[0052] In step 201, the deep learning-based feature extraction algorithm automatically identifies stable and unique feature point locations and descriptive information in the image using a trained neural network model. Image content refers to the visual feature information of scene features recorded in the image, including distinctive visual patterns such as textures, edges, and corners. These features can uniquely characterize specific local areas in the image and are used to establish correspondences between different images. Feature points refer to pixel locations in the image that possess visual characteristics and can uniquely characterize local image content.

[0053] In this embodiment, a pre-trained feature extraction network is used to process each oblique photographic image. The network automatically analyzes the image content and outputs the coordinates and feature description data of the feature points. These feature points can resist changes in illumination and viewing angle, providing stable and reliable basic data for subsequent image matching.

[0054] Step 202: Use the nearest neighbor search and geometric verification algorithm to match the feature points to establish feature matching relationships between images. The nearest neighbor search and geometric verification algorithm is constructed based on KD-Tree FLANN matching and RANSAC fundamental matrix estimation.

[0055] In step 202, nearest neighbor search is the process of quickly finding similar feature points by constructing a feature point index structure. Geometric verification is a method to check the correctness of feature point matching through spatial geometric relationships. Feature matching relationships are the corresponding connections between feature points in different images. A K-Dimensional Tree (KD-Tree) is a tree-shaped data structure used for efficient nearest neighbor search. The Fast Library for Approximate Nearest Neighbors (FLANN) matching method is a feature point matching method based on fast nearest neighbor search. Random Sample Consensus (RANSAC) fundamental matrix estimation is a method for estimating the basic geometric relationships between images using a random sample consensus algorithm.

[0056] In this embodiment, the feature points of all images are first indexed using the KD-Tree structure. The nearest neighbor correspondence between feature points of each image is quickly found using the FLANN matching algorithm. Then, the RANSAC algorithm is used to perform geometric verification on the matching results, and mismatched point pairs that do not conform to the constraints of the fundamental matrix are eliminated. Finally, accurate feature matching relationships between images are established.

[0057] Step 203: Based on the feature matching relationship, construct an image connection graph, wherein the nodes in the image connection graph represent images, the edges represent feature matching relationships between images, and the weight of the edge is a comprehensive metric determined based on the number of matching feature points between images, the reprojection error of matching feature point pairs, and the co-view geometric relationship.

[0058] In step 203, a feature point refers to a pixel location with visual features extracted from a single image, while a matching feature point pair refers to a pair of feature points that establish a correspondence between different images through a feature matching algorithm. Feature points are the basic unit of matching, and matching feature point pairs are the result of feature matching. Spatial correspondence between images can be established through multiple matching feature point pairs.

[0059] In this embodiment, each image is treated as a node in the graph, and the geometrically verified feature matching relationship is used as the edge connecting the node. The weight value of each edge is calculated based on the number of matching points, the accuracy of reprojection error, and the size of the common viewing area, thus constructing an image connection graph that fully reflects the actual connection strength between images. This graph accurately describes the internal connection structure of the image data.

[0060] Here is a specific example: Based on 5,000 multi-angle image data obtained from oblique photogrammetry of a certain urban area A, a deep learning feature extraction algorithm was first used to extract feature points from all images. Then, a fast approximate nearest neighbor search matching algorithm based on a multidimensional tree structure and a random sample consensus (RSC) fundamental matrix estimation algorithm were used to establish feature matching relationships between images. Specifically, a pre-trained feature extraction network was used to process each image, outputting the coordinates of each feature point and 256-dimensional feature description data. In the feature matching stage, a multidimensional tree index structure was first constructed to create a spatial index for 1.5 million feature points in all images. A fast approximate nearest neighbor search algorithm was used to find potential matching pairs between feature points, obtaining initial matching results. Then, a random sample consensus algorithm was used to geometrically verify the matching points of each pair of images. The fundamental matrix was calculated by randomly sampling 7 pairs of matching points, and the number of interior points that met the geometric constraints was counted. After multiple iterations, matching relationships with an interior point ratio exceeding 60% were retained. Finally, 180,000 reliable matching relationships that passed geometric verification were selected from 250,000 initial matching pairs. Based on these verified matching relationships, an image connectivity graph was constructed. The graph contains 5000 nodes representing each image, and the edges between nodes represent verified feature matching relationships. The weight of each edge is calculated by comprehensively considering three factors: the number of matched feature points is the actual number of verified matching point pairs; the reprojection error of the matched feature point pairs is the average deviation of the projected positions of all matching points on the two images; and the strength of the co-view geometric relationship is quantified by the ratio of the co-view area to the total image area. The final constructed image connectivity graph accurately reflects the actual connectivity strength between the 5000 images, providing a reliable basis for subsequent intelligent block segmentation processing.

[0061] In this embodiment, deep learning feature extraction ensures the stability and discriminability of feature points. Combined with efficient nearest neighbor search and rigorous geometric verification, accurate matching relationships are established. The final constructed image connectivity graph can accurately reflect the actual connection strength between images, providing a reliable foundation for subsequent intelligent segmentation and distributed processing, and improving the accuracy and efficiency of oblique photogrammetry data processing.

[0062] To further improve the rationality of oblique photogrammetry data block processing and subsequent fusion effect, in some embodiments, step 102: the image connectivity map is dynamically divided into blocks using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form multiple subnets with internal connectivity and overlap relationships, including: Step 301: Based on the edge weights of the image connectivity graph, construct a weighted adjacency matrix and a degree matrix using a weighted spectral clustering algorithm, and calculate the Laplacian matrix based on the weighted adjacency matrix and the degree matrix.

[0063] In step 301, the weighted adjacency matrix is ​​a square matrix describing the connection strength between nodes in the image connectivity graph, with each element corresponding to the weight of an edge in the graph. The degree matrix is ​​a diagonal matrix, where each diagonal element is the sum of the weights of all connecting edges to the corresponding node. The Laplacian matrix is ​​the feature matrix obtained by subtracting the weighted adjacency matrix from the degree matrix, used to characterize the overall structural properties of the image connectivity graph.

[0064] In this embodiment, a weighted adjacency matrix is ​​constructed based on the weight values ​​of each edge in the image connectivity graph, wherein the matrix element values ​​are directly taken from the connection weights between the corresponding image nodes; the sum of the connection weights of each node is calculated to form a degree matrix; the degree matrix is ​​subtracted from the weighted adjacency matrix to obtain a Laplacian matrix, which contains the topological structure information of the image connectivity graph.

[0065] Step 302: Perform eigenvalue decomposition on the Laplacian matrix to obtain multiple eigenvalues.

[0066] In step 302, eigenvalue decomposition is a mathematical process of decomposing a matrix into eigenvalues ​​and eigenvectors. Eigenvalues ​​are scalar values ​​that characterize the properties of a matrix, and eigenvectors are direction vectors corresponding to the eigenvalues.

[0067] In this embodiment, the Laplacian matrix is ​​decomposed to solve for all eigenvalues ​​and corresponding eigenvectors of the matrix, resulting in a sequence of eigenvalues ​​arranged by size and a set of associated eigenvectors. These eigenvalues ​​reflect the structural features of the image connectivity map at different scales.

[0068] Step 303: Select the first k smallest eigenvalues ​​from the plurality of eigenvalues, and construct a new feature space based on the eigenvectors corresponding to the first k smallest eigenvalues, where k is greater than or equal to 2.

[0069] In step 303, the feature space is a vector space spanned by feature vectors. The new feature space is a low-dimensional space composed of selected feature vectors, which can preserve the main structural information of the original image connectivity map.

[0070] In this embodiment, the first k smallest feature values ​​are selected from the feature value sequence, the feature vectors corresponding to these feature values ​​are extracted, and these feature vectors are used as basis vectors to construct a new k-dimensional feature space. Each image corresponds to a feature vector representation in this space.

[0071] Step 304: Using a multi-constraint spectral clustering algorithm, the image set is divided into k clusters that satisfy preset constraints in the new feature space through iterative optimization. Based on the k clusters, multiple subnets are constructed, wherein the constraints include subnet size balance constraints and minimum common point number constraints between subnets.

[0072] In step 304, the multi-constraint spectral clustering algorithm is an improved clustering method that considers multiple constraints simultaneously during the clustering process. Clusters are image groups obtained through cluster analysis. The subnet size balance constraint requires that the number of images contained in each cluster be relatively balanced. The minimum common point constraint between subnets requires that adjacent clusters maintain sufficient common points.

[0073] In this embodiment, cluster analysis is performed on the feature vector representations of all images in a new feature space. During the clustering process, iterative optimization simultaneously satisfies the constraints of subnet size balance and minimum common point number, dividing the image set into k clusters. Each cluster is converted into a subnet, ensuring that each subnet is tightly connected and that there is sufficient overlap between subnets. This step dynamically adjusts the clustering process through multiple constraints, achieving intelligent segmentation based on actual data features.

[0074] Here is a specific example: An image connectivity graph was constructed based on 5000 multi-angle images obtained from oblique photogrammetry of a certain urban area A. This graph contains 5000 nodes and approximately 180,000 weighted edges. First, a weighted adjacency matrix W was constructed using a weighted spectral clustering algorithm. This matrix is ​​a 5000-row, 5000-column square matrix, where each element... Indicates the first The connection weights between image Z and image J are directly taken from the weights of the corresponding edges in the image connection graph, with values ​​ranging from [0,1]. Next, a degree matrix D is constructed. This matrix is ​​a 5000-order diagonal matrix, where each diagonal element... Equal to the weighted adjacency matrix of the first The sum of all elements in the row, i.e. , indicating the first The sum of the weights of all edges connecting each node. Then calculate the Laplacian matrix. Its calculation formula is ,in For degree matrix, This is a weighted adjacency matrix, which characterizes the overall structural properties of the image connectivity graph. (The last part, "for the Laplacian matrix," appears to be a separate, unrelated statement and is left untranslated.) Eigenvalue decomposition was performed to obtain 5000 eigenvalues. These eigenvalues ​​were then sorted in ascending order to obtain a sequence. ,in Minimum. Select the top 50 smallest eigenvalues ​​from these eigenvalues. The eigenvectors corresponding to these eigenvalues ​​are extracted to form a new 50-dimensional feature space, where each image corresponds to a 50-dimensional feature vector. A multi-constraint spectral clustering algorithm is used to perform clustering analysis in this new feature space. A subnet size balancing constraint is set, with each cluster containing 80 to 120 images. This range is determined by dividing the total number of images (5000) by the preset subnet number (50) to obtain an average value of 100, and then setting a fluctuation range of plus or minus 20 images. Simultaneously, a minimum common point constraint was set between subnets, requiring adjacent subnets to share at least 20% of their connection points. This ratio was experimentally verified to ensure subsequent fusion accuracy. Through iterative optimization, the image set was divided into 50 clusters, each cluster being converted into a subnet, ultimately forming 50 subnets with internal connections and overlaps. Each subnet is tightly connected internally and retains sufficient overlap between subnets, providing an optimization foundation for subsequent distributed processing.

[0075] In this embodiment, weighted spectral clustering accurately captures the actual connectivity between images, and multiple constraints ensure the balance and fusionability of subnet partitioning. The resulting subnets meet the requirements of parallel processing and guarantee the accuracy of subsequent fusion, thereby improving the efficiency and quality of ultra-large-scale oblique photogrammetry data processing.

[0076] To further improve the accuracy and reliability of subnet fusion, in some embodiments, step 104: performing multi-level subnet fusion on all subnets to determine the target overlapping area includes: Step 401: Identify the common coverage area between adjacent subnets based on the common images and common points of adjacent subnets.

[0077] In step 401, common imagery refers to image data that appears simultaneously in adjacent subnets. Common point refers to the same spatial point that exists in the 3D reconstruction results of different subnets. Common coverage area is the geographical area where adjacent subnets spatially overlap.

[0078] In this embodiment of the application, by comparing the image identifiers and 3D point identifiers contained in adjacent subnets, common images and common points belonging to both subnets are identified, and the common coverage area between adjacent subnets is determined based on the spatial distribution of these common elements.

[0079] Step 402: By analyzing the three-dimensional spatial points and camera pose information contained in each subnet, locate the initial overlapping area between adjacent subnets in the common coverage area.

[0080] In step 402, the 3D spatial points are obtained through photogrammetric calculations of their 3D coordinates. Camera pose information includes the camera's position and attitude parameters. The initial overlap region is the inter-mesh overlap range preliminarily determined based on common elements.

[0081] In this embodiment of the application, the coordinate distribution of three-dimensional spatial points contained in each subnet and the spatial relationship of camera pose are analyzed. In the common coverage area, the initial overlapping area between adjacent subnets is located by spatial clustering method. This area contains the spatial range in which there may be connection relationship between the subnets.

[0082] Step 403: Within the initial overlapping area, extract the three-dimensional spatial points shared by adjacent subnets and the corresponding camera observation information.

[0083] In step 403, camera observation information refers to the two-dimensional coordinate data of feature points on the image recorded by the camera during shooting, while "camera pose information" refers to the position and attitude parameters of the camera in three-dimensional space. The connection between the two is that camera observation information is the basic data for calculating camera pose information, and camera pose information can be calculated from camera observation information through multi-view geometric relationships.

[0084] In this embodiment of the application, within the initial overlapping area, the three-dimensional spatial points shared by adjacent subnets and their observation information in different subnet images are extracted through coordinate matching and feature association analysis, including two-dimensional pixel coordinates and corresponding three-dimensional coordinates.

[0085] Step 404: Based on the shared three-dimensional spatial points and the corresponding camera observation information, determine the coordinate transformation relationship between adjacent subnets.

[0086] In step 404, the coordinate transformation relationship is a mathematical expression describing the transformation relationship between two different coordinate systems, including rotation, translation, and scale parameters.

[0087] In this embodiment of the application, based on the coordinate values ​​of the extracted common three-dimensional space points in different subnets, the coordinate transformation parameters between adjacent subnets are calculated by the least squares principle, and the transformation relationship from one subnet coordinate system to another subnet coordinate system is established.

[0088] Step 405: Based on the coordinate transformation relationship and the camera pose information, the initial overlapping region is precisely defined to form the target overlapping region.

[0089] In step 405, precise delineation is the process of determining the boundaries of the overlapping region through optimized calculations.

[0090] In this embodiment, based on the calculated coordinate transformation relationship and camera pose information, the initial overlapping region is optimized and adjusted through spatial projection and error analysis, unreliable regions are eliminated, and regions with sufficient common points and stable geometric relationships are retained to form an accurate target overlapping region.

[0091] Here is a specific example: Based on 5,000 multi-angle images obtained from oblique photogrammetry of a certain urban area A, 50 subnets were formed after distributed processing. Subnet A (containing 100 images and 50,000 3D points) and subnet B (containing 95 images and 48,000 3D points) were selected for fusion processing. First, based on the comparison of image and 3D point identifiers between the two subnets, 32 common images and 2,150 common points were identified. Based on the spatial distribution of these common elements, the common coverage area was determined to be 380 meters east-west and 420 meters north-south. By analyzing the coordinate distribution of all 3D spatial points and the pose information of 215 cameras in the two subnets, density clustering was used to locate the initial overlapping area within the common coverage area. This area, spanning 350 meters east-west and 400 meters north-south, contains 1,850 common points. Within the initial overlapping region, 1800 shared 3D spatial points and their corresponding camera observation information, totaling 21,600 observation records, were extracted from adjacent subnets through coordinate matching. Each observation record contains the 2D pixel coordinates of the feature point on the image and the corresponding 3D point coordinates. Based on these shared 3D spatial points, the coordinate transformation relationship between adjacent subnets was calculated, and the transformation parameters were solved using the least squares principle. The coordinate transformation model is as follows: ,in The unit of measurement for the three-dimensional coordinates of a point in the coordinate system of subnet B is meters. This indicates that a 3x3 rotation matrix is ​​dimensionless. This indicates the three-dimensional coordinates of a point in the coordinate system of subnet A, expressed in meters. The 3D translation vector is represented in meters. The rotation matrix is ​​obtained by solving for the 3D translation vector. Translation vector The unit is meters. Based on this coordinate transformation relationship and camera pose information, the initial overlapping area is precisely defined through spatial projection error analysis. The reprojection error of each common point is calculated, and unreliable areas with errors greater than 2 pixels are eliminated. Finally, the target overlapping area is formed with an east-west span of 320 meters and a north-south span of 380 meters, containing 1560 stable common points and 28 common images, providing a precise spatial range for determining the target overlapping area.

[0092] In this embodiment, the effective overlapping area between subnets is accurately located through multi-level analysis, ensuring the accuracy and reliability of subsequent fusion processing, providing a solid foundation for global consistency optimization, and improving the overall quality of oblique photogrammetry 3D reconstruction.

[0093] To further improve the efficiency and load balancing of distributed computing, in some embodiments, step 103 involves: evaluating the computational load of the subnet's image quantity, point cloud size, and matching complexity; based on the evaluation results, using a work-stealing algorithm to allocate each subnet to a corresponding distributed computing node; and on each distributed computing node, using the Levenberg-Marquardt algorithm to perform local bundle adjustment processing on the subnet to optimize subnet parameters in parallel, including: Step 501: Obtain the point cloud scale by counting the number of images contained in each subnet and calculating the total number of 3D spatial points in each subnet.

[0094] In step 501, the number of images refers to the total number of images contained in each subnet. The total number of 3D spatial points refers to the total number of 3D points in each subnet calculated through feature matching and aerial triangulation. The point cloud size is a subnet data size indicator quantified by the total number of 3D spatial points.

[0095] In this embodiment of the application, the number of images is obtained by traversing the image list of each subnet and the number of all three-dimensional spatial points in each subnet is calculated to obtain the point cloud size. These two data together reflect the basic data volume of the subnet.

[0096] Step 502: Analyze the complexity of feature matching relationships in each subnet to obtain the corresponding matching complexity.

[0097] In step 502, the matching complexity is an index of the difficulty of subnet processing obtained by analyzing the number and distribution characteristics of feature matches.

[0098] In this embodiment of the application, the feature matching relationship between all images in each subnet is analyzed, and the number of matching point pairs, the uniformity of matching distribution, and the matching quality index are statistically analyzed. The matching complexity level of each subnet is evaluated by comprehensively considering these factors.

[0099] Step 503: Using a pre-built subnet computational complexity evaluation model, evaluate the computational load of the number of images, the size of the point cloud, and the matching complexity to obtain the evaluation results.

[0100] In step 503, the subnet computational complexity assessment model is a mathematical model trained using historical data to predict the subnet computational load. The computational load assessment is a quantitative result of the computational resources required for subnet processing calculated by the model.

[0101] In this embodiment of the application, the number of images, point cloud size and matching complexity of each subnet are input into a pre-trained evaluation model. The model outputs the computational load evaluation result of each subnet through weighted calculation and comprehensive analysis. The result reflects the relative computational resources required to process the subnet.

[0102] Step 504: Based on the evaluation results, each subnet is initially allocated to the corresponding distributed computing nodes through dynamic load balancing scheduling based on the work-stealing algorithm, wherein each distributed computing node maintains a corresponding local task queue.

[0103] In step 504, dynamic load balancing scheduling is a mechanism that dynamically adjusts task allocation based on real-time load conditions. The local task queue is a list of unprocessed subnets maintained by each compute node.

[0104] In this embodiment, based on the computational load assessment results, a work-stealing algorithm is used to initially allocate each subnet to a distributed computing node. Each node maintains a local task queue, and the subnets in the queue are sorted in descending order of computational load to ensure that nodes with heavier loads prioritize processing subnets with smaller computational loads.

[0105] Step 505: After a distributed computing node completes the subnet computation in its local task queue, it randomly obtains the corresponding subnet to be processed from the task queues of other distributed computing nodes whose load is greater than its local load.

[0106] In step 505, local load refers to the total computational load of the tasks currently pending on the compute node. The subnets to be processed are the subnet tasks in the task queues of other nodes that have not yet started computation.

[0107] In this embodiment of the application, when a computing node completes the computation of all subnets in its local task queue, it will randomly select other nodes with heavier loads and obtain a subnet to be processed from the tail of its task queue for computation, thereby realizing the dynamic allocation of computing resources.

[0108] Step 506: On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the assigned subnets, and the parameters of the adjusted subnets are optimized asynchronously and in parallel to obtain the parameter-optimized subnets. The assigned subnets include the subnets in the local task queue and the subnets to be processed.

[0109] In step 506, the Levenberg-Marquardt algorithm is a numerical algorithm for nonlinear least squares optimization. Local bundle adjustment is the process of solving for the camera pose and 3D point coordinates within the subnet using an optimization algorithm. The parameter-optimized subnet is a high-precision subnet result obtained after optimization calculations.

[0110] In this embodiment of the application, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the assigned subnet on each computing node. The optimal camera pose parameters and 3D point coordinates are solved by iterative optimization. During the optimization process, each node calculates independently and returns the results through asynchronous communication, and finally obtains the optimized parameters of all subnets.

[0111] Here is a specific example: Based on 5000 multi-angle images obtained from oblique photogrammetry of a certain urban area A, 50 subnets were formed through dynamic block processing. First, the number of images in each subnet was statistically analyzed, resulting in between 80 and 120 images per subnet. Subnet A contained 100 images, and subnet B contained 95 images. The total number of 3D points in each subnet was calculated to obtain the point cloud size. Subnet A contained 50,000 3D points, and subnet B contained 48,000 3D points. The complexity of feature matching relationships in each subnet was analyzed to obtain the matching complexity. The matching complexity was comprehensively evaluated by calculating the number of matching point pairs, the uniformity of matching distribution, and matching quality indicators. The formula for calculating matching complexity is as follows: ,in This indicates that the matching complexity is dimensionless. This indicates that the number of matched point pairs is the actual statistical value. This represents a coefficient indicating the uniformity of the matching distribution, with values ​​between 0 and 1. The average reprojection error is expressed in pixel units. Based on actual calculations, the matching complexity of subnet A is 85, and the matching complexity of subnet B is 78. A pre-built subnet computational complexity evaluation model is used to assess the computational load of the image number, point cloud size, and matching complexity. The evaluation model formula is as follows: ,in This indicates that the calculated load assessment result is dimensionless. This indicates that the image quantity weighting coefficient is 0.4. This indicates that the number of images is based on actual statistical values. This indicates that the point cloud size weighting coefficient is 0.35. The point cloud size is represented by the actual number of 3D points in units of tens of thousands. This indicates that the matching complexity weight coefficient is 0.25. The matching complexity is dimensionless, calculated by substituting the data from subnet A. Similarly, subnet B is calculated to obtain Based on the evaluation results, 50 subnets were initially allocated to 10 distributed computing nodes using dynamic load balancing scheduling based on a work-stealing algorithm. Each node maintained a local task queue of 5 subnets, sorted in descending order of computational load. When distributed computing node 1 completed the computation of the 5 subnets in its local task queue, it detected that the current load of node 2 was 58, which was higher than its local load of 0. A subnet to be processed was randomly obtained from the tail of node 2's task queue. On each distributed computing node, the Levenburg-Marquardt algorithm was used to perform local bundle adjustment on the allocated subnets, including those in the local task queue and the subnet to be processed. Camera pose parameters and 3D point coordinates were solved iteratively. The optimization results were transmitted asynchronously and in parallel during the optimization process, ultimately obtaining the parameter optimization results for all 50 subnets, providing accurate input data for subsequent multi-level subnet fusion.

[0112] In this embodiment, by accurately assessing the computational load and dynamically allocating tasks, the rational utilization of computing resources and load balancing are achieved. Combined with efficient optimization algorithms, the rapid convergence and accuracy improvement of subnet parameters are ensured, thereby improving the overall efficiency of large-scale oblique photogrammetry data processing.

[0113] To further improve the accuracy and robustness of local bundle adjustment, in some embodiments, step 506: the local bundle adjustment of the assigned subnet using the Levenberg-Marquardt algorithm includes: Step 601: During the local bundle adjustment process, the sparse structure of the Hessian matrix is ​​used in conjunction with a preset cost function to reduce the dimensionality of the parameters of the assigned subnet. In this process, Cauchy and robust kernel functions are introduced into the cost function to reduce the impact of mismatched feature points on the parameters of the assigned subnet.

[0114] In step 601, the Hessian matrix is ​​the second derivative matrix of the cost function with respect to the optimization parameters, representing the curvature information of parameter changes. Sparse structure refers to the distribution characteristic where the vast majority of elements in the matrix are zero. The cost function measures the difference between theoretical and actual observations under the current parameters; its basic form is the sum of squares of the reprojection errors of all feature points. Dimensionality reduction is a process of reducing the scale of parameter solutions through mathematical transformations. The Cauchy bar kernel function is a mathematical function that suppresses the influence of outliers by assigning lower weights to larger residuals. Mismatched feature points refer to incorrectly matched feature point pairs due to texture duplication or occlusion.

[0115] In this embodiment, during local bundle adjustment, the sparse structure of the Hessian matrix is ​​first analyzed to identify the special correlation patterns between camera pose parameters and 3D point coordinate parameters. Based on this sparse structure, matrix factorization is used to reduce the dimensionality of the optimization problem, transforming the original problem into a smaller linear system. Simultaneously, a Cauchy rod kernel function is introduced into the cost function, which automatically identifies and reduces the weight of residual terms corresponding to mismatched feature points. Through iterative optimization using the Levenberg-Marquardt algorithm, the influence of mismatched points on parameter estimation is effectively suppressed while maintaining computational efficiency, ultimately obtaining accurate and stable optimization results.

[0116] In this embodiment, by utilizing the sparse structure of the Hessian matrix to improve computational efficiency and combining it with a robust kernel function to effectively suppress the effects of mismatches, the stability and reliability of local bundle adjustment are improved while ensuring optimization accuracy, laying a solid foundation for subsequent global fusion.

[0117] Optionally, step 506: Obtaining the parameter-optimized subnet by adjusting the parameters corresponding to the asynchronous parallel optimization subnet includes: Step 701: Optimize the parameters of the subnet after adjustment using an asynchronous parallel mode, and transmit the optimized parameters to the main distributed computing node. At the same time, the optimization progress data is periodically saved to the storage system through a checkpoint mechanism to obtain the subnet with optimized parameters. When the distributed computing node fails, the local bundle adjustment process is restored from the last successfully saved valid state based on the optimization progress data saved by the checkpoint mechanism.

[0118] In step 701, asynchronous parallel mode refers to a parallel computing method in which each computing node independently executes computing tasks without waiting for each other. The optimized parameters include data such as camera pose and 3D point coordinates obtained through local beam adjustment. The master distributed computing node is the central coordinating node responsible for task scheduling and result aggregation in the distributed computing architecture. This node is established during the initialization of the distributed computing environment and is used to receive and manage the optimization results returned by each computing node. The storage system refers to a storage device or storage service used to persistently save computing progress data. This storage system is configured and established during the initialization of the distributed computing environment and is directly related to the checkpointing mechanism and distributed computing nodes in this application. It is used to ensure that computing can be resumed from the saved optimization progress in the event of node failure. The checkpointing mechanism is a technique for fault recovery by periodically saving the computing progress. The optimization progress data includes intermediate results such as the current iteration number parameter value and convergence status. Valid state refers to a verified and usable intermediate computing state.

[0119] In this embodiment, after completing the local bundle adjustment optimization of the subnet, each distributed computing node immediately transmits the optimized parameters to the master distributed computing node without waiting for other nodes to complete their calculations. Simultaneously, each computing node saves its current optimization progress data to the storage system at preset time intervals or iteration intervals. When a distributed computing node fails, the system automatically detects the anomaly and reads the last successfully saved optimization progress data from the storage system. Based on this data, the local bundle adjustment process is restarted, continuing the calculation from the point of interruption until completion.

[0120] To further improve the accuracy and robustness of subnet fusion, in some embodiments, step 104: constructing a pose graph optimization model based on the common points and the camera pose information, and using the Gauss-Newton method to solve for the globally consistent transformation parameters based on the pose graph optimization model, includes: Step 801: Based on the common points, calculate the relative transformation matrix between adjacent subnets.

[0121] In step 801, the relative transformation matrix is ​​a four-by-four homogeneous transformation matrix that describes the rotation and translation relationship between the two subnet coordinate systems.

[0122] In this embodiment, based on the extracted common 3D points between adjacent subnets, the relative transformation matrix between the coordinate systems of the two subnets is calculated by least squares matching. This matrix includes rotation and translation components, thus establishing a preliminary spatial correspondence between the subnets.

[0123] Step 802: Based on the relative transformation matrix and the camera pose information, construct a pose graph optimization model, wherein the nodes of the pose graph optimization model represent the camera pose of the subnet, and the edges represent the relative transformation matrix between adjacent subnets.

[0124] In this embodiment, the camera pose of each subnet is used as a graph node, and the relative transformation matrix calculated in step 801 is used as the edge connecting adjacent subnet nodes to construct a pose graph optimization model that expresses the spatial constraint relationship between all subnets.

[0125] Step 803: Based on the nodes and edges of the pose graph optimization model, establish an error function, which characterizes the difference between the camera poses and relative transformation matrices of adjacent subnets.

[0126] In step 803, the error function is an objective function that quantifies the difference between observed and estimated values ​​in the pose graph model, characterizing the degree of inconsistency between the camera poses and relative transformation matrices of adjacent subnets. ,in Let represent the error function of the pose graph optimization model. This represents the set of poses of all subnet cameras. This represents the node indexes that represent different subnets in the pose graph. Represents a robust kernel function. Indicates adjacent subnets and The relative transformation matrix between them Subnet The camera pose matrix, Subnet The inverse of the camera pose matrix.

[0127] In this embodiment of the application, an error function based on the relative transformation residual is established according to the node and edge definitions of the pose graph model. This function measures the difference between the actual relative transformation between adjacent subnets and the relative transformation calculated by the camera pose.

[0128] Step 804: Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct the optimization problem.

[0129] In step 804, the robust kernel function is a mathematical function that suppresses the influence of outliers by assigning lower weights to larger residuals. The nonlinear optimization method is a numerical algorithm that iteratively solves a nonlinear minimization problem. The error function is used to quantify the difference between observed and estimated values ​​in the pose graph optimization model, while the robust kernel function is a special mathematical function embedded in the error function. It is used to weight the residual terms in the error function, suppressing the influence of outliers on the overall optimization result by reducing the weight of the residual terms corresponding to mismatched points. Together, they constitute the complete optimization objective function.

[0130] In this embodiment, a robust kernel function is introduced into the error function to construct a robust nonlinear optimization problem. This optimization problem aims to minimize the weighted residual sum of all relative transformation edges and automatically reduce the impact of mismatched common points on the optimization results.

[0131] Step 805: Iteratively calculate the optimization problem using the Gauss-Newton method to obtain the global consistency transformation parameters between subnets.

[0132] In step 805, the Gauss-Newton method is an iterative optimization algorithm for solving nonlinear least squares problems.

[0133] In this embodiment, the Gauss-Newton method is used to iteratively solve the constructed optimization problem. Through linear approximation and repeated iterations, the transformation parameters of all subnets are gradually optimized, and finally, globally consistent transformation parameters that minimize the global error are obtained.

[0134] Here is a specific example: Based on 5000 multi-angle images obtained from oblique photogrammetry of a certain urban area A, 50 subnets were formed after distributed processing. Adjacent subnets A and B were selected for fusion. First, the relative transformation matrix was calculated based on 1560 common points between the two subnets. The relative transformation matrix T_AB was obtained through least-squares fitting. This matrix is ​​a 4x4 homogeneous transformation matrix containing rotation component R_AB and translation component t_AB. The rotation matrix R_AB is a 3x3 orthogonal matrix [[0.9998,-0.0052,0.0183],[0.0053,0.9999,-0.0105],[-0.0182,0.0106,0.9998]], and the translation vector t_AB is [2.15,-1.08,0.43] units in meters. A pose graph optimization model is constructed based on the relative transformation matrix and the camera pose information of subnets A and B. This model contains 50 nodes representing the camera poses of the 50 subnets and 49 edges representing the relative transformation matrices between adjacent subnets. An error function is established based on the nodes and edges of the pose graph optimization model. Based on this error function, a nonlinear optimization method with a robust kernel function is used to construct the optimization problem, where the scaling parameter of the robust kernel function is set to 0.1. The optimization problem is iteratively solved using the Gauss-Newton method, with a maximum number of iterations of 20 and a convergence threshold of 1e-6. Convergence is achieved after 18 iterations, yielding the globally consistent transformation parameters among all subnets, including the pose transformation matrices of 50 subnets in the global coordinate system. The global transformation matrix of subnet A is... The global transformation matrix of subnet B is The root mean square of the relative transformation residuals between all adjacent subnets decreased from the initial 0.15 meters to 0.02 meters, achieving high-precision global consistency alignment of 50 subnets.

[0135] In this embodiment, the spatial constraint relationships between all subnets are effectively integrated through the pose graph optimization model. The influence of mismatch is suppressed by the robust optimization method. The high-precision global consistency transformation parameters are obtained by using the efficient Gauss-Newton method, which lays a solid foundation for building a seamless global 3D model.

[0136] To further improve the absolute accuracy and geographic reference accuracy of the global model, in some embodiments, step 105: when ground control points exist, the coordinates corresponding to the ground control points are used as fixed constraints, and the weighted least squares adjustment algorithm is used to optimize the global consistency transformation parameters as a whole, including: Step 901: Based on the global consistency transformation parameters, establish an adjustment optimization framework.

[0137] In step 901, the adjustment optimization framework is an overall computational structure that uses mathematical methods to uniformly process various observation data and parameters.

[0138] In this embodiment of the application, a unified adjustment optimization framework is constructed based on the obtained global consistency transformation parameters, which includes all subnet transformation parameters, camera pose and 3D point coordinates. This framework can process various types of observation data and optimize all parameters at the same time.

[0139] Step 902: Integrate the coordinates corresponding to the ground control points as fixed constraints into the adjustment optimization framework. In the adjustment optimization framework, the sliding window optimization method is used to adjust the parameters of the surrounding area of ​​the ground control points.

[0140] In step 902, sliding window optimization is an optimization strategy that gradually improves overall accuracy through local adjustments. The surrounding area of ​​the ground control point refers to the spatial influence range centered on the control point. The shape and size of this area are dynamically determined by the sliding window, depending on the geometric relationship between the control point and the surrounding camera stations and 3D points. It is not a fixed circle or rectangle, nor does it have a preset radius or length and width values. Instead, it is an adaptively adjusted range based on the actual network structure and accuracy requirements.

[0141] In this embodiment, the precise coordinates of the ground control points are incorporated into the adjustment framework as fixed constraints. A sliding window optimization method is adopted, with each control point as the center, to locally optimize and adjust the camera pose and 3D point parameters in the surrounding area, thereby gradually improving the model accuracy around the control point.

[0142] Step 903: Based on the adjusted parameters of the surrounding area, and combined with the variance component estimation method, adaptively optimize the weight ratio of various observations.

[0143] In step 903, variance component estimation is a method for determining the relative precision of different observation data through statistical analysis. The weight ratio is the relative weight proportion of different observation data in the adjustment. Adaptive optimization is the process of automatically adjusting the weight configuration based on data quality.

[0144] In this embodiment of the application, based on the residual statistics after adjusting the parameters of the surrounding area of ​​the control point, the variance component estimation method is used to analyze the accuracy level of different data sources such as image observations and control point observations, and adaptively optimizes to determine the optimal weight ratio configuration of various observations in the adjustment.

[0145] Step 904: Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized using the weighted least squares adjustment algorithm to obtain the optimization result.

[0146] In step 904, the optimization result is the optimal solution for all parameters obtained after global adjustment.

[0147] In this embodiment, based on the optimized weighting configuration, a weighted least squares adjustment algorithm is used to optimize the global consistency transformation parameters and the parameters of the control point's surrounding area. The optimal parameter combination that minimizes the sum of squared weighted residuals of all observations is obtained through iterative calculation. "Optimizing the global consistency transformation parameters and the adjusted parameters of the surrounding area" means simultaneously optimizing these two types of parameters using weighted least squares adjustment to obtain a unified optimization result. This result includes the final solution for the global consistency transformation parameters and the parameters of the control point's surrounding area after overall coordination.

[0148] Here is a specific example: Based on 5000 multi-angle images obtained from oblique photogrammetry of a certain urban area A, and after the aforementioned processing, globally consistent transformation parameters for 50 subnets were obtained. First, an adjustment optimization framework was established, containing the transformation parameters of the 50 subnets, 5000 camera pose parameters, and 2 million 3D point coordinate parameters. Ten ground control points were deployed in area A, and their corresponding known coordinates were incorporated into the adjustment optimization framework as fixed constraints, with a control point coordinate accuracy of 0.01 meters. Within the adjustment optimization framework, a sliding window optimization method was used to adjust the parameters within a 50-meter radius around each ground control point. This radius was determined based on half of the average control point spacing of 100 meters, and the window contained approximately 20 camera poses and 8000 3D points. Based on the adjusted surrounding area parameters, the weights of various observations were adaptively optimized using a variance component estimation method, where the variance component estimation formula is as follows: ,in Indicates the first The variance estimate of the observations is in square meters. Indicates the first The residual vector of the observations, Indicates the first The weight matrix of the observations, Indicates the first The redundant observations of the image observations were calculated to have a variance estimate of 0.0004 square meters, and the variance estimate of the control point observations was 0.0001 square meters. Therefore, the weight ratio of the two types of observations was determined to be 1:4. Based on the optimized weight ratio, the weighted least squares adjustment algorithm was used to optimize the global consistency transformation parameters and the adjusted surrounding area parameters. The adjustment model is as follows: ,in Represents the residual vector. Represents the design matrix. This indicates that the parameter vector to be determined includes global consistency transformation parameters and surrounding region parameters. Represents the observation vector, by minimizing Where P is the weight matrix, the adjustment converges after 5 iterations, yielding the optimized result. The mean square error of the control point residuals is calculated using the formula: The calculated result is 0.02 meters, an improvement from 0.05 meters before adjustment. The global consistency transformation parameters of all subnets have been further optimized, realizing high-precision georeference of the oblique photogrammetry 3D model of region A.

[0149] In this embodiment, absolute accuracy is ensured by fixed control point constraints. Combined with sliding window optimization and adaptive weight ratio determination, high-precision overall optimization of the global model is achieved, which improves the geographic reference accuracy and spatial consistency of the oblique photogrammetry 3D results.

[0150] Figure 3 A schematic diagram of a multi-level fusion optimization system for oblique photogrammetry aerial triangulation provided in this application embodiment is shown in the detailed implementation section. The acquisition module 31 is used to acquire multi-view image data using oblique photogrammetry equipment.

[0151] The construction module 32 is used to construct an image connection map based on the feature matching relationship between images in the multi-view image data, and to perform dynamic block processing on the image connection map by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form multiple sub-networks with internal connections and overlapping relationships.

[0152] Evaluation module 33 is used to evaluate the computational load of the number of images, point cloud size and matching complexity of the subnet. Based on the evaluation results, a work-stealing algorithm is used to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet in order to optimize the subnet parameters in parallel.

[0153] The fusion module 34 is used to perform multi-level subnet fusion on all subnets to determine the target overlapping region, extract common points and camera pose information from the target overlapping region corresponding to the subnet with optimized parameters, and construct a pose graph optimization model based on the common points and camera pose information. Based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method.

[0154] The optimization module 35 is used to optimize the global consistency transformation parameters by using the coordinates of the ground control points as fixed constraints and employing the weighted least squares adjustment algorithm when ground control points exist.

[0155] The adjustment module 36 is used to construct a global model based on the optimization results, and to perform lightweight bundle adjustment processing on the global model to generate aerial triangulation processing results.

[0156] The multi-level fusion optimization system for oblique aerial photogrammetry in this application is used to implement the aforementioned multi-level fusion optimization method for oblique aerial photogrammetry. Therefore, the specific implementation of the multi-level fusion optimization system for oblique aerial photogrammetry can be found in the embodiment section of the multi-level fusion optimization method for oblique aerial photogrammetry above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0157] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0158] The above provides a detailed description of a multi-level fusion optimization method and system for oblique photogrammetry aerial triangulation provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A multi-level fusion optimization method for oblique photogrammetry aerial triangulation, characterized in that, include: Obtain multi-view image data using oblique photogrammetry equipment; Based on the feature matching relationship between images in the multi-view image data, an image connection graph is constructed, and a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm is used to dynamically divide the image connection graph into blocks to form multiple subnets with internal connections and overlapping relationships. The computational load of the subnet is evaluated based on the number of images, point cloud size and matching complexity. Based on the evaluation results, a work-stealing algorithm is used to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet in order to optimize the subnet parameters in parallel. Multi-level subnet fusion is performed on all subnets to determine the target overlapping region. Common points and camera pose information are extracted from the target overlapping region corresponding to the subnet with optimized parameters. Based on the common points and camera pose information, a pose graph optimization model is constructed. Based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method. In the presence of ground control points, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized as a whole using the weighted least squares adjustment algorithm. Based on the optimization results, a global model is constructed, and a lightweight beam adjustment process is performed on the global model to generate the aerial triangulation results.

2. The method according to claim 1, characterized in that, The step of constructing an image connectivity graph based on feature matching relationships between images in the multi-view image data includes: A feature extraction algorithm based on deep learning is used to extract feature points that represent image content from the multi-view image data. The feature points are matched using the nearest neighbor search and geometric verification algorithm to establish feature matching relationships between images. The nearest neighbor search and geometric verification algorithm is constructed based on KD-Tree FLANN matching and RANSAC fundamental matrix estimation. Based on the feature matching relationship, an image connection graph is constructed, wherein the nodes in the image connection graph represent images, the edges represent feature matching relationships between images, and the weight of the edge is a comprehensive metric determined based on the number of matching feature points between images, the reprojection error of matching feature point pairs, and the co-view geometric relationship.

3. The method according to claim 1, characterized in that, The image connectivity map is dynamically segmented using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form multiple subnetworks with internal connectivity and overlap relationships, including: Based on the edge weights of the image connectivity graph, a weighted adjacency matrix and a degree matrix are constructed using a weighted spectral clustering algorithm, and the Laplacian matrix is ​​calculated based on the weighted adjacency matrix and the degree matrix. The Laplacian matrix is ​​subjected to eigenvalue decomposition to obtain multiple eigenvalues; Select the k smallest eigenvalues ​​from the plurality of eigenvalues, and construct a new feature space based on the eigenvectors corresponding to the k smallest eigenvalues, where k is greater than or equal to 2; Using a multi-constraint spectral clustering algorithm, the image set is divided into k clusters that satisfy preset constraints in the new feature space through iterative optimization. Based on the k clusters, multiple subnets are constructed, wherein the constraints include subnet size balance constraints and minimum common point number constraints between subnets.

4. The method according to claim 1, characterized in that, The step of performing multi-level subnet fusion on all subnets to determine the target overlapping area includes: Based on the common images and common points of adjacent subnets, identify the common coverage areas between adjacent subnets; By analyzing the three-dimensional spatial points and camera pose information contained in each subnet, the initial overlapping area between adjacent subnets is located in the common coverage area; Within the initial overlapping region, extract the shared three-dimensional spatial points and corresponding camera observation information of adjacent subnets; Based on the shared three-dimensional spatial points and the corresponding camera observation information, the coordinate transformation relationship between adjacent subnets is determined; Based on the coordinate transformation relationship and the camera pose information, the initial overlapping region is precisely defined to form the target overlapping region.

5. The method according to claim 1, characterized in that, The computational load is evaluated based on the number of images, point cloud size, and matching complexity of the subnets. Based on the evaluation results, a work-stealing algorithm is used to allocate each subnet to a corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnets to optimize the subnet parameters in parallel, including: The point cloud scale is obtained by counting the number of images contained in each subnet and calculating the total number of 3D spatial points in each subnet. Analyze the complexity of feature matching relationships in each subnet to obtain the corresponding matching complexity; Using a pre-built subnet computational complexity evaluation model, the computational load of the number of images, the point cloud size, and the matching complexity is evaluated to obtain the evaluation results; Based on the evaluation results, each subnet is initially allocated to the corresponding distributed computing nodes through dynamic load balancing scheduling based on the work-stealing algorithm, wherein each distributed computing node maintains a corresponding local task queue. After a distributed computing node completes the computation of a subnet in its local task queue, it randomly obtains the corresponding subnet to be processed from the task queues of other distributed computing nodes whose load is greater than its local load. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the assigned subnets, and the parameters of the adjusted subnets are optimized asynchronously and in parallel to obtain parameter-optimized subnets. The assigned subnets include subnets in the local task queue and subnets to be processed.

6. The method according to claim 5, characterized in that, The process of using the Levenberg-Marquardt algorithm to perform local bundle adjustment on the assigned subnets includes: During the local bundle adjustment process, the sparse structure of the Hessian matrix is ​​used in conjunction with a preset cost function to reduce the dimensionality of the parameters of the assigned subnet. In this process, Cauchy and robust kernel functions are introduced into the cost function to reduce the impact of mismatched feature points on the parameters of the assigned subnet.

7. The method according to claim 5, characterized in that, The parameters corresponding to the subnet after the asynchronous parallel optimization adjustment are used to obtain the parameter-optimized subnet, including: The parameters of the subnet are optimized using an asynchronous parallel mode, and the optimized parameters are transmitted to the main distributed computing node. At the same time, the optimization progress data is periodically saved to the storage system through a checkpoint mechanism to obtain the subnet with optimized parameters. When the distributed computing node fails, the local bundle adjustment process is restored from the last successfully saved valid state based on the optimization progress data saved by the checkpoint mechanism.

8. The method according to claim 1, characterized in that, The process involves constructing a pose graph optimization model based on the common points and the camera pose information, and then using the Gauss-Newton method to solve for the globally consistent transformation parameters based on the pose graph optimization model. This includes: Based on the common points, calculate the relative transformation matrix between adjacent subnets; Based on the relative transformation matrix and the camera pose information, a pose graph optimization model is constructed, wherein the nodes of the pose graph optimization model represent the camera pose of the subnet, and the edges represent the relative transformation matrix between adjacent subnets. Based on the nodes and edges of the pose graph optimization model, an error function is established, which characterizes the difference between the camera poses and the relative transformation matrix of adjacent subnets. Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct the optimization problem. The optimization problem is solved iteratively using the Gauss-Newton method to obtain the global consistency transformation parameters between subnets.

9. The method according to claim 1, characterized in that, In the case of existing ground control points, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized using a weighted least squares adjustment algorithm, including: Based on the aforementioned global consistency transformation parameters, an adjustment optimization framework is established; The coordinates corresponding to the ground control points are incorporated into the adjustment optimization framework as fixed constraints. In the adjustment optimization framework, the sliding window optimization method is used to adjust the parameters of the surrounding area of ​​the ground control points. Based on the adjusted parameters of the surrounding area, and combined with the variance component estimation method, the weight ratios of various observations are adaptively optimized. Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized using the weighted least squares adjustment algorithm to obtain the optimization results.

10. A multi-level fusion optimization system for oblique photogrammetry aerial triangulation, characterized in that, include: The acquisition module is used to acquire multi-view image data using oblique photogrammetry equipment; The construction module is used to construct an image connectivity graph based on the feature matching relationship between images in the multi-view image data, and to perform dynamic block processing on the image connectivity graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form multiple subnets with internal connections and overlapping relationships. The evaluation module is used to evaluate the computational load of the number of images, point cloud size and matching complexity of the subnet. Based on the evaluation results, the work-stealing algorithm is used to allocate each subnet to the corresponding distributed computing node. On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet in order to optimize the subnet parameters in parallel. The fusion module is used to perform multi-level subnet fusion on all subnets to determine the target overlapping region. It extracts common points and camera pose information from the target overlapping region corresponding to the subnet with optimized parameters, and constructs a pose graph optimization model based on the common points and camera pose information. Based on the pose graph optimization model, the global consistency transformation parameters are obtained by solving the Gauss-Newton method. The optimization module is used to optimize the global consistency transformation parameters as a whole by using the coordinates of the ground control points as fixed constraints and employing the weighted least squares adjustment algorithm when ground control points exist. The adjustment module is used to construct a global model based on the optimization results, and to perform lightweight bundle adjustment processing on the global model to generate aerial triangulation processing results.

Images