Multi-level fusion optimization method and system for oblique photogrammetric space triangulation processing

Abstract

The application provides a multi-level fusion optimization method and system for oblique photogrammetry space three processing, and relates to the technical field of photogrammetry, wherein the method comprises: acquiring multi-view image data and constructing an image connection graph, using weighted spectral clustering combined with multi-constraint spectral clustering for dynamic blocking to form a subnetwork with internal connection and overlapping relationship. Based on the image quantity, point cloud scale and matching complexity evaluation and calculation load, the subnetwork is distributed to distributed computing nodes, and the subnetwork parameters are optimized in parallel. The target overlapping area is determined through multi-level subnetwork fusion, the public points and camera pose are extracted to construct a pose graph model, and the Gauss-Newton method is used to solve the global consistent transformation parameters. Under the constraint of ground control points, overall optimization is carried out through weighted least squares adjustment, a global model is finally constructed, light beam adjustment is performed, and the space three processing result is generated. The application improves the automation degree and three-dimensional reconstruction precision of oblique photogrammetry data processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photogrammetry, in particular to a multi-level fusion optimization method and system for oblique photogrammetry aerial triangulation processing. BACKGROUND

[0002] With the wide application of oblique photogrammetry technology in smart city, geological disaster monitoring and other fields, higher requirements are put forward for efficient processing and accurate three-dimensional reconstruction of large-scale oblique photogrammetry data. The current processing flow needs to deal with tens of thousands of high-definition images with a data size, while ensuring the accuracy and computing efficiency of aerial triangulation processing, which puts forward urgent needs for distributed processing and automatic fusion technology.

[0003] In the prior art, a distributed aerial triangulation processing method based on geographic grid division is adopted. The method divides the survey area into multiple sub-areas according to the regular grid, independently performs local aerial triangulation calculation on each computing node, and finally converts the results of each sub-area to the global coordinate system. The method mainly depends on geographic coordinate range during division, and maintains continuous calculation within each sub-area.

[0004] However, when dealing with complex terrain and densely built-up areas, the geographic grid boundary is easy to cut the strong connection area, resulting in insufficient number of connection points at the junction of sub-areas, affecting the subsequent fusion accuracy. At the same time, the fixed grid division method has limited adaptability to uneven data distribution, which may cause uneven computing load, and some nodes have large computing pressure while other nodes are idle, affecting the overall processing efficiency. SUMMARY

[0005] The present application provides a multi-level fusion optimization method and system for oblique photogrammetry aerial triangulation processing to solve the problems of low automation and low three-dimensional reconstruction accuracy in the prior art.

[0006] To solve the above technical problems, in a first aspect, the present application provides a multi-level fusion optimization method for oblique photogrammetry aerial triangulation processing, comprising:

[0007] Obtaining multi-view image data by using an oblique photogrammetry device;

[0008] Based on the feature matching relationship between the images in the multi-view image data, an image connection graph is constructed, and a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm is used to perform dynamic block processing on the image connection graph to form multiple sub-networks with internal connection and overlap relationship;

[0009] Performing load evaluation on the number of images, the size of point clouds and the matching complexity of the subnets, based on the evaluation results, using a work-stealing algorithm to distribute each subnet to a corresponding distributed computing node, and using a Levenberg-Marquardt algorithm to perform local bundle adjustment processing on the subnet on each distributed computing node to optimize the subnet parameters in parallel;

[0010] Performing multi-level subnet fusion on all subnets to determine a target overlapping area, extracting common points and camera pose information from the target overlapping area corresponding to the subnet after parameter optimization, and based on the common points and the camera pose information, constructing a pose graph optimization model, and based on the pose graph optimization model, using a Gauss-Newton method to obtain global consistency transformation parameters;

[0011] In the presence of ground control points, using a weighted least squares adjustment algorithm to optimize the global consistency transformation parameters as a fixed constraint;

[0012] Based on the optimization results, constructing a global model, and performing lightweight bundle adjustment processing on the global model to generate aerial triangulation results.

[0013] Optionally, the image connection graph is constructed based on the feature matching relationship between the images in the multi-view image data, comprising:

[0014] Based on a deep learning-based feature extraction algorithm, extracting feature points representing image content from the multi-view image data;

[0015] Using a nearest neighbor search and geometric verification algorithm to match the feature points to establish a feature matching relationship between the images, the nearest neighbor search and geometric verification algorithm is constructed based on a KD-Tree-based FLANN matching and RANSAC fundamental matrix estimation;

[0016] Based on the feature matching relationship, an image connection graph is constructed, wherein the nodes in the image connection graph represent images, and the edges represent the existence of feature matching relationships between images, and the weight of the edge is a comprehensive measure determined based on the number of matching feature points, the reprojection error of matching feature point pairs and the co-view geometric relationship.

[0017] Optionally, the weighted spectral clustering algorithm is used in combination with the multi-constraint spectral clustering algorithm to perform dynamic block processing on the image connection graph to form a plurality of subnets with internal connections and overlapping relationships, comprising:

[0018] Based on the edge weights of the image connection graph, a weighted adjacency matrix and a degree matrix are constructed using a weighted spectral clustering algorithm, and based on the weighted adjacency matrix and the degree matrix, a Laplacian matrix is calculated;

[0019] performing eigenvalue decomposition on the Laplacian matrix to obtain a plurality of eigenvalues;

[0020] selecting the first k minimum eigenvalues from the plurality of eigenvalues, and constructing a new feature space based on eigenvectors corresponding to the first k minimum eigenvalues, where k is greater than or equal to 2;

[0021] using a multi-constraint spectral clustering algorithm, dividing the image set into k clusters that satisfy a preset constraint condition through iterative optimization in the new feature space, and constructing a plurality of subnets based on the k clusters, wherein the constraint condition includes a subnet size balance constraint and a minimum common point number constraint between subnets.

[0022] Optionally, the multi-level subnet fusion is performed on all subnets to determine the target overlap region, including:

[0023] identifying a common coverage area between adjacent subnets based on common images and common points of adjacent subnets;

[0024] locating an initial overlap region between adjacent subnets in the common coverage area by analyzing three-dimensional space points and camera pose information contained in each subnet;

[0025] extracting three-dimensional space points and corresponding camera observation information common to adjacent subnets within the initial overlap region;

[0026] determining a coordinate transformation relationship between adjacent subnets based on the common three-dimensional space points and corresponding camera observation information;

[0027] accurately defining the initial overlap region according to the coordinate transformation relationship and the camera pose information to form a target overlap region.

[0028] Optionally, the image number, point cloud size and matching complexity of the subnet are calculated and load evaluated, based on the evaluation result, each subnet is distributed to a corresponding distributed computing node using a work-stealing algorithm, and the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet on each distributed computing node to optimize the subnet parameters in parallel, including:

[0029] obtaining the point cloud size by counting the number of images contained in each subnet and calculating the total number of three-dimensional space points in each subnet;

[0030] analyzing the complexity of feature matching relationship in each subnet to obtain the corresponding matching complexity;

[0031] using a pre-constructed subnet computing complexity evaluation model to perform load evaluation on the image number, the point cloud size and the matching complexity to obtain an evaluation result;

[0032] based on the evaluation result, each subnet is initially allocated to a corresponding distributed computing node through dynamic load balancing scheduling based on a work-stealing algorithm, wherein each distributed computing node maintains a corresponding local task queue;

[0033] When the distributed computing node completes the subnet calculation in the local task queue, a corresponding to-be-processed subnet is randomly obtained from the task queue of another distributed computing node with a load greater than the local load;

[0034] On each distributed computing node, a Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the allocated subnet, asynchronous parallel optimization is performed on the parameters of the adjusted subnet, and a parameter-optimized subnet is obtained, wherein the allocated subnet includes the subnet in the local task queue and the to-be-processed subnet.

[0035] Optionally, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the allocated subnet, and the method comprises the following steps:

[0036] In the local bundle adjustment process, the parameters of the allocated subnet are processed in a reduced dimension manner by using the sparse structure of the Hessian matrix and combining a preset cost function, wherein a Cauchy and robust kernel function is introduced in the cost function to reduce the influence of mis-matched feature points on the parameters of the allocated subnet.

[0037] Optionally, the asynchronous parallel optimization is performed on the parameters of the adjusted subnet to obtain the parameter-optimized subnet, and the method comprises the following steps:

[0038] The asynchronous parallel mode is used to optimize the parameters of the adjusted subnet, and the optimized parameters are transmitted to the master distributed computing node; at the same time, the optimization progress data are saved to a storage system at regular intervals through a checkpoint mechanism to obtain the parameter-optimized subnet, wherein when the distributed computing node fails, the local bundle adjustment process is restored from the last successfully saved valid state based on the optimization progress data saved by the checkpoint mechanism.

[0039] Optionally, the pose graph optimization model is constructed based on the common points and the camera pose information, and the globally consistent transformation parameters are obtained by using a Gauss-Newton method based on the pose graph optimization model, and the method comprises the following steps:

[0040] Based on the common points, a relative transformation matrix between adjacent subnets is calculated;

[0041] Based on the relative transformation matrix and the camera pose information, a pose graph optimization model is constructed, wherein a node of the pose graph optimization model represents a camera pose of a subnet, and an edge represents a relative transformation matrix between adjacent subnets;

[0042] An error function is established according to nodes and edges of the pose graph optimization model, and the error function represents a difference between adjacent subnetwork camera poses and relative transformation matrices.

[0043] Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct an optimization solving problem.

[0044] The global consistency transformation parameters between the subnetworks are solved by iteratively calculating the optimization solving problem by using a Gauss-Newton method.

[0045] Optionally, in the presence of ground control points, the coordinates corresponding to the ground control points are taken as fixed constraints, and the global consistency transformation parameters are optimized as a whole by using a weighted least squares adjustment algorithm, including:

[0046] Based on the global consistency transformation parameters, an adjustment optimization framework is established.

[0047] The coordinates corresponding to the ground control points are taken as fixed constraints and integrated into the adjustment optimization framework, and in the adjustment optimization framework, a sliding window optimization method is used to adjust the parameters of the surrounding area of the ground control points.

[0048] Based on the adjusted parameters of the surrounding area, a variance component estimation method is used to adaptively optimize the weight ratios of various types of observations.

[0049] Based on the optimized weight ratios, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized by using a weighted least squares adjustment algorithm to obtain an optimization result.

[0050] In a second aspect, the application provides a multi-level fusion optimization system for oblique photogrammetric space triangulation, including:

[0051] An acquisition module is configured to acquire multi-view image data by using an oblique photogrammetric measurement device.

[0052] A construction module is configured to construct an image connection graph based on a feature matching relationship between images in the multi-view image data, and perform dynamic block processing on the image connection graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form a plurality of subnetworks having internal connections and overlapping relationships.

[0053] An evaluation module is configured to calculate load evaluation of the number of images, the size of point clouds, and the matching complexity of the subnetworks, and based on the evaluation results, to distribute each subnetwork to a corresponding distributed computing node by using a work-stealing algorithm, and to perform local bundle adjustment processing on the subnetworks by using a Levenberg-Marquardt algorithm on each distributed computing node to optimize the parameters of the subnetworks in parallel.

[0054] a fusion module configured to perform multi-level subnet fusion on all subnets to determine a target overlapping area, extract common points and camera pose information from the target overlapping area corresponding to the parameter-optimized subnets, and construct a pose graph optimization model based on the common points and the camera pose information, and solve global consistency transformation parameters by using a Gauss-Newton method based on the pose graph optimization model;

[0055] an optimization module configured to, in the presence of ground control points, use a weighted least squares adjustment algorithm to perform overall optimization on the global consistency transformation parameters by taking coordinates corresponding to the ground control points as fixed constraints;

[0056] an adjustment module configured to, based on the optimization result, construct a global model, and perform lightweight bundle adjustment processing on the global model to generate an aerial triangulation processing result.

[0057] In the present application, a multi-level fusion optimization method for oblique photography aerial triangulation processing is provided, which includes: acquiring multi-view image data by using an oblique photogrammetry device; constructing an image connection graph based on a feature matching relationship between images in the multi-view image data, and performing dynamic block processing on the image connection graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form a plurality of subnets having internal connections and overlapping relationships; performing load evaluation on the number of images, the size of point clouds, and the matching complexity of the subnets, assigning each subnet to a corresponding distributed computing node by using a work-stealing algorithm based on the evaluation result, performing local bundle adjustment processing on the subnets by using a Levenberg-Marquardt algorithm on each distributed computing node to optimize subnet parameters in parallel; performing multi-level subnet fusion on all subnets to determine a target overlapping area, extracting common points and camera pose information from the target overlapping area corresponding to the parameter-optimized subnets, and constructing a pose graph optimization model based on the common points and the camera pose information, and solving global consistency transformation parameters by using a Gauss-Newton method based on the pose graph optimization model; in the presence of ground control points, taking coordinates corresponding to the ground control points as fixed constraints, and using a weighted least squares adjustment algorithm to perform overall optimization on the global consistency transformation parameters; based on the optimization result, constructing a global model, and performing lightweight bundle adjustment processing on the global model to generate an aerial triangulation processing result.

[0058] The technical solution provided in the present application has the following beneficial effects:

[0059] The present application provides complete and reliable original data basis for subsequent processing, ensures the comprehensiveness and accuracy of the data source. According to the actual connection strength between images, intelligent division is performed, which not only ensures the close connection within the subnet, but also reserves sufficient overlapping areas for subsequent fusion. The reasonable allocation and efficient use of computing resources are realized, the uneven node load is avoided, and the overall computing efficiency is improved. Through pose graph optimization, high-precision alignment between subnets is ensured, and accurate transformation parameters are provided for global model construction. Combined with ground control points, the absolute accuracy of the model is improved, and the geographic reference accuracy of the results is ensured. Further optimize the internal consistency of the model, eliminate system errors, and improve the quality of the final results.

[0060] Further, the present application also acquires image feature points through a deep learning feature extraction algorithm, adopts FLANN matching based on KD-Tree and RANSAC fundamental matrix estimation for feature point matching and geometric verification, establishes the feature matching relationship between images, and constructs an image connection graph based on this. The connection graph takes images as nodes and matching relationships as edges, and the edge weight comprehensively considers the number of matching points, the reprojection error and the common view geometric relationship, forming a graph structure that accurately reflects the image connection strength.

[0061] Moreover, the scheme can establish a graph model that accurately reflects the actual connection relationship between images, providing a reliable basis for subsequent intelligent block division, effectively improving the accuracy and robustness of feature matching, and laying a solid foundation for the overall processing flow.

[0062] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0064] Figure 1 A flow chart of a multi-level fusion optimization method for oblique photography space triangulation processing provided by the embodiment of the present application;

[0065] Figure 2 A specific implementation schematic diagram of a multi-level fusion optimization method for oblique photography space triangulation processing provided by the embodiment of the present application;

[0066] Figure 3 A structure schematic diagram of a multi-level fusion optimization system for oblique photography space triangulation processing provided by the embodiment of the present application. DETAILED DESCRIPTION

[0067] In the large-scale aerial triangulation processing of oblique photogrammetry, the existing distributed processing method based on geographic grid faces challenges. This method adopts fixed grid division of the survey area. When processing complex urban terrain, the grid boundary often cuts the building group and terrain feature area with strong connection relationship, resulting in weak connection constraint at the junction of sub-regions. At the same time, the fixed division mode is difficult to adapt to the unevenness of data distribution, which easily causes load imbalance among computing nodes, affecting the overall processing efficiency. These factors jointly restrict the accuracy and efficiency of large-scale oblique photogrammetric data processing.

[0068] In view of the above problems, the present application provides a multi-level fusion optimization method for oblique photogrammetric aerial triangulation processing. The method first constructs a connection graph according to the actual matching relationship between images, and divides the survey area into subnets with close internal connection and sufficient overlapping area through intelligent clustering; then establishes a computing load evaluation mechanism to realize dynamic allocation and parallel optimization of subnet tasks; finally, through multi-level fusion and overall adjustment, the consistency of the global model is ensured. This method breaks through the limitation of fixed grid division, ensures the effective connection between subnets through data-driven intelligent block division, improves the utilization rate of computing resources by combining dynamic load balancing, realizes high-precision and high-efficiency processing of super large-scale oblique photogrammetric data, and effectively solves the problems of precision loss and low efficiency caused by unreasonable division in the prior art.

[0069] In order to enable the personnel in the technical field to better understand the present application scheme, the present application will be further described in detail below in combination with the drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor belong to the scope of protection of the present application.

[0070] The core of the present application is to provide a multi-level fusion optimization method for oblique photogrammetric aerial triangulation processing, and a specific embodiment thereof is shown in the flowchart as shown in the figure. The method comprises: Figure 1

[0071] Step 101: acquiring multi-view image data by using oblique photogrammetric equipment.

[0072] In step 101, the oblique photogrammetric equipment is a special camera system capable of simultaneously collecting vertical and oblique view images, usually mounted on a UAV or aerial platform. Multi-view image data refers to a set of images taken from different angles of the same area by oblique photogrammetric equipment. Image data includes pixel array, geographic position information and shooting parameters, which constitutes the basic data source for subsequent processing.

[0073] ​In the embodiments of the present application, the target area is flown over by a tilt photography measurement device carrying a multi-lens camera, the device automatically collects images according to a preset flight path and overlap during flight, vertical and multiple tilt angle images are obtained at each shooting point, and all image data together with corresponding position and attitude information are stored as an original data set to provide complete input data for subsequent processing.

[0074] For example, for tilt photography measurement of a certain urban area A, a five-lens tilt photography camera is carried on a UAV platform, flies along a preset flight path, the heading overlap is set to 80%, the lateral overlap is set to 70%, the flight height is 300 meters, a total of 5000 multi-angle images are obtained, each image is attached with POS data including latitude, longitude, elevation and attitude angle, and all data are stored in a general image format for subsequent processing.

[0075] Step 102: based on the feature matching relationship between images in the multi-view image data, an image connection graph is constructed, and a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm is used to perform dynamic block processing on the image connection graph to form a plurality of subnets with internal connection and overlap relationship.

[0076] In step 102, the image connection graph is a graph structure, in which a node represents a single image, an edge represents a feature matching relationship between images, and an edge weight comprehensively reflects the number of matching points, geometric accuracy and common view intensity. Dynamic block processing is a process of adaptively dividing the data set according to the actual connection relationship. The subnet is an image subset obtained by clustering analysis, which has close internal connection and retains an overlap area with other subnets.

[0077] In the embodiments of the present application, first, feature points are extracted from multi-view images and matched to establish the connection relationship between images; then a weighted image connection graph is constructed, and a weighted spectral clustering algorithm is used based on the connection graph for division, considering the subnet size balance and minimum common point number constraint in the clustering process, the image set is divided into multiple subnets through feature decomposition and clustering analysis, each subnet has close internal connection and maintains sufficient overlap area with adjacent subnets.

[0078] For example, feature extraction and matching are performed on the 5000 images obtained in step 101 to construct an image connection graph containing 5000 nodes, and the weight of the edge is calculated from the number of matching points, re-projection error and common view geometric relationship. The image set is divided into 50 subnets by using the weighted spectral clustering algorithm, each subnet contains 80-120 images, and at least 20% overlap is ensured between adjacent subnets to ensure close internal connection of each subnet and sufficient common points between subnets.

[0079] Step 103: Calculate the load evaluation of the number of images, point cloud size and matching complexity of the subnets, based on the evaluation results, use the work-stealing algorithm to distribute each subnet to the corresponding distributed computing node, and use the Levenberg-Marquardt algorithm to perform local bundle adjustment processing on the subnet on each distributed computing node to optimize the subnet parameters in parallel.

[0080] In step 103, the calculation of the load evaluation is a quantitative analysis of the subnet processing complexity, including the number of images, the size of the point cloud and the matching complexity. The distributed computing node represents an independent processing unit participating in parallel computing in this application. These nodes are connected through a computer network to form a distributed computing environment. Each node is responsible for processing the assigned subnet data and has independent computing power and storage space. Through cooperative work, large-scale oblique photography data can be efficiently processed. The work-stealing algorithm is a dynamic task allocation mechanism, and local bundle adjustment is the process of solving the camera pose and three-dimensional point coordinates in the subnet through optimization algorithms.

[0081] In the embodiments of the present application, the number of images, the size of the point cloud and the matching complexity of each subnet are first evaluated to establish a calculation load model. Based on the evaluation results, the work-stealing algorithm is used to distribute the subnets to the distributed computing nodes. Each node uses the Levenberg-Marquardt algorithm to perform local bundle adjustment on the assigned subnet. In the optimization process, matrix sparsity is used to accelerate calculation, and a robust kernel function is introduced to suppress the influence of false matches. Asynchronous communication and checkpoint mechanisms are used to ensure the reliability and efficiency of the calculation.

[0082] For example, 50 subnets obtained in step 102 are evaluated. Subnet A contains 100 images, 50,000 three-dimensional points, and the matching complexity rating is high. It is assigned to computing node 1. The work-stealing algorithm is used to dynamically adjust task allocation. Node 1 uses the Levenberg-Marquardt algorithm to perform local bundle adjustment on subnet A. After 15 iterations, the camera pose and three-dimensional point coordinates converge. The optimization results are transmitted back to the master node through asynchronous communication. A checkpoint is saved every 10% progress.

[0083] Step 104: Perform multi-level subnet fusion on all subnets to determine the target overlap region. Extract common points and camera pose information from the target overlap region corresponding to the parameter-optimized subnet. Based on the common points and the camera pose information, construct a pose graph optimization model. Based on the pose graph optimization model, use the Gauss-Newton method to solve the global consistency transformation parameters.

[0084] In step 104, the target overlap region is the spatial range common to adjacent subnets. The common points are three-dimensional space points common to multiple subnets. The camera pose information includes camera position and attitude parameters. The pose graph optimization model is a graph structure that describes the spatial relationship between subnets. The global consistency transformation parameters are the transformation matrix that unifies the coordinate systems of each subnet.

[0085] In the embodiment of the application, the overlapping area is determined according to the subnet division, the common points and camera poses are extracted from the optimized subnet, a pose graph optimization model is constructed based on the information, wherein the nodes represent the subnet poses and the edges represent the relative transformation relationship, an error function is defined to describe the difference between the actual observation and the model prediction, and a nonlinear optimization method with a robust kernel function is used to iteratively solve the optimal global consistency transformation parameters by the Gauss-Newton method.

[0086] For example, the overlapping area of subnet A and subnet B is identified, 2000 common points and corresponding camera poses are extracted, a pose graph model is constructed, an error function is defined and a robust kernel function is added, the Gauss-Newton method is used for optimization, and the transformation parameters between subnet A and subnet B, including the rotation matrix and the translation vector, are obtained after 20 iterations, so as to realize high-precision alignment of the two subnets.

[0087] Step 105: In the presence of ground control points, the coordinates corresponding to the ground control points are taken as fixed constraints, and the global consistency transformation parameters are optimized as a whole by using a weighted least squares adjustment algorithm.

[0088] In step 105, the presence of ground control points means that ground marker points with known accurate coordinates are arranged in the survey area. The coordinate data of these control points can be introduced into the aerial triangulation process as absolute constraints to improve the absolute accuracy and geographic reference accuracy of the model. The ground control point is a ground marker point with known accurate coordinates, the fixed constraint is to take the control point coordinates as a non-adjustable reference condition, and the weighted least squares adjustment algorithm is a method for solving optimal parameters by minimizing the weighted residual sum of squares.

[0089] In the embodiment of the application, the ground control point coordinates are taken as fixed constraints and integrated into the optimization framework, the camera poses and three-dimensional point parameters in the surrounding area of the control points are optimized by using the sliding window method, the weight proportion of each type of observation value is adaptively adjusted through variance component estimation, and the global consistency transformation parameters and related parameters are optimized and solved as a whole by using the weighted least squares adjustment algorithm based on the adjusted weight configuration.

[0090] For example, 10 ground control points arranged in area A are taken as fixed constraints, the parameters within a range of 50 meters around the control points are optimized by using the sliding window method, the weight ratio of image observation values and control point observation values is adjusted through variance component estimation, and the global transformation parameters are optimized as a whole by using the weighted least squares adjustment. After adjustment, the control point residual error is 0.02 meters, which meets the accuracy requirement.

[0091] Step 106: Based on the optimization result, a global model is constructed, and a lightweight bundle adjustment process is performed on the global model to generate an aerial triangulation result.

[0092] In step 106, the global model is the three-dimensional reconstruction result integrating all subnet data, the lightweight bundle adjustment is the process of quickly and finely optimizing the global model, and the aerial triangulation result is the final achievement containing accurate camera poses and three-dimensional point coordinates.

[0093] In the embodiments of the present application, a unified global three-dimensional model is constructed based on the overall optimization result, lightweight bundle adjustment is performed on all camera poses and three-dimensional point coordinates, residual errors are eliminated through a limited number of iterations, global transformation parameters remain unchanged, and only other parameters are fine-tuned, and finally the aerial triangulation result containing accurate poses and three-dimensional point clouds is output.

[0094] For example, a global model is constructed by integrating 50 subnet data, containing 5000 camera poses and 2 million three-dimensional points; after 5 iterations, the model accuracy is improved and the maximum reprojection error is reduced to 1.5 pixels through lightweight bundle adjustment; and the final aerial triangulation result is output, including the optimized camera parameters, three-dimensional point cloud and accuracy evaluation report.

[0095] The method realizes efficient processing of large-scale oblique photography data through intelligent partitioning, load balancing distributed computing and multi-level fusion optimization, improves the automation degree and achievement quality of aerial triangulation, ensures the high precision and global consistency of the three-dimensional model, and provides reliable technical support for real scene three-dimensional modeling.

[0096] To solve the precision and efficiency problem of image connection relationship establishment in oblique photography data processing, in some embodiments, step 102: based on the feature matching relationship between images in the multi-view image data, an image connection graph is constructed, such as Figure 2 The method comprises the following steps:

[0097] Step 201: based on a deep learning-based feature extraction algorithm, feature points for representing image content are extracted from the multi-view image data.

[0098] In step 201, the deep learning-based feature extraction algorithm automatically identifies the positions and description information of stable and unique feature points in the image through a trained neural network model. Image content refers to the visual feature information of the scene and objects recorded in the image, including texture, edge, corner and other visual patterns with recognition degree, which can uniquely represent a specific local area in the image and be used for establishing correspondence between different images. Feature points refer to pixel positions with visual features in the image, which can uniquely represent local content of the image.

[0099] In the embodiments of the present application, a pre-trained feature extraction network is used to process each oblique photography image, the network automatically analyzes the image content and outputs the coordinates and feature description data of the feature points, and these feature points can resist changes in light and differences in viewing angle, providing stable and reliable basic data for subsequent image matching.

[0100] Step 202: matching the feature points to establish the feature matching relationship between the images by using a nearest neighbor search and a geometric verification algorithm, wherein the nearest neighbor search and the geometric verification algorithm are constructed based on a KD-Tree based FLANN matching and a RANSAC fundamental matrix estimation.

[0101] In step 202, the nearest neighbor search is a process of quickly finding similar feature points by constructing a feature point index structure. The geometric verification is a method of verifying the correctness of the feature point matching through spatial geometric relations. The feature matching relationship is the corresponding connection relationship between the feature points in different images. The K-Dimensional Tree (KD-Tree) is a tree data structure used for efficient nearest neighbor search. The Fast Library for Approximate Nearest Neighbors (FLANN) matching is a feature point matching method based on fast nearest neighbor search. The Random Sample Consensus (RANSAC) fundamental matrix estimation is a method of estimating the basic geometric relationship between images through the random sample consensus algorithm.

[0102] In the embodiments of the present application, first, the KD-Tree structure is used to index the feature points of all images, the FLANN matching algorithm is used to quickly find the nearest neighbor corresponding relationship between the feature points of each image, then the RANSAC algorithm is used to geometrically verify the matching results, and the false matching point pairs that do not meet the fundamental matrix constraint are removed, and finally the accurate feature matching relationship between images is established.

[0103] Step 203: constructing an image connection graph based on the feature matching relationship, wherein the nodes in the image connection graph represent images, the edges represent the existence of the feature matching relationship between the images, and the weight of the edge is a comprehensive measurement determined based on the number of matching feature points between images, the re-projection error of the matching feature point pairs, and the common view geometric relationship.

[0104] In step 203, the feature point refers to a pixel position with visual features extracted from a single image, and the matching feature point pair refers to a pair of feature points that are correspondingly established between different images through a feature matching algorithm; the feature point is the basic unit of matching, and the matching feature point pair is the result of feature matching, and the spatial corresponding relationship between images can be established through multiple matching feature point pairs.

[0105] In the embodiments of the present application, each image is taken as a node of a graph, and the feature matching relationship that has passed the geometric verification is taken as an edge connecting the nodes. The weight value of each edge is calculated according to the number of matching points, the re-projection error accuracy and the size of the common view area, and an image connection graph that fully reflects the actual connection strength between images is constructed. The graph accurately describes the internal connection structure of the image data.

[0106] The following is a specific example:

[0107] Based on 5000 multi-angle image data obtained by performing oblique photogrammetry on a certain urban area A, feature points are first extracted from all images using a deep learning feature extraction algorithm, and a feature matching relationship between images is established by a fast approximate nearest neighbor search matching algorithm based on a multi-dimensional tree structure and a random sample consensus fundamental matrix estimation algorithm. In specific implementation, each image is processed using a pre-trained feature extraction network, which outputs the coordinate position of each feature point and feature description data with 256 dimensions. In the feature matching stage, a multi-dimensional tree index structure is first constructed to establish a spatial index for 1.5 million feature points of all images. The fast approximate nearest neighbor search algorithm is used to find potential matching pairs between the feature points, and an initial matching result is obtained. Then, the random sample consensus algorithm is used to geometrically verify the matching points of each pair of images. The fundamental matrix is calculated by randomly sampling 7 matching points, and the number of inliers that meet the geometric constraint is counted. After multiple iterations, the matching relationship with an inlier proportion of more than 60% is retained, and finally 180,000 reliable matching relationships that have passed the geometric verification are selected from 2.5 million initial matching pairs. Based on these verified matching relationships, an image connection graph is constructed, which includes 5000 nodes representing each image and edges between the nodes representing verified feature matching relationships. The weight value of each edge is obtained by comprehensively calculating three factors: the number of matching feature points, the re-projection error of the matching feature point pair, and the strength of the common view geometric relationship. The number of matching feature points is the actual number of matching pairs that have passed the verification, the re-projection error of the matching feature point pair is the average value of the projection position deviation of all matching points on two images, and the strength of the common view geometric relationship is quantified by the ratio of the common view area to the total area of the image. The finally constructed image connection graph accurately reflects the actual connection strength between the 5000 images, and provides a reliable basis for subsequent intelligent block processing.

[0108] In the embodiments of the present application, the stability and discrimination of the feature points are ensured by deep learning feature extraction, and accurate matching relationships are established by combining efficient nearest neighbor search and strict geometric verification. The finally constructed image connection graph can accurately reflect the actual connection strength between images, provide a reliable basis for subsequent intelligent block and distributed processing, and improve the accuracy and efficiency of oblique photogrammetry data processing.

[0109] To further improve the rationality of the block processing of oblique photography data and the subsequent fusion effect, in some embodiments, step 102: the weighted spectral clustering algorithm is used in combination with the multi-constraint spectral clustering algorithm to perform dynamic block processing on the image connection graph to form a plurality of subnets with internal connections and overlapping relationships, including:

[0110] Step 301: Based on the edge weights of the image connection graph, a weighted adjacency matrix and a degree matrix are constructed using the weighted spectral clustering algorithm, and based on the weighted adjacency matrix and the degree matrix, a Laplacian matrix is calculated.

[0111] In step 301, the weighted adjacency matrix is a square matrix that describes the connection strength between nodes in the image connection graph, and the element value corresponds to the weight of the edge in the graph. The degree matrix is a diagonal matrix, and each diagonal element value is the sum of the weights of all connected edges of the corresponding node. The Laplacian matrix is a feature matrix obtained by subtracting the weighted adjacency matrix from the degree matrix, which is used to represent the overall structural characteristics of the image connection graph.

[0112] In the embodiments of the present application, the weighted adjacency matrix is constructed according to the weight values of the edges in the image connection graph, wherein the matrix element value is directly taken from the connection weight between the corresponding image nodes; the sum of the connection weights of each node is calculated to form the degree matrix; the degree matrix is subtracted from the weighted adjacency matrix to obtain the Laplacian matrix, which contains the topological structure information of the image connection graph.

[0113] Step 302: Perform eigenvalue decomposition processing on the Laplacian matrix to obtain a plurality of eigenvalues.

[0114] In step 302, eigenvalue decomposition processing is a mathematical process of decomposing a matrix into eigenvalues and eigenvectors. The eigenvalue is a scalar value that represents the characteristics of the matrix, and the eigenvector is a directional vector corresponding to the eigenvalue.

[0115] In the embodiments of the present application, the Laplacian matrix is subjected to eigenvalue decomposition to solve all eigenvalues and corresponding eigenvectors of the matrix, and a sorted eigenvalue sequence and its associated eigenvector set are obtained, which reflect the different scale structural characteristics of the image connection graph.

[0116] Step 303: Select the first k smallest eigenvalues from the plurality of eigenvalues, and based on the eigenvectors corresponding to the first k smallest eigenvalues, construct a new feature space, where k is greater than or equal to 2.

[0117] In step 303, the feature space is a vector space spanned by the eigenvectors. The new feature space is a low-dimensional space composed of selected eigenvectors, which can preserve the main structural information of the original image connection graph.

[0118] In this embodiment, the first k smallest feature values ​​are selected from the feature value sequence, the feature vectors corresponding to these feature values ​​are extracted, and these feature vectors are used as basis vectors to construct a new k-dimensional feature space. Each image corresponds to a feature vector representation in this space.

[0119] Step 304: Using a multi-constraint spectral clustering algorithm, the image set is divided into k clusters that satisfy preset constraints in the new feature space through iterative optimization. Based on the k clusters, multiple subnets are constructed, wherein the constraints include subnet size balance constraints and minimum common point number constraints between subnets.

[0120] In step 304, the multi-constraint spectral clustering algorithm is an improved clustering method that considers multiple constraints simultaneously during the clustering process. Clusters are image groups obtained through cluster analysis. The subnet size balance constraint requires that the number of images contained in each cluster be relatively balanced. The minimum common point constraint between subnets requires that adjacent clusters maintain sufficient common points.

[0121] In this embodiment, cluster analysis is performed on the feature vector representations of all images in a new feature space. During the clustering process, iterative optimization simultaneously satisfies the constraints of subnet size balance and minimum common point number, dividing the image set into k clusters. Each cluster is converted into a subnet, ensuring that each subnet is tightly connected and that there is sufficient overlap between subnets. This step dynamically adjusts the clustering process through multiple constraints, achieving intelligent segmentation based on actual data features.

[0122] Here is a specific example:

[0123] An image connectivity graph was constructed based on 5000 multi-angle images obtained from oblique photogrammetry of a certain urban area A. This graph contains 5000 nodes and approximately 180,000 weighted edges. First, a weighted adjacency matrix W was constructed using a weighted spectral clustering algorithm. This matrix is ​​a 5000-row, 5000-column square matrix, where each element... Indicates the first The connection weights between image Z and image J are directly taken from the weights of the corresponding edges in the image connection graph, with values ​​ranging from [0,1]. Next, a degree matrix D is constructed. This matrix is ​​a 5000-order diagonal matrix, where each diagonal element... Equal to the weighted adjacency matrix of the first The sum of all elements in the row, i.e. , indicating the first The sum of the weights of all edges connecting each node. Then calculate the Laplacian matrix. Its calculation formula is ,in For degree matrix, This is a weighted adjacency matrix, which characterizes the overall structural properties of the image connectivity graph. (The last part, "for the Laplacian matrix," appears to be a separate, unrelated statement and is left untranslated.) Eigen decomposition is performed to obtain 5000 eigenvalues, and the eigenvalues are sorted in ascending order to obtain a sequence , wherein The first 50 smallest eigenvalues are selected from the eigenvalues , and the feature vectors corresponding to the eigenvalues are extracted to form a new 50-dimensional feature space. Each image corresponds to a 50-dimensional feature vector in the space. A multi-constraint spectral clustering algorithm is used to perform clustering analysis in the new feature space. The balance constraint of the subnetwork size is set to contain 80 to 120 images in each cluster. The range is determined by dividing the total number of images 5000 by the preset number of subnetworks 50 to obtain an average value 100, and a floating range of positive and negative 20 images is set, ; At the same time, the minimum common point quantity constraint between subnetworks is set to at least 20% of the connection points shared by adjacent subnetworks. The proportion is determined through experimental verification to ensure the subsequent fusion accuracy. Through iterative optimization, the image set is divided into 50 clusters, each cluster is converted into a subnetwork, and finally 50 subnetworks with internal connections and overlapping relationships are formed. The internal connections of each subnetwork are close and sufficient overlapping areas are reserved between subnetworks, providing an optimized basis for subsequent distributed processing.

[0124] In the embodiments of the present application, the actual connection relationship between images is accurately captured by weighted spectral clustering, and the balance and fusibility of subnetwork division are ensured by combining multiple constraint conditions. The formed subnetworks not only meet the parallel processing requirements but also guarantee the subsequent fusion accuracy, thereby improving the efficiency and quality of large-scale oblique photography data processing.

[0125] To further improve the accuracy and reliability of subnetwork fusion, in some embodiments, step 104: performing multi-level subnetwork fusion on all subnetworks to determine the target overlapping area, comprises:

[0126] Step 401: identifying the common coverage area between adjacent subnetworks based on the common images and common points of adjacent subnetworks.

[0127] In step 401, the common image refers to the image data that appears in both adjacent subnetworks. The common point refers to the same spatial point that exists in the three-dimensional reconstruction results of different subnetworks. The common coverage area is the geographical range that the adjacent subnetworks overlap in space.

[0128] In the embodiments of the present application, by comparing the image identifiers and three-dimensional point identifiers contained in adjacent subnetworks, the common images and common points belonging to both subnetworks are identified, and the common coverage area between adjacent subnetworks is determined according to the spatial distribution of these common elements.

[0129] Step 402: locating the initial overlapping area between adjacent subnetworks in the common coverage area by analyzing the three-dimensional spatial points and camera pose information contained in each subnetwork.

[0130] In step 402, the three-dimensional space points are three-dimensional coordinate data calculated by photogrammetry. The camera pose information includes the position and attitude parameters of the camera. The initial overlapping area is a preliminary determined inter-subnet overlapping range based on common elements.

[0131] In the embodiments of the present application, the coordinate distribution of the three-dimensional space points contained in each subnet and the spatial relationship of the camera pose are analyzed, and the initial overlapping area between adjacent subnets is located in the common coverage area by a spatial clustering method. The area contains the spatial range in which the connection relationship between the subnets may exist.

[0132] Step 403: In the initial overlapping area, the three-dimensional space points common to adjacent subnets and the corresponding camera observation information are extracted.

[0133] In step 403, the camera observation information refers to the two-dimensional coordinate data of the feature points recorded in the image by the camera when shooting, and the "camera pose information" refers to the position and attitude parameters of the camera in the three-dimensional space; the connection between the two is that the camera observation information is the basic data for calculating the camera pose information, and the camera pose information can be calculated from the camera observation information through multi-view geometric relationship.

[0134] In the embodiments of the present application, in the initial overlapping area, the three-dimensional space points common to adjacent subnets and the observation information thereof in different subnet images are extracted through coordinate matching and feature association analysis, including two-dimensional pixel coordinates and corresponding three-dimensional coordinates.

[0135] Step 404: Based on the common three-dimensional space points and the corresponding camera observation information, the coordinate transformation relationship between adjacent subnets is determined.

[0136] In step 404, the coordinate transformation relationship is a mathematical expression describing the conversion relationship between two different coordinate systems, including rotation, translation and scale parameters.

[0137] In the embodiments of the present application, based on the coordinate values of the extracted common three-dimensional space points in different subnets, the coordinate transformation parameters between adjacent subnets are calculated through the least square principle, and the transformation relationship from one subnet coordinate system to another subnet coordinate system is established.

[0138] Step 405: According to the coordinate transformation relationship and the camera pose information, the initial overlapping area is accurately defined to form a target overlapping area.

[0139] In step 405, the accurate definition is a process of determining the boundary of the overlapping area by optimization calculation.

[0140] In the embodiments of the present application, according to the calculated coordinate transformation relationship and camera pose information, the initial overlap region is optimized and adjusted through space projection and error analysis, the unreliable region is removed, the region with sufficient common points and stable geometric relationship is retained, and the accurate target overlap region is formed.

[0141] The following is a specific example:

[0142] Based on 5000 multi-angle images obtained by tilt photogrammetry of a certain urban area A, 50 subnets are formed after distributed processing, and adjacent subnet A containing 100 images and 50,000 three-dimensional points and subnet B containing 95 images and 48,000 three-dimensional points are selected for fusion processing. First, based on the image identification and three-dimensional point identification of the two subnets, 32 common images and 2150 common points are identified, and the common coverage area range is determined as 380 meters in the east-west direction and 420 meters in the north-south direction according to the spatial distribution of these common elements. By analyzing the coordinate distribution of all three-dimensional space points in the two subnets and the 215 camera pose information, the initial overlap region is located in the common coverage area using the density clustering method, which has an east-west direction span of 350 meters and a north-south direction span of 400 meters and contains 1850 common points. In the initial overlap region, 1800 three-dimensional space points common to adjacent subnets and their corresponding camera observation information totaling 21,600 observation records are extracted through coordinate matching, each observation record containing two-dimensional pixel coordinates of the feature points on the image and corresponding three-dimensional point coordinates. Based on these common three-dimensional space points, the coordinate transformation relationship between adjacent subnets is calculated, and the transformation parameters are solved using the least squares principle, wherein the coordinate transformation model is , wherein represents the three-dimensional coordinates of the point in the subnet B coordinate system, with units of meters, represents a 3x3 rotation matrix without dimension, represents the three-dimensional coordinates of the point in the subnet A coordinate system, with units of meters, represents a 3-dimensional translation vector with units of meters. By solving, the rotation matrix , the translation vector , with units of meters. According to the coordinate transformation relationship and camera pose information, the initial overlap region is accurately defined through space projection error analysis, the re-projection error of each common point is calculated, the unreliable region with an error greater than 2 pixels is removed, and finally the target overlap region is formed with an east-west direction span of 320 meters and a north-south direction span of 380 meters, containing 1560 stable common points and 28 common images, providing an accurate spatial range for the determination of the target overlap region.

[0143] In the embodiments of the present application, the effective overlapping area between subnets is accurately positioned through multi-level analysis, ensuring the accuracy and reliability of subsequent fusion processing, providing a solid foundation for global consistency optimization, and improving the overall quality of oblique photography three-dimensional reconstruction.

[0144] In order to further improve the efficiency and load balancing of distributed computing, in some embodiments, step 103: the image number, point cloud size and matching complexity of the subnet are calculated for load evaluation, based on the evaluation result, a work stealing algorithm is used to distribute each subnet to the corresponding distributed computing node, and a Levenberg-Marquardt algorithm is used to adjust the local bundle of the subnet on each distributed computing node to optimize the subnet parameters in parallel, including:

[0145] Step 501: The number of images contained in each subnet is counted, and the total number of three-dimensional space points in each subnet is calculated to obtain the point cloud size.

[0146] In step 501, the image number refers to the total number of images contained in each subnet. The total number of three-dimensional space points refers to the total number of three-dimensional points in each subnet obtained through feature matching and space calculation. The point cloud size is a subnet data size index quantified by the total number of three-dimensional space points.

[0147] In the embodiments of the present application, the number of images is counted by traversing the image list of each subnet, and the number of all three-dimensional space points in each subnet is calculated to obtain the point cloud size. These two data together reflect the basic data volume of the subnet.

[0148] Step 502: Analyze the complexity of feature matching relationship in each subnet to obtain the corresponding matching complexity.

[0149] In step 502, the matching complexity is a subnet processing difficulty index obtained by analyzing the number and distribution characteristics of feature matching.

[0150] In the embodiments of the present application, the feature matching relationship between all images in each subnet is analyzed, the number of matching point pairs, matching distribution uniformity and matching quality index are counted, and the matching complexity level of each subnet is evaluated comprehensively.

[0151] Step 503: Use a pre-constructed subnet computing complexity evaluation model to perform load evaluation on the image number, point cloud size and matching complexity to obtain an evaluation result.

[0152] In step 503, the subnet computing complexity evaluation model is a mathematical model trained through historical data for predicting the computing load of the subnet. The computing load evaluation is a quantitative result of the computing resources required for processing the subnet obtained by the model.

[0153] In the embodiment of the present application, the number of images, the size of point clouds and the matching complexity of each subnetwork are input into the pre-trained evaluation model, and the model outputs the calculation load evaluation result of each subnetwork through weighted calculation and comprehensive analysis, which reflects the relative calculation resources required to process the subnetwork.

[0154] Step 504: Based on the evaluation result, each subnetwork is initially allocated to the corresponding distributed computing node through dynamic load balancing scheduling based on the work-stealing algorithm, and each distributed computing node maintains a corresponding local task queue.

[0155] In step 504, dynamic load balancing scheduling is a mechanism for dynamically adjusting task allocation according to real-time load conditions. The local task queue is a list of subnetworks to be processed maintained by each computing node.

[0156] In the embodiment of the present application, based on the calculation load evaluation result, each subnetwork is initially allocated to a distributed computing node using the work-stealing algorithm, and each node maintains a local task queue. The subnetworks in the queue are sorted in descending order of calculation load to ensure that nodes with heavier loads process subnetworks with smaller calculation amounts first.

[0157] Step 505: When the distributed computing node completes the calculation of the subnetworks in the local task queue, a corresponding to-be-processed subnetwork is randomly obtained from the task queue of other distributed computing nodes with a load greater than the local load.

[0158] In step 505, the local load refers to the total calculation amount of the tasks to be processed by the computing node. The to-be-processed subnetwork is a subnetwork task in the task queue of other nodes that has not yet started calculation.

[0159] In the embodiment of the present application, when a certain computing node completes the calculation of all subnetworks in the local task queue, it will randomly select other nodes with heavier loads to obtain a to-be-processed subnetwork from the tail of their task queue for calculation, thereby achieving dynamic allocation of computing resources.

[0160] Step 506: On each distributed computing node, the Levenberg-Marquardt algorithm is used to adjust the local bundle of the allocated subnetworks, and the parameters of the adjusted subnetworks are optimized asynchronously and in parallel to obtain subnetworks with optimized parameters. The allocated subnetworks include the subnetworks in the local task queue and the to-be-processed subnetworks.

[0161] In step 506, the Levenberg-Marquardt algorithm is a numerical algorithm for nonlinear least squares optimization. Local bundle adjustment is the process of solving the camera pose and three-dimensional point coordinates in the subnetwork through optimization algorithms. The subnetwork with optimized parameters is a high-precision subnetwork result obtained after optimization calculation.

[0162] In the embodiments of the present application, on each computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the allocated subnetwork, and the optimal camera pose parameters and three-dimensional point coordinates are solved by iterative optimization. In the optimization process, each node calculates independently and returns the result through asynchronous communication mode, and finally the optimization parameters of all subnetworks are obtained.

[0163] The following is a specific example:

[0164] Based on 5000 multi-angle images obtained by oblique photogrammetry in a certain urban area A, 50 subnetworks are formed by dynamic block processing. First, the number of images contained in each subnetwork is counted to obtain the number of images in each subnetwork, which is between 80 and 120. Subnetwork A contains 100 images, and subnetwork B contains 95 images. The total number of three-dimensional points in each subnetwork is calculated to obtain the point cloud size, wherein subnetwork A contains 50,000 three-dimensional points, and subnetwork B contains 48,000 three-dimensional points. The complexity of the feature matching relationship in each subnetwork is analyzed to obtain the matching complexity, which is evaluated by calculating the number of matching point pairs, the matching distribution uniformity and the matching quality index. The matching complexity calculation formula is , wherein the matching complexity is dimensionless, the number of matching point pairs is the actual statistical value, the matching distribution uniformity is a coefficient between 0 and 1, the average re-projection error is a pixel unit value, and the matching complexity of subnetwork A is 85 and the matching complexity of subnetwork B is 78 according to actual calculation. The image number, point cloud size and matching complexity are calculated and loaded by using the pre-constructed subnetwork calculation complexity evaluation model, and the evaluation model formula is , wherein the calculation load evaluation result is dimensionless, the image number weight coefficient is 0.4, the image number is the actual statistical value, the point cloud size weight coefficient is 0.35, the point cloud size is the actual number of three-dimensional points in units of ten thousand, the matching complexity weight coefficient is 0.25, the matching complexity is dimensionless, and the calculation of subnetwork A data gives , and the calculation of subnetwork B gives Based on the evaluation results, 50 subnets are initially allocated to 10 distributed computing nodes through dynamic load balancing scheduling based on the work-stealing algorithm, each node maintains a local task queue of 5 subnets, and the computing load is sorted from large to small. When the distributed computing node 1 completes the calculation of the 5 subnets in the local task queue, it is detected that the current load of node 2 is 58, which is higher than the local load 0, and a to-be-processed subnet is randomly obtained from the tail of the task queue of node 2. The Levenberg-Marquardt algorithm is used on each distributed computing node to adjust the local bundle of the allocated subnets, including the subnets in the local task queue and the to-be-processed subnets. The camera pose parameters and three-dimensional point coordinates are solved through iterative optimization, and the optimization results are transmitted in an asynchronous parallel manner during the optimization process. Finally, the parameter optimization results of all 50 subnets are obtained, which provide accurate input data for subsequent multi-level subnet fusion.

[0165] In the embodiments of the present application, through accurate computing load evaluation and dynamic task allocation, the rational use of computing resources and load balancing are realized, and combined with efficient optimization algorithms, the fast convergence and precision improvement of subnet parameters are ensured, and the overall efficiency of large-scale oblique photography data processing is improved.

[0166] In order to further improve the accuracy and robustness of local bundle adjustment, in some embodiments, step 506: the Levenberg-Marquardt algorithm is used to adjust the local bundle of the allocated subnets, including:

[0167] Step 601: In the local bundle adjustment process, the sparse structure of the Hessian matrix is used, combined with a preset cost function, to perform dimension reduction processing on the parameters of the allocated subnets, wherein the Cauchy and robust kernel functions are introduced into the cost function to reduce the influence of mis-matched feature point pairs on the parameters of the allocated subnets.

[0168] In step 601, the Hessian matrix is the second derivative matrix of the cost function with respect to the optimization parameters, which represents the curvature information of the parameter change. The sparse structure refers to the distribution characteristics that most elements in the matrix are zero. The cost function is a function that measures the difference between the theoretical observation value and the actual observation value under the current parameters. The basic form of the cost function is the sum of the squares of all feature point re-projection errors. Dimension reduction processing is a process of reducing the size of parameter solving through mathematical transformation. The Cauchy robust kernel function is a mathematical function that suppresses the influence of outliers by assigning lower weights to larger residuals. Mis-matched feature points are feature point pairs that are incorrectly matched due to texture repetition or occlusion, etc.

[0169] In the embodiment of the present application, in the local bundle adjustment process, first, the sparse structure characteristics of the Hessian matrix are analyzed, and the special correlation mode between the camera pose parameters and the three-dimensional point coordinate parameters is identified. Based on this sparse structure, the matrix decomposition technique is used to reduce the dimension of the optimization problem, and the original problem is converted into a smaller linear system. At the same time, the Cauchy robust kernel function is introduced into the cost function, which can automatically identify and reduce the weight of the residual term corresponding to the mismatched feature points. Through the iteration optimization of the Levenberg-Marquardt algorithm, the influence of the mismatched points on the parameter estimation is effectively suppressed while maintaining the calculation efficiency, and finally the accurate and stable optimization result is obtained.

[0170] In the embodiment of the present application, by using the sparse structure of the Hessian matrix to improve the calculation efficiency, and combining the robust kernel function to effectively suppress the influence of mismatched points, the stability and reliability of the local bundle adjustment are improved while ensuring the optimization accuracy, which lays a solid foundation for subsequent global fusion.

[0171] Optionally, step 506: adjusting the parameters corresponding to the sub-network after optimization in an asynchronous parallel mode to obtain a sub-network after parameter optimization.

[0172] Step 701: optimizing the parameters corresponding to the adjusted sub-network in an asynchronous parallel mode, and transmitting the optimized parameters to the main distributed computing node, and periodically saving the optimization progress data to the storage system through the checkpoint mechanism to obtain a sub-network after parameter optimization, wherein when the distributed computing node fails, the local bundle adjustment process is recovered from the last successfully saved valid state based on the optimization progress data saved by the checkpoint mechanism.

[0173] In step 701, the asynchronous parallel mode refers to a parallel computing mode in which each computing node independently performs a computing task without waiting for each other. The optimized parameters include data such as camera pose and three-dimensional point coordinates obtained through local bundle adjustment. The main distributed computing node refers to a central coordination node responsible for task scheduling and result aggregation in the distributed computing architecture, which is established by initializing the distributed computing environment, and is used to receive the optimization results returned by each computing node and perform unified management. The storage system refers to a storage device or storage service used to persistently save the computing progress data, which is configured and established when the distributed computing environment is initialized, and is directly related to the checkpoint mechanism and the distributed computing node in the present application, and is used to ensure that the computation can be recovered from the saved optimization progress when the node fails. The checkpoint mechanism is a technology for fault recovery by periodically saving the computing progress. The optimization progress data includes intermediate results such as the current iteration number parameter value and the convergence state. The valid state refers to a usable computing intermediate state.

[0174] In the embodiments of the present application, each distributed computing node immediately transmits the optimized parameters to the master distributed computing node after completing the local bundle adjustment optimization of the subnetwork, without waiting for other nodes to complete the calculation. Meanwhile, each computing node saves the current optimization progress data into the storage system at a preset time interval or iteration number interval. When a certain distributed computing node fails, the system automatically detects the abnormality and reads the last successfully saved optimization progress data of the node from the storage system, restarts the local bundle adjustment process based on the data, and continues the calculation from the interruption until completion.

[0175] To further improve the accuracy and robustness of subnetwork fusion, in some embodiments, step 104: constructing a pose graph optimization model based on the common points and the camera pose information, and solving global consistency transformation parameters based on the pose graph optimization model using a Gauss-Newton method, includes:

[0176] Step 801: calculating a relative transformation matrix between adjacent subnetworks based on the common points.

[0177] In step 801, the relative transformation matrix is a four-by-four homogeneous transformation matrix describing the rotation and translation relationship between the coordinate systems of two subnetworks.

[0178] In the embodiments of the present application, based on the extracted common three-dimensional points between adjacent subnetworks, the relative transformation matrix between the coordinate systems of two subnetworks is calculated by least squares matching, which includes a rotation component and a translation component, and a preliminary spatial correspondence relationship between subnetworks is established.

[0179] Step 802: constructing a pose graph optimization model based on the relative transformation matrix and the camera pose information, wherein the nodes of the pose graph optimization model represent the camera poses of the subnetworks, and the edges represent the relative transformation matrices between adjacent subnetworks.

[0180] In the embodiments of the present application, the camera poses of each subnetwork are taken as graph nodes, and the relative transformation matrices calculated in step 801 are taken as edges connecting adjacent subnetwork nodes, to construct a pose graph optimization model expressing the spatial constraint relationship between all subnetworks.

[0181] Step 803: establishing an error function according to the nodes and edges of the pose graph optimization model, wherein the error function represents the difference between the camera poses of adjacent subnetworks and the relative transformation matrices.

[0182] In step 803, the error function is a target function quantifying the difference between the observed values and the estimated values in the pose graph model, representing the degree of inconsistency between the camera poses of adjacent subnetworks and the relative transformation matrices, wherein represents the error function of the pose graph optimization model, denotes a set of camera poses of all subnets, denotes the node indices representing different subnets in the pose graph, denotes a robust kernel function, denotes the relative transformation matrix between and , denotes the camera pose matrix of subnet , denotes the inverse of the camera pose matrix of subnet .

[0183] In the embodiments of the present application, an error function based on relative transformation residuals is established according to the node and edge definitions of the pose graph model, which measures the difference between the actual relative transformation between adjacent subnets and the relative transformation calculated by the camera poses.

[0184] Step 804: Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct an optimization problem.

[0185] In step 804, the robust kernel function is a mathematical function that suppresses the influence of outliers by assigning lower weights to larger residuals. The nonlinear optimization method is a numerical algorithm that solves nonlinear minimization problems by iteration. The error function is used to quantify the difference between the observed values and the estimated values in the pose graph optimization model, while the robust kernel function is a special mathematical function embedded in the error function, which is used to weight the residual terms in the error function. By reducing the weight of the residual term corresponding to the mismatched points, the influence of outliers on the overall optimization result is suppressed, and the two together constitute a complete optimization objective function.

[0186] In the embodiments of the present application, the robust kernel function is introduced into the error function to construct a nonlinear optimization problem with robustness, which aims to minimize the weighted residual sum of all relative transformation edges, and automatically reduces the influence of mismatched common points on the optimization result.

[0187] Step 805: The optimization problem is iteratively calculated by the Gauss-Newton method to obtain the global consistency transformation parameters between subnets.

[0188] In step 805, the Gauss-Newton method is an iterative optimization algorithm for solving nonlinear least squares problems.

[0189] In the embodiments of the present application, the Gauss-Newton method is used to iteratively solve the constructed optimization problem, and the transformation parameters of all subnets are gradually optimized by linear approximation and repeated iteration, and finally the global consistency transformation parameters that minimize the global error are obtained.

[0190] The following is a specific example:

[0191] Based on 5000 multi-angle images obtained by tilt photogrammetry of a certain urban area A, 50 subnets are formed after distributed processing, among which adjacent subnet A and subnet B are selected for fusion. First, the relative transformation matrix between the two subnets is calculated based on 1560 common points, and the relative transformation matrix T_AB is obtained by least squares fitting. The matrix is a 4x4 homogeneous transformation matrix containing rotation component R_AB and translation component t_AB, wherein the rotation matrix R_AB is a 3x3 orthogonal matrix [[0.9998, -0.0052, 0.0183], [0.0053, 0.9999, -0.0105], [-0.0182, 0.0106, 0.9998]], and the translation vector t_AB is [2.15, -1.08, 0.43] unit meters. Based on the relative transformation matrix and the camera pose information of subnet A and subnet B, a pose graph optimization model is constructed, which includes 50 nodes representing the camera poses of 50 subnets and 49 edges representing the relative transformation matrix between adjacent subnets. According to the nodes and edges of the pose graph optimization model, an error function is established . Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct an optimization solving problem, where the scale parameter of the robust kernel function is set to 0.1. The optimization solving problem is iteratively calculated by the Gauss-Newton method, with the maximum number of iterations set to 20 and the convergence threshold set to 1e-6. After 18 iterations, the global consistency transformation parameters between all subnets are solved, including the pose transformation matrix of subnet A , the global transformation matrix of subnet B , and the root mean square of the relative transformation residual between all adjacent subnets is reduced from the initial 0.15 meters to 0.02 meters, achieving high-precision global consistency alignment of 50 subnets.

[0192] In the embodiments of the present application, the spatial constraint relationship between all subnets is effectively integrated through the pose graph optimization model, the influence of mis-matching is suppressed by combining the robust optimization method, and the high-precision global consistency transformation parameters are solved by using the efficient Gauss-Newton method, which lays a solid foundation for constructing seamless global three-dimensional models.

[0193] To further improve the absolute accuracy and geographic reference accuracy of the global model, in some embodiments, step 105: in the presence of ground control points, the coordinates corresponding to the ground control points are taken as fixed constraints, and the global consistency transformation parameters are optimized as a whole by using a weighted least squares adjustment algorithm, including:

[0194] Step 901: based on the global consistency transformation parameters, an adjustment optimization framework is established.

[0195] In step 901, the adjustment optimization framework is a whole calculation structure for uniformly processing various observation data and parameters by mathematical methods.

[0196] In the embodiment of the present application, based on the obtained global consistency transformation parameters, a unified adjustment optimization framework containing all subnetwork transformation parameters, camera poses and three-dimensional point coordinates is constructed, which can simultaneously process various observation data and optimize all parameters.

[0197] Step 902: The coordinates corresponding to the ground control points are fused into the adjustment optimization framework as fixed constraints, and in the adjustment optimization framework, a sliding window optimization method is used to adjust the parameters of the surrounding area of the ground control points.

[0198] In step 902, the sliding window optimization is an optimization strategy for gradually improving the overall accuracy by local adjustment. The surrounding area of the ground control point refers to the spatial influence range centered on the control point. The shape and size of the area are dynamically determined by the sliding window, which depends on the geometric relationship between the control point and the surrounding camera stations and three-dimensional points, and is not a fixed circle or rectangle, nor a preset radius or length value, but an adaptively adjusted range according to the actual network structure and accuracy requirements.

[0199] In the embodiment of the present application, the accurate coordinates of the ground control points are fused into the adjustment framework as fixed constraints, and a sliding window optimization method is used to locally optimize and adjust the camera poses and three-dimensional point parameters in the surrounding area of each control point, gradually improving the model accuracy in the surrounding area of the control point.

[0200] Step 903: Based on the adjusted parameters of the surrounding area, combined with the variance component estimation method, the weight ratio of each type of observation value is adaptively optimized.

[0201] In step 903, the variance component estimation is a method for determining the relative accuracy of different observation data through statistical analysis. The weight ratio is the relative weight proportion of different observation data in adjustment. The adaptive optimization is a process of automatically adjusting the weight configuration according to the data quality.

[0202] In the embodiment of the present application, based on the residual statistics after adjusting the parameters of the surrounding area of the control points, the accuracy level of different data sources such as image observation values and control point observation values is analyzed by using the variance component estimation method, and the optimal weight ratio configuration of each type of observation value in adjustment is adaptively optimized.

[0203] Step 904: Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized by using the weighted least squares adjustment algorithm to obtain the optimization result.

[0204] In step 904, the optimization result is the optimal solution of all parameters after overall adjustment.

[0205] In the embodiments of the present application, based on the optimized weight ratio configuration, the global consistency transformation parameters and the parameters of the control point surrounding area are optimized and solved as a whole by using the weighted least squares adjustment algorithm, and the optimal parameter combination that minimizes the weighted residual sum of squares of all observations is obtained through iterative calculation. The optimization of the global consistency transformation parameters and the parameters of the adjusted surrounding area means that the two types of parameters are jointly optimized by using the weighted least squares adjustment to obtain a unified optimization result, which contains the final solution of the global consistency transformation parameters and the control point surrounding area parameters after overall coordination.

[0206] The following is a specific example:

[0207] Based on 5000 multi-angle images obtained by performing oblique photogrammetry on a certain urban area A, the global consistency transformation parameters of 50 subnets are obtained through the foregoing processing. First, an adjustment optimization framework is established, which includes the transformation parameters of 50 subnets, 5000 camera pose parameters and 2 million three-dimensional point coordinate parameters. Ten ground control points are arranged in area A, and the known coordinates of these control points are taken as fixed constraints and integrated into the adjustment optimization framework. The control point coordinate accuracy is 0.01 meters. In the adjustment optimization framework, a sliding window optimization method is used to adjust the parameters within 50 meters around each ground control point, and the range is determined according to half of the average distance between control points, i.e. 100 meters. The window contains about 20 camera poses and 8000 three-dimensional points. Based on the adjusted surrounding area parameters, the weight ratio of each type of observation is adaptively optimized by combining the variance component estimation method, and the variance component estimation formula is , wherein represents the variance estimate unit square meter of the type observation, represents the residual vector of the type observation, represents the weight matrix of the type observation, represents the number of redundant observations of the type observation, and The variance estimate of the image observation is 0.0004 square meters, and the variance estimate of the control point observation is 0.0001 square meters, and then the weight ratio of the two types of observations is determined as 1 to 4. Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted surrounding area parameters are optimized as a whole by using the weighted least squares adjustment algorithm, and the adjustment model is , wherein represents the residual vector, represents the design matrix, denotes the observation vector, and is minimized by where P is the weight matrix, and the adjustment converges after 5 iterations to obtain the optimized result, and the control point residual error is calculated by the formula , and the calculation result is 0.02 meters, which is improved from 0.05 meters before adjustment, and the global consistency transformation parameters of all subnets are further optimized to realize high-precision geographic referencing of the regional A oblique photogrammetry three-dimensional model.

[0208] In the embodiments of the present application, the absolute accuracy is ensured by the control point fixed constraint, combined with the sliding window optimization and adaptive weight ratio determination, the high-precision overall optimization of the global model is realized, and the geographic referencing accuracy and spatial consistency of the oblique photogrammetry three-dimensional result are improved.

[0209] Figure 3 The structure schematic diagram of the multi-level fusion optimization system for oblique photogrammetry space three processing provided in the embodiments of the present application, and the specific implementation part describes:

[0210] The acquisition module 31 is configured to acquire multi-view image data by using an oblique photogrammetry device.

[0211] The construction module 32 is configured to construct an image connection graph based on a feature matching relationship between images in the multi-view image data, and perform dynamic block processing on the image connection graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm, to form a plurality of subnets having internal connection and overlapping relationship.

[0212] The evaluation module 33 is configured to calculate load evaluation on the number of images, the size of point cloud and the matching complexity of the subnets, and based on the evaluation result, each subnet is distributed to a corresponding distributed computing node by using a work-stealing algorithm, and the Levenberg-Marquardt algorithm is used on each distributed computing node to perform local bundle adjustment processing on the subnet, to optimize the subnet parameters in parallel.

[0213] The fusion module 34 is configured to perform multi-level subnet fusion on all subnets to determine a target overlapping area, extract common points and camera pose information from the target overlapping area corresponding to the parameter-optimized subnet, and construct a pose graph optimization model based on the common points and the camera pose information, and solve the global consistency transformation parameters by using the Gauss-Newton method based on the pose graph optimization model.

[0214] The optimization module 35 is configured to, in the presence of ground control points, take the coordinates corresponding to the ground control points as fixed constraints, and use the weighted least squares adjustment algorithm to perform overall optimization on the global consistency transformation parameters.

[0215] The adjusting module 36 is configured to construct a global model based on the optimization result, and perform lightweight bundle adjustment on the global model to generate the aerial triangulation result.

[0216] The multi-level fusion optimization system for oblique photography aerial triangulation of the embodiments of the present application is used to implement the multi-level fusion optimization method for oblique photography aerial triangulation as described above, and therefore the specific embodiments of the multi-level fusion optimization system for oblique photography aerial triangulation can be seen from the embodiment part of the multi-level fusion optimization method for oblique photography aerial triangulation as described above, and the specific embodiments can be referred to the description of the corresponding embodiment part, which will not be described here again.

[0217] Those skilled in the art will further appreciate that the individual elements of the examples described in association with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow chart illustrations instead of in terms of their structural implementation. As noted above, a skilled artisan will recognize the functionality described herein can be implemented in hardware, software or a combination of both. The described functionality can be implemented by one or more computer programs executed by one or more programmable computers or processors (e.g., a shared processor, a dedicated processor, or a processor shared by other elements) under the control of one or more computer programs. The computer program can be written in any form of languages, including compiled or interpreted languages, and each said program can be stored on any suitable data carrier, such as a ROM, for example, a DVD-ROM, a CD-ROM, or a floppy disk, or used in accordance with a method of on-line distribution, for example, by using the Internet or an intranet. The computer programs can also be executed by more than one processor, or implemented in parallel on a plurality of processors. The computer programs can also be written as one or more computer programs that, when executed, carry out one or more computer programs functions in response to instructions from one or more computer programs.

[0218] The multi-level fusion optimization method and system for oblique photography aerial triangulation provided by the present application are described in detail above. The principles and implementation manners of the present application are described by using specific examples in this paper, and the above embodiment description is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the present application.

Claims

1. A multi-level fusion optimization method for oblique aerial triangulation processing, characterized in that, The method comprises the following steps: acquiring multi-view image data by using a oblique photogrammetry device; constructing an image connection graph based on the feature matching relationship between images in the multi-view image data, and performing dynamic block processing on the image connection graph by using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form a plurality of subnets with internal connection and overlapping relationship; calculating the load of the number of images, the size of point cloud and the matching complexity of the subnet, and based on the evaluation result, distributing each subnet to the corresponding distributed computing node by using a work-stealing algorithm, and performing local bundle adjustment processing on the subnet by using the Levenberg-Marquardt algorithm on each distributed computing node to optimize the subnet parameters in parallel; performing multi-level subnet fusion on all subnets to determine the target overlapping area, extracting common points and camera pose information from the target overlapping area corresponding to the subnet after parameter optimization, and constructing a pose graph optimization model based on the common points and the camera pose information, and solving the global consistency transformation parameters by using the Gauss-Newton method based on the pose graph optimization model; in the case of existing ground control points, using the coordinates of the ground control points as fixed constraints to perform overall optimization on the global consistency transformation parameters by using the weighted least squares adjustment algorithm; based on the optimization result, constructing a global model, and performing lightweight bundle adjustment processing on the global model to generate an aerial triangulation result; the multi-level subnet fusion on all subnets to determine the target overlapping area comprises: identifying the common coverage area between adjacent subnets based on the common images and common points of adjacent subnets; locating the initial overlapping area between adjacent subnets in the common coverage area by analyzing the three-dimensional space points and camera pose information contained in each subnet; extracting three-dimensional space points and corresponding camera observation information common to adjacent subnets within the initial overlapping area; determining the coordinate transformation relationship between adjacent subnets based on the common three-dimensional space points and corresponding camera observation information; accurately defining the initial overlapping area to form a target overlapping area according to the coordinate transformation relationship and the camera pose information.

2. The method of claim 1, wherein, the construction of the image connection graph based on the feature matching relationship between images in the multi-view image data comprises: extracting feature points representing image content from the multi-view image data based on a deep learning-based feature extraction algorithm; matching the feature points by using a nearest neighbor search and geometric verification algorithm to establish the feature matching relationship between images, wherein the nearest neighbor search and geometric verification algorithm is constructed based on the KD-Tree FLANN matching and RANSAC fundamental matrix estimation; constructing an image connection graph based on the feature matching relationship, wherein the nodes in the image connection graph represent images, the edges represent the feature matching relationship between images, and the weight of the edge is a comprehensive measure determined based on the number of matching feature points, the re-projection error of matching feature point pairs and the common view geometric relationship.

3. The method of claim 1, wherein, The weighted spectral clustering algorithm is combined with the multi-constraint spectral clustering algorithm to perform dynamic block processing on the image connection graph to form a plurality of subnets having internal connections and overlapping relationships, including: Based on the edge weights of the image connection graph, a weighted adjacency matrix and a degree matrix are constructed using a weighted spectral clustering algorithm, and based on the weighted adjacency matrix and the degree matrix, a Laplacian matrix is calculated; The Laplacian matrix is subjected to eigenvalue decomposition processing to obtain a plurality of eigenvalues; The first k smallest eigenvalues are selected from the plurality of eigenvalues, and based on the eigenvectors corresponding to the first k smallest eigenvalues, a new feature space is formed, wherein k is greater than or equal to 2; Using a multi-constraint spectral clustering algorithm, the image set is divided into k clusters that satisfy the preset constraint conditions in the new feature space through iterative optimization, and based on the k clusters, a plurality of subnets are constructed, wherein the constraint conditions include subnet size balance constraints and minimum common point quantity constraints between subnets.

4. The method of claim 1, wherein, The image quantity, point cloud size, and matching complexity of the subnet are calculated and load evaluated, and based on the evaluation results, a work-stealing algorithm is used to distribute each subnet to a corresponding distributed computing node, and the Levenberg-Marquardt algorithm is used to perform local bundle adjustment processing on the subnet on each distributed computing node to optimize the subnet parameters in parallel, including: The number of images contained in each subnet is counted, and the total number of three-dimensional space points in each subnet is calculated to obtain the point cloud size; The complexity of the feature matching relationship in each subnet is analyzed to obtain the corresponding matching complexity; The image quantity, the point cloud size, and the matching complexity are calculated and load evaluated using a pre-constructed subnet computing complexity evaluation model to obtain evaluation results; Based on the evaluation results, a dynamic load balancing scheduling based on a work-stealing algorithm is used to initially distribute each subnet to a corresponding distributed computing node, wherein each distributed computing node maintains a corresponding local task queue; When a distributed computing node completes the subnet calculation in the local task queue, it randomly acquires a corresponding to-be-processed subnet from the task queue of another distributed computing node with a load greater than the local load; On each distributed computing node, the Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the allocated subnet to asynchronously and in parallel optimize the parameters of the adjusted subnet to obtain the subnet after parameter optimization, and the allocated subnet includes the subnet in the local task queue and the to-be-processed subnet.

5. The method of claim 4, wherein, The Levenberg-Marquardt algorithm is used to perform local bundle adjustment on the allocated subnet, including: In the local bundle adjustment process, the sparse structure of the Hessian matrix is used in combination with a preset cost function to perform dimensionality reduction processing on the parameters of the allocated subnet, wherein the Cauchy and robust kernel functions are introduced into the cost function to reduce the influence of mis-matched feature points on the parameters of the allocated subnet.

6. The method of claim 4, wherein, The asynchronous parallel optimization of the parameters of the adjusted subnet to obtain the subnet after parameter optimization includes: The parameters corresponding to the adjusted subnets are optimized in an asynchronous parallel mode, and the optimized parameters are transmitted to the main distributed computing node; meanwhile, the optimization progress data is saved to the storage system at regular intervals through the checkpoint mechanism, so as to obtain the subnets after parameter optimization; when the distributed computing node fails, the optimization progress data saved based on the checkpoint mechanism is used to restore the local bundle adjustment process from the last successfully saved valid state.

7. The method of claim 1, wherein, The pose graph optimization model is constructed based on the common points and the camera pose information, and the global consistency transformation parameters are obtained by solving the pose graph optimization model using the Gauss-Newton method, including: Based on the common points, the relative transformation matrix between adjacent subnets is calculated; Based on the relative transformation matrix and the camera pose information, a pose graph optimization model is constructed, wherein the nodes of the pose graph optimization model represent the camera poses of the subnets, and the edges represent the relative transformation matrix between adjacent subnets; According to the nodes and edges of the pose graph optimization model, an error function is established, which represents the difference between the camera poses of adjacent subnets and the relative transformation matrix; Based on the error function, a nonlinear optimization method with a robust kernel function is used to construct an optimization solving problem; The global consistency transformation parameters between subnets are obtained by iteratively calculating the optimization solving problem using the Gauss-Newton method.

8. The method of claim 1, wherein, In the presence of ground control points, the coordinates corresponding to the ground control points are used as fixed constraints, and the global consistency transformation parameters are optimized as a whole using the weighted least squares adjustment algorithm, including: Based on the global consistency transformation parameters, an adjustment optimization framework is established; The coordinates corresponding to the ground control points are used as fixed constraints and integrated into the adjustment optimization framework, and in the adjustment optimization framework, the parameters of the surrounding area of the ground control points are adjusted using the sliding window optimization method; Based on the adjusted parameters of the surrounding area, the weight ratio of each type of observation value is adaptively optimized by combining the variance component estimation method; Based on the optimized weight ratio, the global consistency transformation parameters and the adjusted parameters of the surrounding area are optimized using the weighted least squares adjustment algorithm to obtain the optimization result.

9. A multi-level fusion optimization system for oblique aerial triangulation processing, characterized by, The method comprises the following steps: An acquisition module is configured to acquire multi-view image data using a oblique photogrammetry device; A construction module is configured to construct an image connection graph based on the feature matching relationship between images in the multi-view image data, and perform dynamic block processing on the image connection graph using a weighted spectral clustering algorithm combined with a multi-constraint spectral clustering algorithm to form a plurality of subnets with internal connection and overlap relationship; An evaluation module is configured to calculate the load evaluation of the number of images, the size of point cloud and the matching complexity of the subnets, and based on the evaluation result, each subnet is allocated to a corresponding distributed computing node using a work-stealing algorithm, and the local bundle adjustment processing of the subnet is performed on each distributed computing node using the Levenberg-Marquardt algorithm to optimize the parameters of the subnet in parallel. The fusion module is configured to perform multi-level subnet fusion on all subnets to determine a target overlapping area, extract common points and camera pose information from the target overlapping area corresponding to the parameter-optimized subnets, construct a pose graph optimization model based on the common points and the camera pose information, and obtain global consistency transformation parameters by using a Gauss-Newton method based on the pose graph optimization model; The optimization module is configured to, in the presence of ground control points, take coordinates corresponding to the ground control points as fixed constraints, and perform overall optimization on the global consistency transformation parameters by using a weighted least squares adjustment algorithm; The adjustment module is configured to, based on an optimization result, construct a global model, perform lightweight beam adjustment processing on the global model, and generate an aerial triangulation processing result. The multi-level subnet fusion on all subnets to determine a target overlapping area includes: identifying a common coverage area between adjacent subnets based on common images and common points of the adjacent subnets; locating an initial overlapping area between the adjacent subnets in the common coverage area by analyzing three-dimensional space points and camera pose information contained in each subnet; extracting three-dimensional space points and corresponding camera observation information common to the adjacent subnets within the initial overlapping area; determining a coordinate transformation relationship between the adjacent subnets based on the common three-dimensional space points and the corresponding camera observation information; accurately defining the initial overlapping area according to the coordinate transformation relationship and the camera pose information to form a target overlapping area.

Images