Construction method of wide-area image control feature library based on multi-period images and real scene three-dimensional model

Abstract

The application discloses a wide-area image control feature library construction method based on multi-period images and real scene three-dimensional models, and comprises the following steps: automatically extracting feature points from preprocessed multi-period images and performing cross-image matching to obtain a same-name point pair set across periods; performing depth constraint projection on the same-name point pair across periods to obtain initial object side point coordinates; performing filtering and screening to obtain screened historical three-dimensional models; performing multi-view forward intersection triangulation and least square optimization based on exterior orientation elements, the same-name points across periods, the screened initial object side point coordinates and the screened historical three-dimensional models to obtain three-dimensional coordinates of each object side point and to construct a multi-view correlation mapping relationship between the object side points and multi-period image feature points; and constructing a feature library based on the three-dimensional coordinates of the object side points and the multi-view correlation mapping relationship between the object side points and the multi-period image feature points and dynamically updating the feature library. The application overcomes the problems of insufficient image control point layout, difficult reuse across periods, low efficiency and unstable precision of manual extraction in the prior art.

Description

Technical Field

[0001] This invention relates to the field of photogrammetry, and in particular to a method, system, storage medium, and electronic device for constructing a wide-area image control feature library based on multi-period images and real-world 3D models. Background Technology

[0002] With the continuous advancement of "3D reality" construction, the demand for wide-area, multi-phase, and high-precision 3D geographic information data is constantly increasing. As the fundamental control data for aerial triangulation and 3D model construction, the layout and quality of control points directly affect the accuracy and reliability of 3D reality results. However, existing control point management and application methods have significant shortcomings: (1) Limited accuracy of aerial triangulation: Under large-scale and complex terrain conditions, it is difficult to fully deploy control points, which leads to a decrease in the accuracy of aerial triangulation and affects the generation of high-precision real-scene 3D models and geometric consistency.

[0003] (2) Feature data is isolated and difficult to reuse across periods: Traditional methods rely on ground control points and single-period images to construct image control points. They lack unified management of historical images and multi-period data, which makes it difficult to repeatedly extract and update feature points and cannot efficiently achieve cross-period reuse.

[0004] (3) Low efficiency and insufficient accuracy of general image control point extraction: Existing feature library construction relies heavily on manual extraction, making it difficult to achieve automated extraction and standardized storage of multi-source data such as point cloud, Mesh, DOM, and DEM.

[0005] (4) Difficulty in updating multiple phases of images: As new images are continuously acquired, there is a lack of effective methods to integrate the features of new images with the historical feature library, making it difficult to achieve long-term maintenance, cross-phase updates and accuracy control of the image control feature library, which limits the efficiency and sustainability of wide-area real scene 3D modeling. Summary of the Invention

[0006] This invention provides a method, system, storage medium, and electronic device for constructing a wide-area camera control feature library based on multi-period images and real-world 3D models. It can solve the problems of insufficient layout, difficulty in cross-period reuse, low efficiency of manual extraction, and unstable accuracy of traditional camera control points in wide-area, multi-period, and high-precision real-world 3D modeling.

[0007] This invention provides a method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, including: Acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and perform preprocessing; Automated feature point extraction and cross-image matching are performed on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period identical point pairs. Based on historical aerial triangulation information and historical 3D models, depth-constrained projection is performed on pairs of corresponding points across different periods to obtain initial object point coordinates. The initial object point coordinates are then filtered to obtain filtered initial object point coordinates. The points on the preprocessed historical 3D model are filtered to obtain the filtered historical 3D model. Based on the exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates, and the selected historical 3D model, multi-view forward intersection triangulation and least squares optimization are performed to obtain the 3D coordinates of each object point, and a multi-view association mapping relationship between the object point and the feature points of multiple periods of images is constructed. A feature library is constructed based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple image periods, and the feature library is dynamically updated.

[0008] Furthermore, based on the aforementioned method for constructing a wide-area image control feature library using multi-period images and a real-world 3D model, automated feature point extraction and cross-image matching are performed on the preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period corresponding point pairs, including: A feature extraction algorithm is used to segment the preprocessed multi-phase oblique photogrammetric images into multiple image blocks. The multiple image blocks are then input into a feature extraction network to obtain key point locations and feature descriptors. A global operation is performed on the preprocessed multi-stage oblique photographic images to generate an edge mask. Based on the edge mask, the extracted feature points are filtered, retaining only feature points whose pixel coordinates are located in the effective edge area and removing feature points with poor discernibility in areas with uniform texture. Based on the feature descriptor, the filtered feature points are matched to obtain multiple pairs of cross-period same-name point pairs, forming an initial set of cross-period same-name point pairs; The initial set of intertemporal identical point pairs is verified to obtain the final set of intertemporal identical point pairs.

[0009] Furthermore, according to the above-mentioned method for constructing a wide-area image control feature library based on multi-period images and a real-world 3D model, the historical aerial triangulation information includes exterior orientation elements. Based on the historical aerial triangulation information and the historical 3D model, depth-constrained projection is performed on pairs of corresponding points across different periods to obtain initial object point coordinates. The initial object point coordinates are then filtered to obtain filtered initial object point coordinates, including: Based on the exterior orientation elements and the historical 3D model, the cross-period corresponding point pairs are mapped to the 3D space to obtain the initial object point coordinates; Calculate the Euclidean distance between the initial object point coordinates and the nearest point in the historical 3D model. If the Euclidean distance is less than or equal to a preset depth constraint threshold, the initial object point is determined to have a reasonable spatial position and is retained. If the Euclidean distance is greater than the preset depth constraint threshold, the initial object point is determined to have a projection deviation and is discarded. Using statistical methods 3 The outlier detection algorithm removes local outliers, resulting in the initial object coordinates after filtering.

[0010] Furthermore, according to the above-mentioned method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, based on the exterior orientation elements and the historical 3D model, cross-period corresponding point pairs are mapped to 3D space to obtain the initial object space point coordinates, expressed by the following formula:

[0011] in, For the preliminary three-dimensional object space coordinates, For exterior orientation line elements, R is the depth coefficient, and R is the image exterior orientation rotation matrix. The inverse of the image intrinsic parameter matrix. (u,v) represents the homogeneous pixel coordinates of the corresponding point, and (u,v) represents the original pixel position of the feature point on the image.

[0012] Furthermore, according to the above-mentioned method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, the points on the preprocessed historical 3D model are filtered to obtain the filtered historical 3D model, including: Gaussian filtering is applied to each point in the preprocessed historical 3D model. The evaluation indicators are determined based on the historical 3D model accuracy report. Points in the historical 3D model that do not meet the evaluation indicators are removed to obtain the filtered historical 3D model.

[0013] Furthermore, according to the above-mentioned method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, evaluation indicators are determined based on historical 3D model accuracy reports. Points in the historical 3D models that do not meet the evaluation indicators are removed to obtain the filtered historical 3D models, including: The mean plane error and mean elevation error of each point are calculated based on the mean of the model points in the historical 3D model. When both the mean plane error and the mean elevation error are greater than the corresponding error threshold, the corresponding point is removed. Calculate the density of triangular patches. When the density of triangular patches is greater than a preset density threshold, remove points in the corresponding region. Calculate the texture clarity. When the texture clarity is less than a preset texture clarity threshold, remove points in the corresponding region. Calculate the point cloud dispersion. When the point cloud dispersion is greater than a preset point cloud dispersion threshold, remove points in the corresponding region. Finally, obtain the filtered historical 3D model.

[0014] Furthermore, based on the aforementioned method for constructing a wide-area image control feature library using multi-period images and real-world 3D models, multi-view forward intersection triangulation is performed using exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates, and the selected historical 3D models, expressed by the following formula:

[0015] Subscript To correspond to images from different perspectives and different periods, ( () represents the coordinates of the same feature point in the camera coordinate system of the l-th image; The objective function for least squares optimization is:

[0016] Where E is the total reprojection error, and m is the total number of image viewpoints / periods involved in the calculation. Let be the coordinates of the actual pixel with the same name on the l-th image. The projected pixel coordinates are calculated from the current object point coordinates. The iterative convergence condition is

[0017] in, This represents the coordinate correction amount for a single iteration of the 3D object point. This is the threshold for iterative convergence.

[0018] This invention also provides a wide-area image control feature library construction system based on multi-period images and real-world 3D models, comprising: The acquisition and preprocessing module is used to acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and to perform preprocessing. The image feature extraction and matching module is used to automatically extract feature points and perform cross-image matching on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period same-name point pairs. The same-point depth constraint projection and filtering module is used to perform depth constraint projection on cross-period same-point pairs based on historical aerial triangulation information and historical 3D models to obtain initial object point coordinates, and to filter the initial object point coordinates to obtain filtered initial object point coordinates. The historical 3D model point filtering module is used to filter and select points on the preprocessed historical 3D model to obtain the filtered historical 3D model. The 3D object point solution and multi-view association module is used to perform multi-view forward intersection triangulation and least squares optimization based on exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates and the selected historical 3D model to obtain the 3D coordinates of each object point and construct the multi-view association mapping relationship between the object point and the feature points of multiple periods of images. The feature library construction and update module is used to construct a feature library based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple phases of images, and to dynamically update the feature library.

[0019] The present invention also provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded by a processor to execute any of the above-described methods for constructing a wide-area image control feature library based on multi-period images and real-world 3D models.

[0020] The present invention also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in the steps of the wide-area image control feature library construction method based on multi-period images and real-scene 3D models described above.

[0021] The present invention provides a method, system, storage medium, and electronic device for constructing a wide-area image control feature library based on multi-period images and real-world 3D models. The present invention has the following beneficial effects: (1) By automatically extracting image features, matching across periods and solving three-dimensional object points, manual intervention is significantly reduced, the utilization efficiency of multi-period images and historical models is improved, and the process of large-scale real-scene three-dimensional modeling is made faster and more stable.

[0022] (2) The constructed multi-phase image control feature library provides dense and reliable control point information, which can achieve high-precision aerial triangulation calculation even under conditions of few or no control points, thereby improving the flexibility and applicability of aerial triangulation calculation; (3) Through multi-view optimization, depth constraint and screening mechanism, the generated three-dimensional object points and feature library have high reliability in spatial accuracy and spatiotemporal consistency, providing a guarantee for refined three-dimensional modeling; (4) The feature library supports version management and continuous updates, which facilitates long-term data maintenance and expansion. It is suitable for wide-area, multi-phase, high-precision real-scene 3D modeling and related geographic information application scenarios. Attached Figure Description

[0023] The technical solution and other beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0024] Figure 1 The flowchart illustrates a method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, as provided in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the structure of a wide-area image control feature library construction system based on multi-period images and real-world 3D models provided in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention provides a method, system, storage medium, and electronic device for constructing a wide-area image control feature library based on multi-period imagery and a real-world 3D model. The wide-area image control feature library construction system provided by this invention can be integrated into an electronic device, such as a terminal or server. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0029] Please see Figure 1 , Figure 1 The flowchart illustrates a method for constructing a wide-area image control feature library based on multi-period images and a real-world 3D model, provided in an embodiment of the present invention. This method, applied in electronic devices, includes the following steps: S1 acquires multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and performs preprocessing.

[0030] Multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, were prepared and preprocessed to form standardized basic data. Historical images and aerial triangulation information were used for initial feature library construction, while newer images were used for feature library updates and matching verification. The historical 3D model was a 3D mesh model generated from historical aerial triangulation information.

[0031] The preprocessing process includes: (1) Image distortion correction: Distortion correction is performed on multi-phase oblique radiographs to eliminate errors caused by radial and tangential distortion.

[0032] (2) Unified coordinate system: After distortion correction, the Bursa seven-parameter coordinate transformation model is used to unify the aerial triangulation information of multiple images and the historical real scene three-dimensional model to the same geodetic coordinate system.

[0033] (3) Image-3D model spatial alignment: Based on the historical real scene 3D model, use exterior orientation elements to spatially align the vertices of the 3D model with multiple images to ensure that the spatial position deviation between the image and the model, and between images from different periods, is within the threshold range.

[0034] The final output includes preprocessed multi-period images, corresponding aerial triangulation information, and historical 3D models, providing a consistent foundation for subsequent feature extraction and matching.

[0035] S2 performs automated feature point extraction and cross-image matching on the preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period identical point pairs.

[0036] In one embodiment, step S2 includes the following steps: S21. The preprocessed multi-phase oblique photogrammetric images are segmented into multiple image blocks using a feature extraction algorithm. The multiple image blocks are then input into the feature extraction network to obtain key point locations and feature descriptors.

[0037] Specifically, the SuperPoint feature extraction algorithm is used to segment the preprocessed multi-stage oblique photogrammetric images into multiple image blocks. A feature extraction network is constructed using a neural network mechanism. This feature extraction includes multiple convolutional layers, through which key point locations and descriptors are extracted.

[0038] S22, perform global operation on the preprocessed multi-stage oblique photography images to generate an edge mask, and filter the extracted feature points based on the edge mask, retaining only feature points whose pixel coordinates are located in the effective edge area, and removing feature points with poor recognition in the texture uniform area.

[0039] Specifically, Canny edge detection is used to generate region masks, which are then used to assist in filtering the extracted feature points. First, Gaussian filtering is applied to the preprocessed multi-stage oblique photographic images for noise reduction. Then, the image gradient is calculated pixel by pixel to obtain the edge intensity and gradient direction of each pixel. Non-maximum suppression is used to remove non-extreme pixels at the edges, retaining only the fine edges at the edge center line, and removing redundant pixels and blurred parts of the edges. Next, a dual threshold is used to filter the edges, retaining feature points with high recognition in the edge region and removing feature points with no obvious recognition in the uniform texture region.

[0040] S23, based on the feature descriptor, the filtered feature points are matched to obtain multiple pairs of cross-period same-name point pairs, forming an initial set of cross-period same-name point pairs.

[0041] Specifically, for the feature descriptors extracted from the multi-period oblique photogrammetric images, a "coarse matching + fine matching" approach is used to complete feature point matching. In the coarse matching stage, a KD-tree index is constructed to perform a fast nearest neighbor search on the target image feature descriptors, calculating the Euclidean distance between two feature descriptors. The smaller the Euclidean distance value, the higher the similarity of the key points corresponding to the two feature descriptors. Initial matching point pairs with high similarity are selected, while mismatched candidate points with extremely low similarity are eliminated, reducing the computational load of subsequent fine matching.

[0042] In the fine matching stage, the RANSAC algorithm is used to fit the fundamental matrix for the initial matched point pairs. The geometric rationality of the matched point pairs is verified by constructing constraints, and mismatched points that do not meet the geometric constraints are eliminated. Finally, a stable initial set of cross-period same-name point pairs is output.

[0043] S24, verify the inter-period same-name point pairs in the initial inter-period same-name point pair set to obtain the final inter-period same-name point pair set.

[0044] Specifically, for the cross-period corresponding point pairs and camera parameters screened in the previous step, the reprojection error verification method is adopted. By comparing the actual pixel coordinates of the corresponding points with the pixel coordinates obtained by backprojection, the deviation between the two (i.e., reprojection error) is calculated. Combined with the set error threshold (for example, in this embodiment, the given error threshold range is less than or equal to 1.5 pixels, which can be adjusted within the range of 1.2-1.8 pixels) and the error calculation step size of 0.1 pixels, point pairs with reprojection errors exceeding the standard are eliminated, and the final cross-period corresponding point set is output.

[0045] S3, based on historical aerial triangulation information and historical 3D models, depth-constrained projection is performed on pairs of corresponding points across periods to obtain the initial object point coordinates. The initial object point coordinates are then filtered to obtain the filtered initial object point coordinates.

[0046] The historical aerial triangulation information includes exterior orientation elements, and step S3 includes the following steps: S31, based on the exterior orientation elements and the historical 3D model, maps the cross-period corresponding point pairs to the 3D space to obtain the initial object point coordinates.

[0047] Specifically, using the exterior orientation elements (camera position and attitude) of historical aerial triangulation information and historical 3D models, the generated cross-period corresponding points are mapped from image pixel coordinates to 3D space through an aerial triangulation projection algorithm, generating preliminary object point coordinates, expressed by the following formula:

[0048] in, For the preliminary three-dimensional object space coordinates, For exterior orientation line elements, R is the depth coefficient, and R is the image exterior orientation rotation matrix. The inverse of the image intrinsic parameter matrix. (u,v) represents the homogeneous pixel coordinates of the corresponding point, and (u,v) represents the original pixel position of the feature point on the image.

[0049] S32, calculate the Euclidean distance between the initial object point coordinates and the nearest point in the historical 3D model. When the Euclidean distance is less than or equal to the preset depth constraint threshold, the spatial position of the initial object point is determined to be reasonable and retained. When the Euclidean distance is greater than the preset depth constraint threshold, the initial object point is determined to have projection deviation and is removed.

[0050] Specifically, using the point cloud information of the historical 3D model, depth constraints are applied to the initial object points. The spatial rationality of the initial object points is verified by calculating the Euclidean distance between the initial object points and the nearest points in the historical 3D model, and outliers caused by projection deviations are eliminated. When the Euclidean distance is less than or equal to a preset depth constraint threshold, the spatial position of the initial object point is deemed reasonable and retained; when the Euclidean distance is greater than the preset depth constraint threshold, the initial object point is deemed to have projection deviations and is eliminated.

[0051] It should be noted that the preset depth constraint threshold is determined based on the historical 3D model accuracy, average point cloud spacing, image ground resolution, and target database accuracy, and is preferably 0.2 m to 0.5 m. In wide-area city-level modeling scenarios, it can be adjusted to 0.5 m to 1.0 m, and in high-precision low-altitude oblique photography scenarios, it can be adjusted to 0.05 m to 0.20 m.

[0052] S33, using statistical methods 3 The outlier detection algorithm removes local outliers, resulting in the initial object coordinates after filtering.

[0053] Specifically, using statistical methods 3 The outlier detection algorithm calculates the mean values ​​of the X, Y, and Z directions for all initial object point coordinates. and standard deviation , will satisfy Outlier removal was performed for each initial object point. By combining the neighborhood consistency judgment algorithm, local outliers are eliminated and dense and consistent point sets are retained, and finally, the high-confidence initial object point coordinates, image features and the correspondence between the three-dimensional object points are obtained.

[0054] S4 filters and selects points on the preprocessed historical 3D model to obtain the filtered historical 3D model.

[0055] In one embodiment, step S4 includes the following steps: S41 performs Gaussian filtering on each point in the preprocessed historical 3D model.

[0056] A Gaussian filtering algorithm is used to perform a weighted average of the neighborhood points around each point in the preprocessed historical 3D model, filtering out noise and isolated points, and outputting effective point cloud data as the core input for subsequent accuracy evaluation and screening.

[0057] S42. Based on the historical 3D model accuracy report, determine the evaluation indicators, remove points in the historical 3D model that do not meet the evaluation indicators, and obtain the filtered historical 3D model.

[0058] Specifically, based on historical 3D model accuracy reports, planar mean square error and elevation mean square error are established as core accuracy evaluation indicators. Simultaneously, for areas with low modeling accuracy (overly complex geometry, insufficient texture information, or high modeling noise), corresponding region filtering conditions are set to provide judgment criteria for automated filtering. The effective point cloud data is divided into several grid regions. Using the automated accuracy evaluation algorithm in step S42, planar mean square error, elevation mean square error, triangular patch density, texture clarity, and point cloud dispersion are calculated region by region. After comparison with preset thresholds, points that do not meet the requirements are eliminated, retaining only high-precision regions such as hardened surfaces and stable features. Finally, the filtered historical 3D model point cloud and associated model accuracy evaluation report are output, providing support for object point solving.

[0059] In one embodiment, step S42 includes the following steps: S421: Calculate the mean plane error and mean elevation error of each point based on the mean value of the model points in the historical 3D model. When both the mean plane error and mean elevation error are greater than the corresponding error threshold, the corresponding point is removed.

[0060] The thresholds for plane mean square error and elevation mean square error can be adjusted according to the actual modeling accuracy requirements.

[0061] The plane error and elevation error of each point are calculated using the following formula:

[0062]

[0063] in, This is the plane error. This is the elevation error. This represents the mean of the x-coordinate, y-coordinate, and elevation of the model points.

[0064] S422: Calculate the density of triangular patches. When the density of triangular patches is greater than a preset density threshold, remove the points in the corresponding region. Calculate the texture clarity. When the texture clarity is less than a preset texture clarity threshold, remove the points in the corresponding region. Calculate the point cloud dispersion. When the point cloud dispersion is greater than a preset point cloud dispersion threshold, remove the points in the corresponding region. Finally, the filtered historical 3D model is obtained.

[0065] Specifically, for regions with overly complex geometric structures, the density of triangular facets is calculated. , ( (where S is the total number of triangular faces within a single grid region and S is the surface area of ​​the corresponding grid region). A higher density indicates a more fragmented and complex local geometry, likely representing vegetation, complex structures, or distorted regions, resulting in higher modeling noise and error. This embodiment of the invention sets a threshold. pcs / m 2 (The threshold can be adjusted according to the actual model accuracy), when The points in that area are removed at that time.

[0066] For areas with insufficient texture information, calculate texture sharpness C. ( (where C represents the horizontal and vertical gradient values ​​of the i-th pixel within the region, and n is the total number of pixels within the region). A smaller C value indicates a more monotonous texture (weak texture areas such as water surfaces, bare ground, and solid-color walls), less clear texture, and lower modeling accuracy. This embodiment of the invention sets a threshold value. (The threshold can be adjusted according to the actual model accuracy), when The points in that area are removed at that time.

[0067] For areas with high modeling noise, calculate the point cloud discretization. , ( The spatial distance between a single point and the center of the regional point cloud. (where n is the average distance between all points in the region and n is the number of points in the region). The larger the value of D, the more severe the point cloud drift, noise, and holes, and the lower the model reliability. In this embodiment of the invention, a threshold is set. (The threshold can be adjusted according to the actual model accuracy), when The points in that area are removed at that time.

[0068] S5 uses exterior orientation elements, cross-period corresponding points, the initial object point coordinates after screening, and the historical 3D model after screening to perform multi-view forward intersection triangulation and least squares optimization to obtain the 3D coordinates of each object point and construct a multi-view association mapping relationship between object points and feature points of multiple periods of images.

[0069] First, a coarse calculation of 3D object points is performed using multi-view triangulation: The high-precision cross-period image corresponding to the same ground feature output in step S2 is used as the core observation object. The preprocessed image exterior orientation elements obtained in step S1 provide attitude and position parameters. Combined with the filtered historical 3D model points obtained in step S4, initial elevation constraints are provided. The high-reliability initial object point coordinates output in step S3 provide reliable initial spatial iteration values, avoiding interference from anomalous observations. Based on the principle of photogrammetric collinearity equations, multi-view forward intersection triangulation is performed on the corresponding pixels of the same ground feature in multiple periods and from multiple perspectives. The observation equations of different images are iteratively solved to obtain the 3D spatial coordinates corresponding to the corresponding pixels, ultimately yielding the initial 3D object point coordinates. This completes the initial mapping from 2D image corresponding points to 3D spatial points, obtaining coarsely located object point data with basic spatial accuracy. The multi-view collinearity equation is as follows:

[0070] Subscript To correspond to images from different perspectives and different periods, ( () represents the coordinates of the same feature point in the camera coordinate system of the l-th image; Next, the coordinate error is globally optimized and corrected using the least squares algorithm: Based on the initial object point coordinate data after screening, combined with the true values ​​of the real corresponding pixel coordinates of each image retained in step S2 and the accurate image exterior orientation elements after preprocessing in step S1 as constraints, the optimization objective is to minimize the sum of reprojection errors of all observed images. The least squares iterative optimization algorithm is adopted to continuously compare the deviation between the calculated pixel coordinates of the object point projected onto the image and the actual matched corresponding pixel coordinates, and continuously iterate and correct the three-dimensional coordinates of the object point to offset the systematic and random errors in the intersection solution process. Finally, convergent and high-precision optimized three-dimensional object point coordinates are obtained, which greatly reduces the reprojection error, effectively improves the spatial positioning accuracy of the object point, and meets the accuracy admission standards of the image control feature library.

[0071] The objective function for the above least squares optimization is:

[0072] Where E is the total reprojection error, the target position is optimized to minimize it, and m is the total number of image viewpoints / periods involved in the calculation. The coordinates of the actual corresponding pixel on the l-th image (the observed true value). The projected pixel coordinates (calculated values) are obtained by inverse calculation from the current object point coordinates.

[0073] The iterative convergence condition is

[0074] in, This represents the coordinate correction amount for a single iteration of the 3D object point. The iterative convergence threshold (engineering value ε= m).

[0075] Finally, the association mapping relationship between object points and feature points of multi-period images is constructed: Based on the high-precision 3D object points optimized in step S3, combined with all cross-period image feature point data and complete matching correspondence retained in step S2, all optimized 3D object points are traversed one by one, and all original feature points corresponding to them in historical images and new images are matched one by one. The image number, multi-view pixel coordinates, feature description association information corresponding to each object point are completely recorded. A two-way mapping relationship between 3D object points and 2D image feature points is established to form a complete multi-view association dataset, realizing traceable association matching of single point location, multi-period images, and multi-view observations, providing data support for subsequent multi-period updates of feature library and aerial triangulation solution tracing.

[0076] After the above processing, the final output is an optimized high-precision 3D object point set and a multi-view correlation table between object points and feature points from multiple image periods.

[0077] S6 constructs a feature library based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple image periods, and dynamically updates the feature library.

[0078] The optimized 3D object points and their corresponding cross-period image relationships are uniformly stored in the database and managed in multiple versions to achieve continuous updates and cross-period reuse of the feature library, supporting aerial triangulation applications under conditions of little or no control.

[0079] (1) Standardized storage of multi-phase image control feature data: Based on the optimized high-precision three-dimensional object points output in step S5, the multi-view correlation mapping relationship between object points and feature points of multi-phase images, combined with the feature point descriptors extracted in step S2, and the accuracy, matching, and stability related index data formed in each step of the process, a dedicated feature library management system is built using a relational database to uniformly collect all core feature data, and fully input the three-dimensional spatial coordinates of each object point, the coordinates of the same pixel in cross-phase images, feature description information, the number of multi-view observations, matching stability parameters, reprojection error and coordinate optimization deviation, etc., to complete the initial construction of the multi-phase image control feature library, forming a standardized image control feature dataset with complete data, standardized fields, and searchability, and realizing centralized management of high-precision image control features.

[0080] (2) Constructing a multi-period dynamic update mechanism for the feature library: For newly added external image data, after the preprocessing process in step S1 to complete standardization processes such as image correction and coordinate unification, and after feature extraction and matching in step S2, the feature points of the new image are matched with the original feature points corresponding to the object points already in the feature library. For the existing object points that are successfully matched, their dynamic parameters such as the number of observations in multiple periods and the stability of matching are updated in real time. For the newly identified unmatched feature points, the processing process in steps S3 to S5 is automatically called to complete the projection screening, three-dimensional coordinate solution and accuracy optimization of the new feature points before they are added to the library. Finally, the dynamic iterative update of the feature library is realized, which effectively accommodates newly added image data and continuously expands the sample size and cross-period adaptability of the feature library.

[0081] Furthermore, precision checks and quality control are conducted: This embodiment represents the final quality verification step in the entire image control feature library construction process. It involves a comprehensive quality check and purification of the preprocessed image data, historical model data, optimized feature library results, and accuracy reports. This is achieved through processes such as reprojection accuracy verification, secondary screening of multi-dimensional outliers, and statistical analysis. Low-precision, abnormal, and invalid object points are removed, ultimately outputting a final version of the multi-phase image control feature library with compliant accuracy and stable reliability. This ensures the feature library can be directly used for cross-phase aerial triangulation and high-precision multi-phase 3D modeling. The specific steps include: (1) Object point reprojection accuracy verification: Based on the three-dimensional object point data in the multi-period image control feature library output in step S6, combined with the standardized multi-period image data preprocessed in step S1, and compared with the original image corresponding pixel points, all object points in the feature library are reverse reprojected to the corresponding multi-period image viewpoint. The deviation between the reprojected pixel position and the actual matched pixel position is compared point by point. The point accuracy level is judged according to the preset accuracy threshold. Object points with excessive deviation are uniformly marked as low-precision abnormal points, providing a basis for subsequent point removal and purification.

[0082] (2) Secondary screening to remove abnormal and invalid object points: Based on the low-precision point data marked in (1) of this step, the high-precision historical model point set after screening in step S4, combined with the depth constraints adopted in step S3, 3 The statistical anomaly detection and neighborhood consistency judgment method conducts a comprehensive secondary purification and screening of all object points in the feature library. It verifies the spatial distance deviation between object points and high-precision historical model points, the rationality of the overall coordinate distribution of points, and the spatial consistency of neighboring points. It batch removes marked low-precision points, spatially discrete anomalies, coordinate statistical anomalies, and invalid points with poor neighborhood consistency, which greatly improves the reliability and spatial consistency of the overall feature library.

[0083] After the above processing, a final, highly reliable, multi-stage image control feature library is output, which has undergone layers of verification and optimization. This achieves a standardized quality closed loop for the feature library results, providing reliable data support for subsequent real-scene 3D modeling and aerial triangulation.

[0084] Based on the method described in the above embodiments, this embodiment will further describe it from the perspective of a wide-area image control feature library construction system based on multi-phase images and real-scene 3D models. The wide-area image control feature library construction system based on multi-phase images and real-scene 3D models can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server, or other devices. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0085] Please see Figure 2 , Figure 2 This invention specifically describes a wide-area image control feature library construction system based on multi-period images and real-world 3D models, which is applied in electronic devices. This system may include: The acquisition and preprocessing module is used to acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and to perform preprocessing. The image feature extraction and matching module is used to automatically extract feature points and perform cross-image matching on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period same-name point pairs. The same-point depth constraint projection and filtering module is used to perform depth constraint projection on cross-period same-point pairs based on historical aerial triangulation information and historical 3D models to obtain initial object point coordinates, and to filter the initial object point coordinates to obtain filtered initial object point coordinates. The historical 3D model point filtering module is used to filter and select points on the preprocessed historical 3D model to obtain the filtered historical 3D model. The 3D object point solution and multi-view association module is used to perform multi-view forward intersection triangulation and least squares optimization based on exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates and the selected historical 3D model to obtain the 3D coordinates of each object point and construct the multi-view association mapping relationship between the object point and the feature points of multiple periods of images. The feature library construction and update module is used to construct a feature library based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple phases of images, and to dynamically update the feature library.

[0086] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0087] In addition, this embodiment of the invention also provides an electronic device, which may be a computer, tablet computer, or other similar device. This electronic device can implement the steps in any embodiment of the wide-area image control feature library construction method based on multi-period images and real-world 3D models provided in this embodiment of the invention. Therefore, it can achieve the beneficial effects that any of the wide-area image control feature library construction methods based on multi-period images and real-world 3D models provided in this embodiment of the invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0088] Figure 3 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the wide-area image control feature library construction method based on multi-period images and real-scene 3D models provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0089] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control.

[0091] Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0092] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0093] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0094] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0095] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations: Acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and perform preprocessing; Automated feature point extraction and cross-image matching are performed on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period identical point pairs. Based on historical aerial triangulation information and historical 3D models, depth-constrained projection is performed on pairs of corresponding points across different periods to obtain initial object point coordinates. The initial object point coordinates are then filtered to obtain filtered initial object point coordinates. The points on the preprocessed historical 3D model are filtered to obtain the filtered historical 3D model. Based on the exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates, and the selected historical 3D model, multi-view forward intersection triangulation and least squares optimization are performed to obtain the 3D coordinates of each object point, and a multi-view association mapping relationship between the object point and the feature points of multiple periods of images is constructed. A feature library is constructed based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple image periods, and the feature library is dynamically updated.

[0096] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0097] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the wide-area image control feature library construction method based on multi-period images and real-world 3D models provided by the present invention.

[0098] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0099] Since the instructions stored in the storage medium can execute the steps in any embodiment of the wide-area image control feature library construction method based on multi-period images and real-scene 3D models provided in the embodiments of the present invention, the beneficial effects that any wide-area image control feature library construction method based on multi-period images and real-scene 3D models provided in the embodiments of the present invention can achieve can be seen in the previous embodiments, and will not be repeated here.

[0100] The foregoing has provided a detailed description of a method, system, storage medium, and electronic device for constructing a wide-area image control feature library based on multi-period images and real-world 3D models provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models, characterized in that, The method includes: Acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and perform preprocessing; Automated feature point extraction and cross-image matching are performed on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period identical point pairs. Based on historical aerial triangulation information and historical 3D models, depth-constrained projection is performed on pairs of corresponding points across different periods to obtain initial object point coordinates. The initial object point coordinates are then filtered to obtain filtered initial object point coordinates. The points on the preprocessed historical 3D model are filtered to obtain the filtered historical 3D model. Based on the exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates, and the selected historical 3D model, multi-view forward intersection triangulation and least squares optimization are performed to obtain the 3D coordinates of each object point, and a multi-view association mapping relationship between the object point and the feature points of multiple periods of images is constructed. A feature library is constructed based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple image periods, and the feature library is dynamically updated.

2. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 1, characterized in that, Automated feature point extraction and cross-image matching are performed on preprocessed multi-period oblique photogrammetric images to obtain a set of corresponding point pairs across different periods, including: A feature extraction algorithm is used to segment the preprocessed multi-phase oblique photogrammetric images into multiple image blocks. The multiple image blocks are then input into a feature extraction network to obtain key point locations and feature descriptors. A global operation is performed on the preprocessed multi-stage oblique photographic images to generate an edge mask. Based on the edge mask, the extracted feature points are filtered, retaining only feature points whose pixel coordinates are located in the effective edge area and removing feature points with poor discernibility in areas with uniform texture. Based on the feature descriptor, the filtered feature points are matched to obtain multiple pairs of cross-period same-name point pairs, forming an initial set of cross-period same-name point pairs; The initial set of intertemporal identical point pairs is verified to obtain the final set of intertemporal identical point pairs.

3. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 1, characterized in that, The historical aerial triangulation information includes exterior orientation elements. Based on the historical aerial triangulation information and the historical 3D model, depth-constrained projection is performed on pairs of corresponding points across different periods to obtain initial object point coordinates. The initial object point coordinates are then filtered to obtain filtered initial object point coordinates, including: Based on the exterior orientation elements and the historical 3D model, the cross-period corresponding point pairs are mapped to the 3D space to obtain the initial object point coordinates; Calculate the Euclidean distance between the initial object point coordinates and the nearest point in the historical 3D model. If the Euclidean distance is less than or equal to a preset depth constraint threshold, the initial object point is determined to have a reasonable spatial position and is retained. If the Euclidean distance is greater than the preset depth constraint threshold, the initial object point is determined to have a projection deviation and is discarded. Using statistical methods 3 The outlier detection algorithm removes local outliers, resulting in the initial object coordinates after filtering.

4. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 3, characterized in that, Based on the exterior orientation elements and the historical 3D model, the cross-period corresponding point pairs are mapped to 3D space to obtain the initial object space point coordinates, which are expressed by the following formula: in, For the preliminary three-dimensional object space coordinates, For exterior orientation line elements, R is the depth coefficient, and R is the image exterior orientation rotation matrix. The inverse of the image intrinsic parameter matrix. (u,v) represents the homogeneous pixel coordinates of the corresponding point, and (u,v) represents the original pixel position of the feature point on the image.

5. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 1, characterized in that, The points on the preprocessed historical 3D model are filtered to obtain the filtered historical 3D model, including: Gaussian filtering is applied to each point in the preprocessed historical 3D model. The evaluation indicators are determined based on the historical 3D model accuracy report. Points in the historical 3D model that do not meet the evaluation indicators are removed to obtain the filtered historical 3D model.

6. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 5, characterized in that, Evaluation indicators are determined based on the historical 3D model accuracy report. Points in the historical 3D model that do not meet the evaluation indicators are removed, resulting in a filtered historical 3D model, including: The mean plane error and mean elevation error of each point are calculated based on the mean of the model points in the historical 3D model. When both the mean plane error and the mean elevation error are greater than the corresponding error threshold, the corresponding point is removed. Calculate the density of triangular patches. When the density of triangular patches is greater than a preset density threshold, remove points in the corresponding region. Calculate the texture clarity. When the texture clarity is less than a preset texture clarity threshold, remove points in the corresponding region. Calculate the point cloud dispersion. When the point cloud dispersion is greater than a preset point cloud dispersion threshold, remove points in the corresponding region. Finally, obtain the filtered historical 3D model.

7. The method for constructing a wide-area image control feature library based on multi-period images and real-scene 3D models according to claim 1, characterized in that, Multi-view forward intersection triangulation is performed based on exterior orientation elements, corresponding points across periods, the selected initial object space coordinates, and the selected historical 3D model, expressed by the following formula: Subscript To correspond to images from different perspectives and different periods, ( () represents the coordinates of the same feature point in the camera coordinate system of the l-th image; The objective function for least squares optimization is: Where E is the total reprojection error, and m is the total number of image viewpoints / periods involved in the calculation. Let be the coordinates of the actual pixel with the same name on the l-th image. The projected pixel coordinates are calculated from the current object point coordinates. The iterative convergence condition is in, This represents the coordinate correction amount for a single iteration of the 3D object point. This is the threshold for iterative convergence.

8. A wide-area image control feature library construction system based on multi-period images and real-scene 3D models, characterized in that, include: The acquisition and preprocessing module is used to acquire multiple oblique photogrammetric images and corresponding aerial triangulation information, as well as historical 3D models, and to perform preprocessing. The image feature extraction and matching module is used to automatically extract feature points and perform cross-image matching on preprocessed multi-period oblique photogrammetric images to obtain a set of cross-period same-name point pairs. The same-point depth constraint projection and filtering module is used to perform depth constraint projection on cross-period same-point pairs based on historical aerial triangulation information and historical 3D models to obtain initial object point coordinates, and to filter the initial object point coordinates to obtain filtered initial object point coordinates. The historical 3D model point filtering module is used to filter and select points on the preprocessed historical 3D model to obtain the filtered historical 3D model. The 3D object point solution and multi-view association module is used to perform multi-view forward intersection triangulation and least squares optimization based on exterior orientation elements, cross-period corresponding points, the selected initial object point coordinates and the selected historical 3D model to obtain the 3D coordinates of each object point and construct the multi-view association mapping relationship between the object point and the feature points of multiple periods of images. The feature library construction and update module is used to construct a feature library based on the three-dimensional coordinates of the object point and the multi-view correlation mapping relationship between the object point and feature points of multiple phases of images, and to dynamically update the feature library.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions, which are adapted to be loaded by a processor to execute the method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The system includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the steps in the method for constructing a wide-area image control feature library based on multi-period images and real-world 3D models as described in any one of claims 1 to 7.

Images