Three-dimensional modeling method and system based on oblique photography technology

Abstract

According to the three-dimensional modeling method and system based on the oblique photography technology, by means of a three-dimensional modeling scheme based on the oblique photography technology, an oblique image, POS and ground image control data of a specified measurement area are collected firstly, and then a real scene three-dimensional model is generated according to the process operation of aerial triangulation measurement, multi-view image dense matching, TIN structure construction and texture combined generation. Manual intervention is reduced, the modeling period is shortened, and scenes with high timeliness requirements are met; accurate image exterior orientation elements and ground point coordinates provided by aerial triangulation provide geometric constraints for multi-view image matching, so that the matching is efficient and accurate, a large amount of accurate three-dimensional point cloud data is quickly generated, and the overall efficiency is improved; the automatic process reduces the dependence on professionals and saves the labor cost; the TIN structure can more accurately describe the surface shape of the ground feature, and after the texture is combined, the generated real scene three-dimensional model can better present the geometric features and appearance details of a real scene, and the reality sense and the detail effect are better.

Description

A 3D modeling method and system based on oblique photogrammetry Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a 3D modeling method and system based on oblique photogrammetry. Background Technology

[0002] In many fields such as geographic information mapping, urban planning, virtual reality, and cultural heritage protection, the demand for 3D modeling of real-world scenes is growing. Accurate 3D models can provide more intuitive and comprehensive spatial information, which helps in various analyses and decision-making processes.

[0003] Traditional 3D modeling methods mainly rely on manual measurement and aerial photogrammetry. However, while manual measurement can obtain relatively accurate models, it suffers from low efficiency and high cost. Manual measurement requires a lot of time and manpower, and it is difficult to measure complex scenes. The manual modeling process is cumbersome, requires a high level of technical skill from the modelers, and has a long modeling cycle, making it difficult to meet the needs of rapid modeling of large-scale scenes. In addition, traditional aerial photogrammetry usually uses vertical photography, which acquires relatively limited image information and lacks sufficient side information of vertical objects such as buildings, resulting in a lack of detail and realism in the generated 3D models. Summary of the Invention

[0004] In view of this, the present invention proposes a 3D modeling method and system based on oblique photogrammetry, which can solve the defects of existing technologies such as low efficiency, high cost, difficulty in meeting the needs of rapid modeling of large-scale scenes, and lack of detail and realism in the generated 3D models.

[0005] The technical solution of this invention is implemented as follows:

[0006] A 3D modeling method based on oblique photogrammetry includes:

[0007] Oblique imagery data, POS data, and ground image control data of a designated survey area are collected using oblique photogrammetry.

[0008] Oblique photogrammetry aerial triangulation is performed based on oblique image data, POS data, and ground image control data of the designated survey area to obtain the image exterior orientation elements and ground point coordinates.

[0009] Dense matching of multi-view images is performed based on image exterior orientation elements and ground point coordinates to generate 3D point cloud data.

[0010] Constructing a TIN structure based on 3D point cloud data;

[0011] Based on the constructed TIN structure and combined with the texture information of the oblique image, a real-world 3D model is generated.

[0012] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the acquisition of oblique image data, POS data, and ground image control data for a specified survey area using oblique photogrammetry specifically includes:

[0013] An aerial platform equipped with an oblique photography device and a POS device is used to perform oblique photography data acquisition tasks in a designated survey area. During the acquisition process, oblique image data and POS data of the designated survey area are acquired simultaneously. At the same time, ground control points are set up on the ground in the designated survey area, and ground control data is acquired using a full field measurement method. The oblique image data is used to record multi-angle image information of ground features in the designated survey area, the POS data is used to record the spatial position and attitude information of the aerial platform during photography, and the ground control data is used to provide ground control reference.

[0014] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the step of performing oblique photogrammetric aerial triangulation based on oblique image data, POS data, and ground image control data of a specified survey area to obtain image exterior orientation elements and ground point coordinates specifically includes:

[0015] Using the spatial position and attitude information of the flight platform recorded by the POS data, the initial exterior orientation elements of each oblique image in the oblique image data are initially determined.

[0016] Based on the oblique image data, the image plane coordinates of the connection point were measured;

[0017] Based on the initial exterior orientation elements, the image plane coordinates of the ligature points, and the ground image control data, a joint adjustment calculation is performed to obtain the image exterior orientation elements and the coordinates of ground points within the specified survey area.

[0018] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the step of performing joint adjustment calculations based on initial exterior orientation elements, image plane coordinates of tie points, and ground image control data to obtain image exterior orientation elements and ground point coordinates within a specified survey area specifically includes:

[0019] Based on the initial exterior orientation elements, and in accordance with the collinearity condition equation, combined with the image plane coordinates of the connection points and the coordinates of the ground control points in the ground control data, an error equation is constructed that includes the relationship between the image plane coordinates of the connection points and the coordinates of the ground control points.

[0020] By combining the error equations of all the connection points, we obtain the joint adjustment system;

[0021] An iterative algorithm is used to solve the joint adjustment system to obtain the image exterior orientation elements and the coordinates of ground points within the specified survey area.

[0022] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the step of generating 3D point cloud data by performing dense matching of multi-view images based on image exterior orientation elements and ground point coordinates specifically includes:

[0023] By using the exterior orientation elements of the images, feature points on different images are projected onto the object space, and combined with the coordinates of ground points, geometric constraint relationships between multi-view images are constructed.

[0024] Based on the ground point coordinates, search for corresponding feature points on multiple related images to determine candidate pairs of corresponding points between images;

[0025] Based on the geometric constraints between multi-view images, candidate pairs of corresponding points between images are matched to determine the final pairs of corresponding points between images.

[0026] Based on the pairs of corresponding points between the final images, the three-dimensional coordinates of the object space corresponding to each corresponding point are calculated using the principle of spatial forward intersection.

[0027] All the calculated 3D points are organized and summarized to form dense 3D point cloud data.

[0028] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the construction of a TIN structure based on 3D point cloud data specifically includes:

[0029] The acquired 3D point cloud data is preprocessed to obtain preprocessed 3D point cloud data;

[0030] The Delaunay triangulation algorithm is used to perform initial triangulation on the preprocessed 3D point cloud data to construct the initial TIN structure.

[0031] An energy function is defined to optimize the initial TIN structure. The energy function includes a shape energy term and a smoothing energy term. The shape energy term measures how close the shape of the triangle is to that of an equilateral triangle, and the smoothing energy term measures the degree of change in the normal vector between adjacent triangles.

[0032] By minimizing the energy function and adjusting the positions of the vertices in the TIN structure, the final TIN structure is obtained.

[0033] As a further optional solution to the aforementioned 3D modeling method based on oblique photogrammetry, the generation of a real-world 3D model based on the constructed TIN structure and combined with the texture information of the oblique image specifically includes:

[0034] Based on the shooting parameters of the oblique image and the geometric information of each triangular facet in the TIN structure, the visible area of ​​each triangular facet in the oblique image is calculated, and the texture mapping relationship between the triangular facet and the oblique image is established.

[0035] For each triangular facet, if it has a visible area in multiple tilted images, then based on the multi-image fusion algorithm, the texture information of the corresponding areas in the multiple images is fused to obtain the best texture information of the triangular facet.

[0036] The fused texture information is mapped onto the triangular facets of the TIN structure according to the established texture mapping relationship, generating a realistic 3D model with real textures.

[0037] A 3D modeling system based on oblique photogrammetry technology includes:

[0038] The data acquisition module is used to acquire oblique image data, POS data, and ground image control data of a designated survey area based on oblique photogrammetry technology.

[0039] The aerial triangulation module is used to perform oblique photogrammetry aerial triangulation based on oblique image data, POS data and ground image control data of a specified survey area to obtain the image exterior orientation elements and ground point coordinates.

[0040] The dense matching module is used to perform dense matching of multi-view images based on the image exterior orientation elements and ground point coordinates to generate 3D point cloud data.

[0041] The TIN structure building module is used to build TIN structures based on 3D point cloud data.

[0042] The model generation module is used to generate a realistic 3D model based on the constructed TIN structure and the texture information of the oblique image.

[0043] A computing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described three-dimensional modeling methods based on oblique photogrammetry.

[0044] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described three-dimensional modeling methods based on oblique photogrammetry.

[0045] The beneficial effects of this invention are as follows: After acquiring oblique image data, POS data, and ground image control data for a designated survey area using oblique photogrammetry, aerial triangulation, multi-view image dense matching, TIN structure construction, and generation of a realistic 3D model by combining texture information are performed sequentially. These steps are closely integrated, reducing manual intervention and significantly shortening the modeling cycle. This allows for rapid processing of large-scale scene data, meeting the needs of applications with high timeliness requirements. Simultaneously, multi-view image dense matching generates 3D point cloud data based on image exterior orientation elements and ground point coordinates. The accurate image exterior orientation elements and ground point coordinates obtained through precise aerial triangulation provide accurate geometric constraints for multi-view image matching, enabling more efficient matching. The process is more efficient and accurate, capable of quickly generating large amounts of precise 3D point cloud data, further improving overall modeling efficiency. In addition, the automated data processing workflow reduces reliance on professional surveyors and modelers, lowering labor costs. Furthermore, accurate image exterior orientation elements and ground point coordinates are obtained through oblique photogrammetry aerial triangulation, providing a precise geometric basis for dense matching of multi-view images and TIN structure construction. The TIN structure built based on precise 3D point cloud data can more accurately describe the surface shape of ground features. Combined with the texture information of oblique images, the generated realistic 3D model can better reflect the geometric features and appearance details of the real scene, enhancing the realism and detail of the model. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 is a flowchart of a three-dimensional modeling method based on oblique photogrammetry according to the present invention;

[0048] Figure 2 is a schematic diagram of the composition of a three-dimensional modeling system based on oblique photogrammetry according to the present invention;

[0049] Figure 3 is a schematic diagram of the composition of a computing device according to the present invention. Detailed Implementation

[0050] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Referring to Figures 1 to 3, a 3D modeling method based on oblique photogrammetry includes:

[0052] Oblique photogrammetry is used to acquire oblique image data, POS data, and ground image control data for a designated survey area, specifically including:

[0053] An aerial platform equipped with an oblique photography device and a POS device is used to perform oblique photography data acquisition tasks in a designated survey area. During the acquisition process, oblique image data and POS data of the designated survey area are acquired simultaneously. At the same time, ground control points are set up on the ground in the designated survey area, and ground control data is acquired using a full field measurement method. The oblique image data is used to record multi-angle image information of ground features in the designated survey area, the POS data is used to record the spatial position and attitude information of the aerial platform during photography, and the ground control data is used to provide ground control references for subsequent data processing.

[0054] Specifically, a flight platform equipped with an oblique photography device and a POS device is used for data acquisition. The oblique photography device can capture images of the designated survey area from multiple different angles, allowing the acquired oblique image data to record multi-angle image information of ground features. Compared with traditional vertical photography, it can more comprehensively capture detailed features such as the sides and tops of ground features, providing rich texture and geometric information for subsequent 3D modeling, which helps to build a more realistic and detailed 3D model. The POS device simultaneously records the spatial position and attitude information of the flight platform during the acquisition process, i.e., POS data. This data provides a key basis for the accurate geolocation and orientation of the images, ensuring the accurate position and angle of the oblique image data in space and improving the spatial accuracy of the data. Ground control points are set up on the ground in the designated survey area, and ground control data is acquired using full field measurement methods. These control points provide a stable ground control benchmark for the entire data acquisition and processing process, which can effectively eliminate or reduce data deviations caused by factors such as flight platform instability and sensor errors, further improving the overall accuracy and reliability of the data.

[0055] During the flight platform data acquisition process, oblique image data and POS data are acquired simultaneously, avoiding multiple flights or repeated data collection of the same area, which greatly improves the efficiency of data acquisition. At the same time, this synchronous acquisition method ensures a strict time correspondence between image data and POS data, simplifies the subsequent data processing process, and reduces data processing time and costs. The method of using ground control points measured in the entire field has strong adaptability. Control points can be reasonably deployed according to the actual conditions of the survey area, such as topography and landforms. Whether it is a densely built-up urban area, a mountainous area, or other complex terrain areas, accurate ground control data can be obtained through reasonable deployment of control points, ensuring that the data acquisition scheme can be effectively implemented in different types of survey areas, and improving the applicability of this technical solution in various complex scenarios.

[0056] Oblique photogrammetry and aerial triangulation are performed based on oblique image data, POS data, and ground image control data for the designated survey area to obtain image exterior orientation elements and ground point coordinates, specifically including:

[0057] Using the spatial position and attitude information of the flight platform recorded by the POS data, the initial exterior orientation element of each oblique image in the oblique image data is initially determined. This initial exterior orientation element provides an initial approximation for subsequent calculations.

[0058] Based on oblique image data, the image plane coordinates of the connection points are accurately measured. The connection points include ground control points and common connection points that can be clearly identified on multiple oblique images. For ground control points, their image plane coordinates are determined by on-site measurement and precise matching with the images. For common connection points, an image matching algorithm is used to perform automatic or semi-automatic measurement on multiple related images to obtain their image plane coordinates.

[0059] Based on the initial exterior orientation elements, the image plane coordinates of the ligature points, and the ground image control data, a joint adjustment calculation is performed to obtain the image exterior orientation elements and the coordinates of ground points within the specified survey area.

[0060] Specifically, using the spatial position and attitude information of the flight platform recorded in the POS data, the initial exterior orientation elements of each oblique image in the oblique image data are initially determined. POS data has high accuracy, providing reliable initial approximations for subsequent calculations, reducing error accumulation during the calculation process, and making the final determined image exterior orientation elements more accurate, laying a precise foundation for subsequent data processing and 3D modeling. Based on the oblique image data, the image plane coordinates of connection points (including ground control points and common connection points) are accurately measured. For ground control points, their image plane coordinates are determined through on-site measurements and precise matching with the images, ensuring that the ground control points are within the image plane coordinates. Accurate positioning on images; for common connection points, image matching algorithms are used for automatic or semi-automatic measurement on multiple related images, which can make full use of the information of multi-view images, improve the accuracy and reliability of measurement, and further ensure the precision of aerial triangulation; joint adjustment calculation is performed based on the initial exterior orientation elements, the image plane coordinates of the connection points, and the ground image control data. The joint adjustment can comprehensively consider the information and error characteristics of various data, optimize the image exterior orientation elements and ground point coordinates as a whole, effectively eliminate or reduce the influence of various errors, thereby obtaining more accurate image exterior orientation elements and ground point coordinates within the specified survey area, and improving the overall precision of aerial triangulation.

[0061] When acquiring the image plane coordinates of common connection points, an image matching algorithm is used for automatic or semi-automatic measurement. Compared with traditional manual measurement methods, this greatly improves the efficiency of data processing, reduces manual intervention and human error, and can significantly shorten processing time, especially in large-scale data processing scenarios, meeting the need for rapid acquisition of measurement results. This technical solution forms a complete and logically clear aerial triangulation measurement process. From the determination of initial exterior orientation elements and the measurement of image plane coordinates of connection points to joint adjustment calculations, each step is closely linked and interconnected. This optimized process design makes the data processing process smoother, reduces unnecessary repetitive operations and data conversion steps, and further improves the overall data processing efficiency.

[0062] In some embodiments, the step of performing joint adjustment calculations based on initial exterior orientation elements, image plane coordinates of tie points, and ground image control data to obtain image exterior orientation elements and ground point coordinates within a specified survey area specifically includes:

[0063] Based on the initial exterior orientation elements, and according to the collinearity condition equation (which describes the geometric relationship between the image point, the camera center, and the corresponding ground point), combined with the image plane coordinates of the connecting point and the coordinates of the ground control points in the ground control data, an error equation is constructed that includes the relationship between the image plane coordinates of the connecting point and the coordinates of the ground control points. Specifically, the observed image plane coordinates of the connecting point are compared with the theoretical image plane coordinates calculated using the collinearity condition equation based on the coordinates of its corresponding ground point (the coordinates of the ground control points are known, and the coordinates of other ground points are to be solved) and the current exterior orientation elements. The residual between the two is obtained, and the error equation is constructed accordingly.

[0064] By combining the error equations of all the connection points and taking into account the geometric relationship between multiple oblique images, a holistic joint adjustment system is formed. This joint adjustment system comprehensively considers the information of all connection points to improve the accuracy and reliability of the measurement.

[0065] An iterative algorithm is used to solve the joint adjustment system. Based on the least squares principle, the sum of squared residuals of all image point coordinate observations is minimized. In each iteration, a certain adjustment strategy (such as calculating the adjustment amount based on the partial derivative of the error equation) is adopted to adjust the image exterior orientation elements and ground point coordinates according to the current residuals. After multiple iterations, the residuals meet the preset accuracy requirements, and finally the accurate image exterior orientation elements and ground point coordinates within the specified survey area are obtained.

[0066] Specifically, based on the initial exterior orientation elements, and according to the collinearity condition equation, combined with the image plane coordinates of the tie points and the coordinates of ground control points in the ground control data, an error equation is constructed that includes the relationship between the image plane coordinates of the tie points and the coordinates of the ground control points. The collinearity condition equation is a fundamental theoretical equation in photogrammetry. The error equation constructed based on it can accurately describe the geometric relationship between image coordinates and ground coordinates, providing an accurate mathematical model for subsequent adjustment calculations and helping to improve the accuracy of the final measurement results. The error equations of all tie points are combined to obtain a joint adjustment system, which is solved using an iterative algorithm. The joint adjustment can comprehensively consider the information of all tie points. Through iterative algorithms, the image exterior orientation elements and ground point coordinates are continuously adjusted, so that the measurement results gradually approach the true values. This overall optimization method can effectively eliminate or reduce the impact of measurement errors of individual tie points on the overall results, further improving the measurement accuracy of image exterior orientation elements and ground point coordinates within the specified survey area.

[0067] Multi-view imagery dense matching is performed based on image exterior orientation elements and ground point coordinates to generate 3D point cloud data, specifically including:

[0068] The SIFT algorithm is used to extract features from multi-view tilted images to obtain feature points on the images; the feature points on different images are projected onto the object space using the image exterior orientation elements, and the geometric constraint relationship between the multi-view images is constructed by combining the ground point coordinates.

[0069] Based on the ground point coordinates, search for corresponding feature points on multiple related images to determine candidate pairs of corresponding points between images;

[0070] Based on the geometric constraints between multi-view images, a multi-view least squares matching method based on object-side geometric constraints is adopted to match the corresponding pairs of points between images, ensuring that the corresponding pairs of points are matched.

[0071] Based on the pairs of corresponding points between the final images, the three-dimensional coordinates of the object space corresponding to each corresponding point are calculated using the principle of spatial forward intersection.

[0072] All calculated 3D points are organized and summarized to form dense 3D point cloud data. Then, statistical filtering is used to filter the point cloud data to remove noise points and mismatched points, thereby improving the quality of the 3D point cloud data.

[0073] Specifically, the SIFT (Scale Invariant Feature Transform) algorithm is used to extract features from multi-view tilted images. SIFT is invariant to rotation, scaling, and brightness changes, enabling the extraction of stable and representative image feature points. These feature points are projected onto the object space using image exterior orientation elements, and geometric constraints between multi-view images are constructed using ground point coordinates. These geometric constraints accurately describe the spatial correspondence between feature points on different images, providing precise guidance for subsequent search and matching of corresponding points, significantly improving the accuracy of dense matching of multi-view images. After searching for candidate corresponding point pairs based on ground point coordinates, a multi-view least squares matching method based on object space geometric constraints is used to match the corresponding points, based on the geometric constraints between multi-view images. The least squares matching method can continuously iterate and optimize to ensure that the matching results satisfy the geometric constraints as much as possible, guaranteeing the matching of corresponding point pairs and further improving the matching accuracy. This lays the foundation for the subsequent accurate calculation of the three-dimensional coordinates of the object space.

[0074] Based on the finally determined pairs of corresponding points between images, the three-dimensional coordinates of each corresponding point in object space are calculated using the principle of spatial forward intersection. Spatial forward intersection is a method for accurately calculating three-dimensional coordinates, which can make full use of the geometric information of the images and ensure that the calculated three-dimensional point coordinates have high accuracy. After organizing and summarizing all the calculated three-dimensional points to form three-dimensional point cloud data, statistical filtering is used to filter the point cloud data. Statistical filtering can effectively remove noise points and mismatched points based on the statistical characteristics of point cloud data, further improving the quality of three-dimensional point cloud data and making the point cloud data more accurately and clearly reflect the surface features of objects.

[0075] Constructing a TIN (Triangulated Irregular Network) structure based on 3D point cloud data specifically includes:

[0076] The acquired 3D point cloud data is preprocessed to obtain preprocessed 3D point cloud data. The preprocessing includes denoising and filtering. The denoising uses a statistical filtering method to remove outliers from the 3D point cloud data. The filtering uses a Gaussian filtering method to smooth the 3D point cloud data and reduce noise interference.

[0077] The Delaunay triangulation algorithm is used to perform initial triangulation on the preprocessed 3D point cloud data to construct an initial TIN (triangular network of irregularities) structure. During the Delaunay triangulation process, for each newly inserted point, it is checked whether it meets the Delaunay criterion. If it does not meet the criterion, the connection relationship of the triangle is adjusted by edge swapping to make it meet the Delaunay criterion.

[0078] An energy function is defined to optimize the initial TIN (triangular mesh) structure. The energy function includes a shape energy term and a smoothing energy term. The shape energy term measures how close the shape of the triangle is to that of an equilateral triangle, and the smoothing energy term measures the degree of change in the normal vector between adjacent triangles.

[0079] By minimizing the energy function, the positions of the vertices in the TIN (Triangulated Irregular Network) structure are adjusted to make the triangles in the TIN structure more regular in shape and the transition between adjacent triangles smoother, thus obtaining the final TIN structure.

[0080] Specifically, the Delaunay triangulation algorithm is used to perform initial triangulation on the preprocessed 3D point cloud data. The Delaunay triangulation algorithm has the characteristics of rigorous theory and high implementation efficiency. It can quickly divide discrete point cloud data into non-overlapping triangles, and the generated triangles are as close as possible to equilateral triangles, providing a good initial structure for subsequent optimization and ensuring the efficiency of the TIN (triangular network of irregularities) structure construction process.

[0081] An energy function containing shape energy and smoothness energy terms is defined to optimize the initial TIN (Triangulated Irregular Network) structure. The shape energy term measures how closely the shape of the triangles resembles an equilateral triangle. Minimizing the shape energy term makes the triangles in the TIN structure more regular in shape, reduces the occurrence of elongated triangles, and improves the geometric stability of the TIN structure. The smoothness energy term measures the degree of change in the normal vector between adjacent triangles. Minimizing the smoothness energy term makes the transition between adjacent triangles smoother, allowing the TIN structure to better fit the surface shape of the object, thereby improving the quality and accuracy of the TIN structure.

[0082] By minimizing the energy function and adjusting the position of the vertices in the TIN (Triangulated Irregular Network) structure, the TIN structure is optimized as a whole. This overall optimization method can comprehensively consider the shape of the triangles and the smoothness between adjacent triangles, and adjust the TIN structure from a global perspective. This makes the final TIN structure more accurately describe the surface of the object represented by the 3D point cloud data, thus improving the accuracy and quality of the TIN structure.

[0083] Based on the constructed TIN (Triangular Irregular Network) structure and combined with the texture information of the oblique imagery, a realistic 3D model is generated, specifically including:

[0084] Based on the shooting parameters of the oblique image and the geometric information of each triangular facet in the TIN (triangular mesh) structure, the visible area of ​​each triangular facet in the oblique image is calculated, and the texture mapping relationship between the triangular facet and the oblique image is established.

[0085] For each triangular facet, if it has a visible area in multiple tilted images, an algorithm based on multi-image fusion is used to comprehensively consider factors such as the sharpness and color consistency of each image, and to fuse the texture information of the corresponding areas in multiple images to obtain the best texture information of the triangular facet.

[0086] The fused texture information is mapped onto the triangular facets of the TIN (Triangular Irregular Network) structure according to the established texture mapping relationship, generating a realistic 3D model with real textures.

[0087] Specifically, based on the shooting parameters of the oblique image and the geometric information of each triangular facet in the TIN structure, the visible area of ​​each triangular facet on the oblique image is calculated, and a texture mapping relationship is established. This texture mapping method based on precise geometric calculation and shooting parameters can ensure that the texture on the oblique image is accurately mapped to the corresponding triangular facet, making the generated real-world 3D model closer to the real scene in appearance, greatly enhancing the realism and visualization effect of the model.

[0088] For triangular patches that are visible in multiple oblique images, a multi-image fusion algorithm is used to fuse the texture information of the corresponding regions in multiple images. Multi-image fusion can combine the advantages of different images. For example, different images may have differences in lighting, angle, etc. By fusion, more comprehensive and clearer texture information can be obtained, which further improves the realism and detail of the model, enabling the model to better reflect the appearance characteristics of actual ground features.

[0089] By calculating the visible area of ​​each triangular facet on the tilted image, it is ensured that all triangular facets can obtain the corresponding texture information, avoiding blank or unrealistic situations on the model surface due to missing textures, and guaranteeing the integrity of texture information. When fusing texture information, the multi-image fusion algorithm comprehensively considers factors such as the sharpness and color consistency of each image, and obtains the best texture information through a reasonable fusion strategy. This helps to eliminate problems such as blurring and color deviation that may exist in a single image, improves the quality and accuracy of texture information, and makes the texture of the model surface clearer and more natural.

[0090] A 3D modeling system based on oblique photogrammetry technology includes:

[0091] The data acquisition module is used to acquire oblique image data, POS data, and ground image control data of a designated survey area based on oblique photogrammetry technology.

[0092] The aerial triangulation module is used to perform oblique photogrammetry aerial triangulation based on oblique image data, POS data and ground image control data of a specified survey area to obtain the image exterior orientation elements and ground point coordinates.

[0093] The dense matching module is used to perform dense matching of multi-view images based on the image exterior orientation elements and ground point coordinates to generate 3D point cloud data.

[0094] The TIN structure building module is used to build TIN structures based on 3D point cloud data.

[0095] The model generation module is used to generate a realistic 3D model based on the constructed TIN structure and the texture information of the oblique image.

[0096] A computing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described three-dimensional modeling methods based on oblique photogrammetry.

[0097] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described three-dimensional modeling methods based on oblique photogrammetry.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A 3D modeling method based on oblique photogrammetry, characterized in that, include: Oblique photogrammetry is used to collect oblique image data, POS data, and ground image control data for a designated survey area. Oblique photogrammetric aerial triangulation is performed based on these data to obtain image exterior orientation elements and ground point coordinates. Multi-view dense matching is then performed using the image exterior orientation elements and ground point coordinates to generate 3D point cloud data. A TIN structure is constructed based on the 3D point cloud data. Finally, a real-world 3D model is generated based on the constructed TIN structure and the texture information from the oblique imagery.

2. The three-dimensional modeling method based on oblique photogrammetry according to claim 1, characterized in that, The acquisition of oblique image data, POS data, and ground image control data for a designated survey area based on oblique photogrammetry technology specifically includes: using a flight platform equipped with an oblique photogrammetry device and a POS device to perform oblique photogrammetry data acquisition tasks on the designated survey area; simultaneously acquiring oblique image data and POS data of the designated survey area during the acquisition process; and simultaneously deploying image control points on the ground in the designated survey area and acquiring ground image control data using a full field measurement method; wherein, the oblique image data is used to record multi-angle image information of ground features in the designated survey area, the POS data is used to record the spatial position and attitude information of the flight platform during the photogrammetry, and the ground image control data is used to provide ground control references.

3. The 3D modeling method based on oblique photogrammetry according to claim 2, characterized in that, The process of performing oblique photogrammetry and aerial triangulation based on oblique image data, POS data, and ground image control data for a designated survey area to obtain image exterior orientation elements and ground point coordinates specifically includes: using the spatial position and attitude information of the flight platform recorded in the POS data to initially determine the initial exterior orientation elements of each oblique image in the oblique image data; measuring the image plane coordinates of the connection points based on the oblique image data; and performing joint adjustment calculations based on the initial exterior orientation elements, the image plane coordinates of the connection points, and the ground image control data to obtain the image exterior orientation elements and ground point coordinates within the designated survey area.

4. The 3D modeling method based on oblique photogrammetry according to claim 3, characterized in that, The step of performing joint adjustment calculations based on initial exterior orientation elements, image plane coordinates of tie points, and ground control data to obtain image exterior orientation elements and ground point coordinates within a specified survey area specifically includes: constructing an error equation containing the relationship between the image plane coordinates of tie points and ground control point coordinates, based on the initial exterior orientation elements and the collinearity condition equation, combined with the image plane coordinates of tie points and the ground control point coordinates in the ground control data; combining the error equations of all tie points to obtain a joint adjustment system; and solving the joint adjustment system using an iterative algorithm to obtain image exterior orientation elements and ground point coordinates within the specified survey area.

5. The three-dimensional modeling method based on oblique photogrammetry according to claim 4, characterized in that, The method for generating 3D point cloud data through dense matching of multi-view images based on image exterior orientation elements and ground point coordinates specifically includes: projecting feature points on different images into object space using image exterior orientation elements, and constructing geometric constraints between multi-view images by combining them with ground point coordinates; searching for corresponding feature points on multiple related images based on ground point coordinates to determine candidate corresponding point pairs between images; matching the candidate corresponding point pairs between images based on the geometric constraints between multi-view images to determine the final corresponding point pairs between images; calculating the object space 3D coordinates corresponding to each corresponding point using the principle of spatial forward intersection based on the final corresponding point pairs between images; and organizing and summarizing all the calculated 3D points to form dense 3D point cloud data.

6. The three-dimensional modeling method based on oblique photogrammetry according to claim 5, characterized in that, The construction of a TIN structure based on 3D point cloud data specifically includes: preprocessing the acquired 3D point cloud data to obtain preprocessed 3D point cloud data; using the Delaunay triangulation algorithm to perform initial triangulation on the preprocessed 3D point cloud data to construct an initial TIN structure; defining an energy function to optimize the initial TIN structure, wherein the energy function includes a shape energy term and a smoothing energy term, the shape energy term being used to measure the degree of similarity between the shape of the triangles and that of equilateral triangles, and the smoothing energy term being used to measure the degree of change in the normal vector between adjacent triangles; and adjusting the positions of the vertices in the TIN structure by minimizing the energy function to obtain the final TIN structure.

7. The three-dimensional modeling method based on oblique photogrammetry according to claim 6, characterized in that, The method of generating a realistic 3D model based on the constructed TIN structure and the texture information of the oblique image includes: calculating the visible area of ​​each triangular facet in the oblique image according to the shooting parameters of the oblique image and the geometric information of each triangular facet in the TIN structure, and establishing a texture mapping relationship between the triangular facet and the oblique image; for each triangular facet, if it has a visible area in multiple oblique images, then based on a multi-image fusion algorithm, the texture information of the corresponding areas in multiple images is fused to obtain the optimal texture information of the triangular facet; the fused texture information is mapped onto each triangular facet of the TIN structure according to the established texture mapping relationship to generate a realistic 3D model with real texture.

8. A three-dimensional modeling system based on oblique photogrammetry, characterized in that, include: The data acquisition module is used to acquire oblique image data, POS data, and ground image control data of a designated survey area based on oblique photogrammetry technology. The aerial triangulation module is used to perform oblique photogrammetry aerial triangulation based on oblique image data, POS data, and ground image control data of a specified survey area to obtain image exterior orientation elements and ground point coordinates; the dense matching module is used to perform multi-view dense matching based on image exterior orientation elements and ground point coordinates to generate 3D point cloud data; the TIN structure construction module is used to construct a TIN structure based on the 3D point cloud data; and the model generation module is used to generate a real-scene 3D model based on the constructed TIN structure and the texture information of the oblique image.

9. A computing device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the three-dimensional modeling method based on oblique photogrammetry as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the three-dimensional modeling method based on oblique photogrammetry as described in any one of claims 1-7.

Images