Point cloud-based geographic environment three-dimensional model construction method and system
Through multi-source data fusion and advanced image processing technology, a high-precision three-dimensional model is generated, which solves the problem of contradiction between spatial resolution and temporal resolution of remote sensing image data, and realizes rapid and large-scale reconstruction of terrain environment data , suitable for a variety of application scenarios.
Patent Information
- Application Number
- CN202411193221.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-26
AI Technical Summary
The prior art is difficult to take into account both the spatial resolution and the temporal resolution of remote sensing image data at the same time, resulting in the inability to meet the large-scale, high-precision, and rapidly changing ground information remote sensing monitoring needs.
Using a three-dimensional model construction method of geographic environment based on point cloud, through the fusion of multi-source data of drone image data, satellite remote sensing data and high-precision point cloud data, image enhancement algorithm, edge detection algorithm, convolutional neural network, and feature extraction and matching algorithms, point cloud dilution algorithms, asymmetry TIN encryption filtering algorithms and 3D texture mapping technology to generate high-precision near-real-time three-dimensional models.
It realizes rapid, large-scale and near-real-time terrain environment data generation, reduces manpower and time costs, is suitable for rapid reconstruction of the geographical environment, and has a wide range of application prospects.
Smart Images

Figure CN119027606A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of three-dimensional reconstruction of geographic environment, and in particular to a method and system for constructing a three-dimensional model of a geographic environment based on point cloud. Background Art
[0002] With the rapid development of satellite remote sensing technology, mankind has entered a new era of multi-layer, three-dimensional, multi-angle, all-round and all-weather earth observation. Satellite remote sensing has gradually integrated into people's daily work and life, providing people with a large amount of scientific data and dynamic information at all times. However, due to technical limitations, it is usually difficult for sensors to take into account both the spatial resolution and temporal resolution of the data when acquiring ground data, resulting in irreconcilable contradictions in the spatial resolution and temporal resolution of remote sensing image data. Single remote sensing data cannot meet the needs of large-scale, high-precision, and rapidly changing ground information remote sensing monitoring.
[0003] Therefore, how to generate a high-precision, near-real-time stereo model based on the existing data in the mission area through multi-source data fusion processing and analysis, quickly identify changes in the mission area, and accurately reconstruct the geographic environment has become a pain point in the application of geospatial data in the industry. Summary of the invention
[0004] The purpose of the present invention is to address the problems existing in the prior art and propose a method and system for constructing a three-dimensional model of a geographic environment based on point cloud. The method and system are based on near-real-time drone image data, large-scale satellite remote sensing data and high-precision point cloud data, have high computational efficiency, can greatly reduce manpower and time costs, and provide a new way to generate fast, large-scale and near-real-time terrain environment data.
[0005] To achieve the above purpose, the technical solution adopted is:
[0006] A method for constructing a three-dimensional model of a geographic environment based on point cloud, comprising:
[0007] Obtain multi-source environmental data of the same study area, including point cloud data and drone image data obtained by drones, and remote sensing image data obtained by remote sensing platforms;
[0008] Use image enhancement algorithms to preprocess UAV image data, and use edge detection algorithms to preprocess remote sensing image data;
[0009] The building area is extracted from the remote sensing image based on the convolutional neural network, which is used as a mask for cropping the building area of the drone;
[0010] Extract the building area from the drone image and merge it with the remote sensing image through feature extraction and matching algorithms;
[0011] A point cloud thinning algorithm that takes terrain features into consideration is used to filter and simplify point cloud data, removing redundant and irrelevant data while keeping terrain features unchanged;
[0012] Based on the progressive TIN encryption filtering algorithm, a digital surface model DSM of the study area is constructed to generate the height, slope and direction information of the terrain;
[0013] Using 3D texture mapping technology, a high-precision three-dimensional model for rapid reconstruction of the geographic environment is generated based on the fused images and processed DSM data.
[0014] According to the method for constructing a three-dimensional model of a geographic environment based on point cloud of the present invention, further, the image enhancement algorithm adopts histogram equalization, and the process of preprocessing the drone image data by using histogram equalization is as follows: firstly, the histogram of the original image is counted, and then the cumulative distribution function CDF is calculated, and then the pixel value after equalization is calculated, and finally the pixel value in the original image is replaced by the pixel value after equalization;
[0015] The edge detection algorithm uses the Sobel operator. The preprocessing process of remote sensing image data using the Sobel operator is as follows: The Sobel operator contains two sets of 3x3 matrices, namely the horizontal matrix and vertical matrix By performing a planar convolution with the original image, we can obtain the approximate horizontal and vertical brightness differences respectively.
[0016] According to the method for constructing a three-dimensional model of a geographic environment based on point cloud of the present invention, the building area of the drone image and the remote sensing image are further spliced and fused by feature extraction and matching algorithms, and the process is as follows:
[0017] ① Using SIFT algorithm, extract the feature points and feature descriptors of the building area of the drone image and the preprocessed remote sensing image. Use brute force matcher to match the two adjacent images to generate a set of feature point pairs.
[0018] ②RANSAC screens reliable point pairs;
[0019] ③SVD decomposition to find the minimum norm least squares solution and solve the homography matrix H of the two images 12 ;
[0020] ④Convert all images to the same pixel plane and perform image stitching.
[0021] According to the method for constructing a three-dimensional model of a geographical environment based on point cloud of the present invention, further, a point cloud thinning algorithm taking into account terrain features is used to screen and simplify point cloud data, including:
[0022] ① First, take k neighboring points of any point pi in the point cloud to fit the quadratic surface z(x,y), 1≤i≤n, n is the number of point clouds, and then calculate the average curvature of pi based on the first and second basic quantities of the quadratic surface; the quadratic surface equation is expressed as follows:
[0023] z(x,y)=ax 2 +bxy+cy 2 +dx+ey+f
[0024] Among them, a, b, c, d, e, f are equation coefficients, and x and y represent surface coordinates;
[0025] ② Establish spatial index for the original point cloud data and use three-dimensional KD-tree to implement k-neighborhood search;
[0026] ③ Fitting of local quadratic surface: The least square method is used to fit the local quadratic surface by the data point pi and its k neighboring points, and the objective function is established as:
[0027]
[0028] Among them, x i ,y i represents the coordinates of a point on the quadratic surface, z(x i ,y i ) is the quadratic surface prediction value, z i is the actual value of the quadratic surface;
[0029] ④ Calculation of curvature: the average curvature H of the local surface is obtained through the first and second basic quantities of the quadratic surface;
[0030] ⑤ Fit the local quadratic surface from the target point pi and its k neighboring points, calculate the average curvature Hi, and traverse all point clouds;
[0031] ⑥ Sort the point cloud according to the average curvature from small to large. According to the thinning rate, set the flag flag = 1 for the points with the highest average curvature, that is, the non-feature points to be thinned out. The other points are set to 0 by default.
[0032] ⑦ Traverse the marked points and perform a neighborhood search with a radius of R. If the flags of the searched neighborhood points are all 1, the point is retained and the flag is set to 0. Otherwise, the point is thinned out. The radius R represents the upper limit of the point cloud hole range allowed to appear in the thinning area.
[0033] According to the method for constructing a three-dimensional model of a geographic environment based on point cloud of the present invention, further, based on a progressive TIN encryption filtering algorithm, constructing a digital surface model DSM of a study area comprises:
[0034] ①First calculate the number of grids
[0035] Calculate the extreme value of the point cloud (x, y) coordinates, and use the extreme value of the (x, y) coordinates to get the point cloud width w and length h. Then the number of rows and columns of the grid is:
[0036]
[0037] Among them, n row Indicates the number of grid rows, n column represents the number of grid columns, m represents the grid size, and ceil(·) represents the rounding function when calculating rows and columns;
[0038] ②Select seed point
[0039] The lowest point in each grid and the four extreme points of x,y min , X max , Y min , Y max As seed points; mark all seed points as ground points and the rest as non-ground points;
[0040] ③ In each iteration, the point-by-point judgment is performed to determine whether it is a ground point based on the threshold;
[0041] ④ At the end of each iteration, the newly found ground points are added to the TIN model.
[0042] According to the method for constructing a three-dimensional model of a geographic environment based on point cloud of the present invention, further, at the end of each iteration, the newly found ground points are added to the TIN model in the following manner:
[0043] ①Determine the ground point P ground The triangle T' triange ;
[0044] ② Calculate T' uriange The length of each side after projection on the horizontal plane. If the length of any side projection is greater than the threshold 1, the ground point is added to the TIN model, otherwise it is not added;
[0045] ③ Repeat the iterative process until no more points are added to the ground point set.
[0046] According to the method for constructing a three-dimensional model of a geographic environment based on point cloud of the present invention, further, using 3D texture mapping technology to generate a high-precision three-dimensional model for rapid reconstruction of the geographic environment comprises:
[0047] ① First, comprehensively evaluate the image quality associated with each facet in the 3D model from three aspects: occlusion, projection area, and color consistency;
[0048] ②Then the texture view selection problem of all patches is transformed into a global energy optimization problem based on Markov random fields, and the top N best images are found as candidate texture maps for each patch through the cyclic belief propagation algorithm;
[0049] ③Finally, a multi-view fusion algorithm based on a distance weight map is used to fuse all the candidate texture maps to obtain the final 3D model texture map.
[0050] Furthermore, the present invention also provides a system for constructing a three-dimensional model of a geographic environment based on point cloud, comprising:
[0051] The data acquisition module is used to obtain multi-source environmental data of the same research area, including point cloud data and drone image data obtained by drones, and remote sensing image data obtained by remote sensing platforms;
[0052] The preprocessing module is used to preprocess the UAV image data using the image enhancement algorithm and to preprocess the remote sensing image data using the edge detection algorithm;
[0053] The building recognition module is used to extract the building area from the remote sensing image based on the convolutional neural network, which is used as a mask for cropping the building area of the drone;
[0054] The stitching and fusion module is used to extract the building area from the drone image and stitch it with the remote sensing image through feature extraction and matching algorithms;
[0055] Point cloud thinning module, which is used to filter and simplify point cloud data using a point cloud thinning algorithm that takes terrain features into account, removing redundant and irrelevant data while keeping terrain features unchanged;
[0056] DSM construction module, which is used to construct the digital surface model DSM of the study area based on the progressive TIN encryption filtering algorithm and generate the height, slope and direction information of the terrain;
[0057] The 3D model reconstruction module is used to generate a high-precision 3D model for rapid reconstruction of the geographic environment based on the fused images and processed DSM data using 3D texture mapping technology.
[0058] The beneficial effects achieved by adopting the above technical solution are:
[0059] 1. The present invention can process drone image data and remote sensing image data quickly and effectively, and uses feature extraction and matching algorithms to splice and fuse the building area of drone images and remote sensing images; considering the complexity and redundancy of obtaining point clouds, a point cloud thinning algorithm that takes terrain features into account is used to screen point cloud data, while keeping the terrain features unchanged and reducing redundant data, and then based on the progressive TIN encryption filtering algorithm, a digital surface model (DSM) of the study area is constructed; 3D texture mapping technology is used to generate a high-precision three-dimensional model for rapid reconstruction of the geographical environment based on the fused image and processed DSM data. This method can quickly obtain high-precision three-dimensional models over a large range in near real time, and has broad application prospects in social and economic production, emergency disaster relief, urban planning and design, etc.
[0060] 2. The present invention has the advantages of simple method, high construction accuracy, high calculation efficiency, and the ability to greatly reduce manpower and time costs, and is suitable for the study of rapid reconstruction of geographical environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.
[0062] Figure 1 This is one of the flow charts of the method for constructing a three-dimensional model of a geographic environment based on point cloud according to an embodiment of the present invention;
[0063] Figure 2 This is the second flow chart of the method for constructing a three-dimensional model of a geographic environment based on point cloud according to an embodiment of the present invention. DETAILED DESCRIPTION
[0064] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.
[0065] In order to better understand the present invention, the various terms involved are explained below.
[0066] Landsat8 data: Landsat8 is the eighth satellite of the US Landsat Program (Landsat). The Landsat-8 satellite carries two sensors, namely the OLI Land Imager (Operational Land Imager) and the TIRS Thermal Infrared Sensor (Tapered Infrared Sensor).
[0067] DSM: Digital Surface Model (DSM) refers to a ground elevation model that includes the heights of surface buildings, bridges, trees, etc.
[0068] Point cloud data: Point cloud data refers to a set of vectors in a three-dimensional coordinate system. Each point in the set represents a set of x, y, z geometric coordinates and a vector value. This vector value can be expressed as different signal strengths depending on the application scenario. When these points are combined together, a point cloud is formed, which is a collection of data points in space that represents a 3D shape or object.
[0069] Point cloud thinning: Point cloud thinning is a method of simplifying point cloud data. Its main purpose is to reduce the number of points while retaining important features and shape information in the point cloud data as much as possible.
[0070] This embodiment discloses a method for constructing a three-dimensional model of a geographic environment based on point cloud. Figure 1 and Figure 2 As shown, the following steps are included:
[0071] Step S101: Acquire multi-source environmental data of the same study area, including point cloud data and drone image data acquired through drones, and remote sensing image data acquired through remote sensing platforms.
[0072] Download and purchase satellite image data and laser point cloud data of the research area. Use drone 3D oblique photogrammetry technology to take near-real-time drone images of the research area. Perform pre-processing operations such as splicing, cropping, and correction on the relevant data to form experimental sample data.
[0073] The sensors or cameras carried by the drones take photos of the study area to obtain high-resolution images or point cloud data. In this step, the parameters such as the flight altitude, angle, speed, and sensor settings of the drone will affect the quality of the final data. The remote sensing image data in this example uses Landsat8 remote sensing image data.
[0074] Step S102: pre-process the drone image data using an image enhancement algorithm, and pre-process the remote sensing image data using an edge detection algorithm.
[0075] The acquired data is preprocessed, including format conversion, noise removal, coordinate conversion, etc., to ensure the accuracy and consistency of the data and improve the efficiency of subsequent processing.
[0076] This embodiment uses an image enhancement algorithm to process drone images, and enhances the image's contrast, brightness and other features by adjusting the image's grayscale value or color value, making the image clearer, brighter and easier to analyze. The algorithm uses histogram equalization, and the specific processing steps are as follows:
[0077] ① Count the histogram of the original image. Count the number of times each pixel value appears, and then normalize them to get the frequency of each pixel value.
[0078] ②Then calculate the cumulative distribution function (CDF). CDF is the integral of the frequency distribution function (PDF), which represents the probability of each pixel value appearing in the original image. CDF can be obtained by accumulating PDF. For a gray value i, the calculation formula of CDF is as follows:
[0079]
[0080] Among them, P(i) represents the frequency of occurrence of pixels with gray value i in the image.
[0081] ③ Calculate the equalized pixel value. Map each pixel value in the original image to a new pixel value so that the equalized histogram approximates a uniformly distributed histogram. This mapping function can be expressed by the following formula:
[0082]
[0083] Among them, H(i) represents the pixel value after equalization, M and N represent the width and height of the image respectively, L represents the range of pixel values, and min represents the minimum pixel value in the original image.
[0084] ④Replace the pixel values in the original image with the equalized pixel values, thus completing the histogram equalization process.
[0085] The original remote sensing image data is obtained through satellites, and the obtained original remote sensing images are preprocessed, including radiation correction, geometric correction and atmospheric correction, etc. The purpose of these corrections is to eliminate noise and errors in the images and improve the quality and accuracy of the data.
[0086] The image preprocessing algorithm of this embodiment is an edge detection algorithm, which uses the Sobel operator to highlight the edge features of the image to facilitate the subsequent identification of building areas. The operator contains two groups of 3x3 matrices, namely the horizontal matrix and vertical matrix By performing a planar convolution with the original image, we can obtain the approximate horizontal and vertical brightness difference values respectively. The formulas are as follows:
[0087]
[0088] Wherein, A represents the original image, Gx and Gy represent the grayscale values of the image after horizontal and vertical edge detection respectively.
[0089] Multiple remote sensing images are registered to align them in the same coordinate system. This can be achieved through image processing techniques to ensure accuracy and consistency in subsequent analysis.
[0090] Step S103: extracting the building area from the remote sensing image based on the convolutional neural network, and using it as a mask for cropping the building area of the drone.
[0091] A machine learning-based building recognition algorithm is used to extract feature information related to the target object from remote sensing images. The convolutional neural network (CNN) algorithm is used to extract building areas, and the shape and structure of buildings are intelligently identified from satellite remote sensing images to form satellite remote sensing image mask data. The specific formula of the CNN convolutional neural network is:
[0092] Y=f(W*X+B)
[0093] Where W represents the weight matrix, X represents the input value, B represents the offset, represents the activation function, and Y represents the output value.
[0094] Collect a dataset containing images of buildings. These images should be high-resolution so that buildings can be clearly identified. The dataset should contain various building types, angles, and backgrounds. Preprocessing includes scaling, normalizing, and data augmentation to improve the generalization ability of the model. Build a CNN model. Use the common ResNet model. When building the model, you also need to determine the size of the input image as 256×256, as well as parameters such as the size and number of convolution kernels. After building the model, use the training data to train the model. During the training process, the parameters of the model are constantly adjusted to minimize the error between the predicted results and the true results. After the model is trained, use the test data to evaluate the performance of the model. Evaluation indicators include accuracy, precision, recall, etc. By adjusting the parameters of the model, the performance of the model can be further improved. Finally, post-process the output of the model and perform operations such as binarization and edge detection on the extracted building areas to better identify the outline of the building.
[0095] Step S104: extract the building area from the drone image, and merge it with the remote sensing image through feature extraction and matching algorithms.
[0096] Based on the satellite remote sensing image mask data, the drone images of the study area are cropped to obtain high-precision drone images of the building area. The image stitching method based on feature points is used to match and stitch the same-name points in the drone images and satellite remote sensing images to generate high-precision, near-real-time texture image data.
[0097] The building area identified by the remote sensing image data is used as the potential change area to extract the building area of the drone image, and then the remote sensing image and the building area of the high-resolution drone image are spliced and fused according to the feature extraction and matching algorithm. This embodiment adopts the image splicing algorithm based on SIFT, which has high feature matching accuracy and good adaptability to image deformation, brightness change and noise. It is a commonly used method for image splicing. The process is as follows:
[0098] ① Using SIFT algorithm, extract the feature points and feature descriptors of the building area of the drone image and the preprocessed remote sensing image. Use brute force matcher to match the two adjacent images to generate a set of feature point pairs.
[0099] ②RANSAC screens reliable point pairs;
[0100] ③SVD decomposition to find the minimum norm least squares solution and solve the homography matrix H of the two images 12 ;
[0101] ④Convert all images to the same pixel plane and perform image stitching.
[0102] Step S105: Use a point cloud thinning algorithm that takes terrain features into consideration to screen and simplify the point cloud data, remove redundant and irrelevant data while keeping the terrain features unchanged, and the amount of point cloud data after thinning is greatly reduced, thereby improving data processing efficiency.
[0103] The point cloud data thinning algorithm is used to generate a high-precision digital surface model (DSM) based on high-density point cloud data to better reflect the landform and terrain characteristics of the study area. The calculation process of the point cloud thinning algorithm is as follows:
[0104] ① First, take k neighboring points of any point pi (1≤i≤n, n is the number of point clouds) in the point cloud to fit the quadratic surface z(x,y), and then calculate the average curvature of pi based on the first basic quantity and the second basic quantity of the quadratic surface; the quadratic surface equation is expressed as follows:
[0105] z(x,y)=ax 2 +bxy+cy 2 +dx+ey+f
[0106] Among them, a, b, c, d, e, f are the coefficients of the equation, and x and y represent the surface coordinates.
[0107] ② In order to quickly and efficiently obtain the neighborhood information of each point in massive point cloud data, it is necessary to establish a spatial index for the original point cloud data and use a three-dimensional KD-tree to implement k-neighborhood search.
[0108] ③ Fitting of local quadratic surface: The least square method is used to fit the local quadratic surface by the data point pi and its k neighboring points, and the objective function is established as:
[0109]
[0110] Among them, x i ,y i represents the coordinates of a point on the quadratic surface, z(x i ,y i ) is the quadratic surface prediction value, z i is the actual value of the quadratic surface.
[0111] ④ Calculation of curvature: the average curvature H of the local surface is obtained through the first and second basic quantities of the quadratic surface.
[0112] ⑤ Fit the local quadratic surface by the target point pi and its k neighborhood points, calculate the average curvature Hi, and traverse all point clouds.
[0113] ⑥ Sort the point cloud according to the average curvature from small to large. According to the thinning rate, set the flag flag = 1 for the points with the highest average curvature, that is, the non-feature points to be thinned out. The default value of other points is 0.
[0114] ⑦ Traverse the marked points and perform a neighborhood search with a radius of R. If the flags of the searched neighborhood points are all 1, the point is retained and the flag is set to 0. Otherwise, the point is thinned out. The radius R represents the upper limit of the point cloud hole range allowed to appear in the thinning area.
[0115] Step S106: Based on the progressive TIN encryption filtering algorithm, a digital surface model DSM of the study area is constructed to generate the height, slope and direction information of the terrain, providing basic data for subsequent three-dimensional model generation.
[0116] A variety of filtering algorithms can be used to smooth the point cloud data to improve the accuracy and smoothness of the DSM. This embodiment uses a progressive TIN encryption filtering algorithm. The specific calculation process is as follows:
[0117] ①First calculate the number of grids
[0118] Calculate the extreme values of the x and y coordinates of the point cloud, and use the extreme values of the x and y coordinates to get the width w and length h of the point cloud. The number of rows and columns of the grid is:
[0119]
[0120] Among them, n row Indicates the number of grid rows, n column represents the number of grid columns, m represents the grid size, and ceil(·) represents the rounding function when calculating rows and columns.
[0121] ②Select seed point
[0122] The lowest point in each grid and the four extreme points of x,y min , X max , Y min , Y max As seed points; mark all seed points as ground points and the rest as non-ground points.
[0123] ③ In each iteration, the point-by-point judgment is performed to determine whether it is a ground point based on the threshold. The specific judgment is as follows:
[0124] (1) Positioning point cloud P potential (x P ,y P , z p ) in the triangle T triangle : Find the triangle where point P is located, that is, P is located inside, on the boundary or at the vertex of the triangle.
[0125] (2) Calculate the slope Striangle of the triangular plane: If Striangle is not greater than the terrain angle parameter t, go to (3), otherwise go to (4).
[0126] (3) Calculate two important parameters: find the nearest triangle vertex of P to form a straight line, and calculate the distance between the straight line and triangle T triangle The line-surface angle A angle . Then calculate P to T triangle The distance is denoted as D distance If A angle Not greater than parameter d, and D distance If it is not greater than parameter d, the point is considered to be a ground point, and the judgment of this point ends and the judgment of the next point is carried out.
[0127] (4) Construct P potential Mirror point of triangle T: Find triangle The highest vertex, denoted as P vertex (c v ,y u , z w ), then P potential The mirror point of is represented as:
[0128] x mirror =2x v -xP
[0129] y mirror =2y v -y P
[0130] x mirror =z P
[0131] (x mirror ,y mirror , z nirror ) represents the three-dimensional coordinates of the mirror point. As in step (3), calculate the angle and distance of the mirror point. If the threshold condition in (3) is met, then P potential It is a ground point and the next point is determined.
[0132] ④ At the end of each iteration, the newly found ground points are added to the TIN model in the following way.
[0133] a. Determine the ground point P ground The triangle T' triange .
[0134] b. Calculate T' uriange The length of each side after projection on the horizontal plane. If the projection length of any side is greater than the threshold 1, the ground point is added to the TIN model, otherwise it is not added.
[0135] c. Repeat the iterative process until no more points are added to the ground point set.
[0136] Step S107: Using 3D texture mapping technology, based on the fused image and processed DSM data, a high-precision three-dimensional model for rapid reconstruction of the geographic environment is generated.
[0137] Using 3D texture mapping technology, high-precision DSM and texture images are fused at the pixel level to generate a high-precision rapid reconstruction model of the geographical environment of the study area, thereby improving the spatiotemporal accuracy and visual effects of the geographical environment of the study area.
[0138] The texture coordinate system is a two-dimensional coordinate system, in which the coordinate points represent the pixel positions in the texture data. The texture coordinate system of a three-dimensional model is a three-dimensional coordinate system, in which the coordinate points represent the positions on the surface of the model. When mapping texture data, you first need to match the texture data with the three-dimensional model. Usually, the texture data is a rectangular image, and the surface of the three-dimensional model may be a curved surface or a complex geometric body. Therefore, when mapping texture data to a three-dimensional model, it is necessary to transform and map the texture coordinates. The specific steps are:
[0139] ①First, comprehensively evaluate the image quality associated with each facet in the 3D model from three aspects: occlusion, projection area and color consistency.
[0140] ②Then the texture view selection problem of all patches is transformed into a global energy optimization problem based on Markov random fields, and the top N best images are found as candidate texture maps for each patch through the loopy belief propagation algorithm.
[0141] ③Finally, a multi-view fusion algorithm based on a distance weight map is used to fuse all the candidate texture maps to obtain the final 3D model texture map.
[0142] The present invention utilizes large-scale remote sensing image data, point cloud data and drone image data to establish a high-precision three-dimensional model. The method has high computational efficiency and is simple in method. It can quickly establish a high-precision three-dimensional model over a large area, and the model reconstruction effect is good.
[0143] Corresponding to the above method, this embodiment also discloses a system for constructing a three-dimensional model of a geographic environment based on point cloud, comprising:
[0144] The data acquisition module is used to obtain multi-source environmental data of the same study area, including point cloud data and drone image data obtained by drones, and remote sensing image data obtained by remote sensing platforms.
[0145] The preprocessing module is used to preprocess the UAV image data using the image enhancement algorithm and to preprocess the remote sensing image data using the edge detection algorithm.
[0146] The building recognition module is used to extract building areas from remote sensing images based on convolutional neural networks, which serves as a mask for cropping drone building areas.
[0147] The stitching and fusion module is used to extract the building area from the drone image and stitch it with the remote sensing image through feature extraction and matching algorithms.
[0148] The point cloud thinning module is used to filter and simplify point cloud data using a point cloud thinning algorithm that takes terrain features into account, removing redundant and irrelevant data while keeping the terrain features unchanged.
[0149] The DSM building module is used to build a digital surface model DSM of the study area based on the progressive TIN encryption filtering algorithm to generate the height, slope and direction information of the terrain.
[0150] The 3D model reconstruction module is used to generate a high-precision 3D model for rapid reconstruction of the geographic environment based on the fused images and processed DSM data using 3D texture mapping technology.
[0151] Unless otherwise specifically stated, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
[0152] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0153] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.
[0154] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.
[0155] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A method for constructing a three-dimensional model of a geographic environment based on point cloud, characterized in that: Include: Obtain multi-source environmental data of the same study area, including point cloud data and drone image data obtained by drones, and remote sensing image data obtained by remote sensing platforms; Use image enhancement algorithms to preprocess UAV image data, and use edge detection algorithms to preprocess remote sensing image data; The building area is extracted from the remote sensing image based on the convolutional neural network, which is used as a mask for cropping the building area of the drone; Extract the building area from the drone image and merge it with the remote sensing image through feature extraction and matching algorithms; A point cloud thinning algorithm that takes terrain features into consideration is used to filter and simplify point cloud data, removing redundant and irrelevant data while keeping terrain features unchanged; Based on the progressive TIN encryption filtering algorithm, a digital surface model DSM of the study area is constructed to generate the height, slope and direction information of the terrain; Using 3D texture mapping technology, a high-precision three-dimensional model for rapid reconstruction of the geographic environment is generated based on the fused images and processed DSM data.
2. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 1, characterized in that: The image enhancement algorithm uses histogram equalization. The preprocessing process of drone image data using histogram equalization is as follows: first, the histogram of the original image is counted, then the cumulative distribution function CDF is calculated, and then the pixel value after equalization is calculated. Finally, the pixel value in the original image is replaced with the pixel value after equalization. The edge detection algorithm uses the Sobel operator. The preprocessing process of remote sensing image data using the Sobel operator is as follows: The Sobel operator contains two sets of 3x3 matrices, namely the horizontal matrix and vertical matrix By performing a planar convolution with the original image, we can obtain the approximate horizontal and vertical brightness differences respectively.
3. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 1, characterized in that: The building area of the drone image and the remote sensing image are spliced and fused through feature extraction and matching algorithms. The process is as follows: ① Using SIFT algorithm, extract the feature points and feature descriptors of the building area of the drone image and the preprocessed remote sensing image. Use brute force matcher to match the two adjacent images to generate a set of feature point pairs. ②RANSAC screens reliable point pairs; ③SVD decomposition to find the minimum norm least squares solution and solve the homography matrix H of the two images 12 ; ④Convert all images to the same pixel plane and perform image stitching.
4. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 1, characterized in that: The point cloud data is filtered and simplified using a point cloud thinning algorithm that takes terrain features into account, including: ① First, take k neighboring points of any point pi in the point cloud to fit the quadratic surface z(x,y), 1≤i≤n, n is the number of point clouds, and then calculate the average curvature of pi based on the first and second basic quantities of the quadratic surface; the quadratic surface equation is expressed as follows: z(x,y)=ax 2 +bxy+cy 2 +dx+ey+f Among them, a, b, c, d, e, f are equation coefficients, and x and y represent surface coordinates; ② Establish spatial index for the original point cloud data and use three-dimensional KD-tree to implement k-neighborhood search; ③ Fitting of local quadratic surface: The least square method is used to fit the local quadratic surface by the data point pi and its k neighboring points, and the objective function is established as: Among them, x i ,y i represents the coordinates of a point on the quadratic surface, z(x i ,y i ) is the quadratic surface prediction value, z i is the actual value of the quadratic surface; ④ Calculation of curvature: the average curvature H of the local surface is obtained through the first and second basic quantities of the quadratic surface; ⑤ Fit the local quadratic surface from the target point pi and its k neighboring points, calculate the average curvature Hi, and traverse all point clouds; ⑥ Sort the point cloud according to the average curvature from small to large. According to the thinning rate, set the flag flag = 1 for the points with the highest average curvature, that is, the non-feature points to be thinned out. The other points are set to 0 by default. ⑦ Traverse the marked points and perform a neighborhood search with a radius of R. If the flags of the searched neighborhood points are all 1, the point is retained and the flag is set to 0. Otherwise, the point is thinned out. The radius R represents the upper limit of the point cloud hole range allowed to appear in the thinning area.
5. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 1, characterized in that: Based on the progressive TIN encryption filtering algorithm, the digital surface model DSM of the study area is constructed, which includes: ①First calculate the number of grids Calculate the extreme value of the point cloud (x, y) coordinates, and use the extreme value of the (x, y) coordinates to get the point cloud width w and length h. Then the number of rows and columns of the grid is: Among them, n row Indicates the number of grid rows, n column represents the number of grid columns, m represents the grid size, and ceil(·) represents the rounding function when calculating rows and columns; ②Select seed point The lowest point in each grid and the four extreme points of x,y min , X max , Y min , Y max As seed points; mark all seed points as ground points and the rest as non-ground points; ③ In each iteration, the point-by-point judgment is performed to determine whether it is a ground point based on the threshold; ④ At the end of each iteration, the newly found ground points are added to the TIN model.
6. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 5, characterized in that: At the end of each iteration, the newly found ground points are added to the TIN model in the following way: ①Determine the ground point P ground The triangle T' triange ; ② Calculate T' uriange The length of each side after projection on the horizontal plane. If the length of any side projection is greater than the threshold 1, the ground point is added to the TIN model, otherwise it is not added; ③ Repeat the iterative process until no more points are added to the ground point set.
7. The method for constructing a three-dimensional model of a geographical environment based on point cloud according to claim 1, characterized in that: The use of 3D texture mapping technology to generate high-precision 3D models for rapid reconstruction of geographic environments includes: ① First, comprehensively evaluate the image quality associated with each facet in the 3D model from three aspects: occlusion, projection area, and color consistency; ②Then the texture view selection problem of all patches is transformed into a global energy optimization problem based on Markov random fields, and the top N best images are found as candidate texture maps for each patch through the cyclic belief propagation algorithm; ③Finally, a multi-view fusion algorithm based on a distance weight map is used to fuse all the candidate texture maps to obtain the final 3D model texture map.
8. A system for constructing a three-dimensional model of a geographic environment based on point cloud, characterized in that: Include: The data acquisition module is used to obtain multi-source environmental data of the same research area, including point cloud data and drone image data obtained by drones, and remote sensing image data obtained by remote sensing platforms; The preprocessing module is used to preprocess the UAV image data using the image enhancement algorithm and to preprocess the remote sensing image data using the edge detection algorithm; The building recognition module is used to extract the building area from the remote sensing image based on the convolutional neural network, which is used as a mask for cropping the building area of the drone; The stitching and fusion module is used to extract the building area from the drone image and stitch it with the remote sensing image through feature extraction and matching algorithms; Point cloud thinning module, which is used to filter and simplify point cloud data using a point cloud thinning algorithm that takes terrain features into account, removing redundant and irrelevant data while keeping terrain features unchanged; DSM construction module, which is used to construct the digital surface model DSM of the study area based on the progressive TIN encryption filtering algorithm and generate the height, slope and direction information of the terrain; The 3D model reconstruction module is used to generate a high-precision 3D model for rapid reconstruction of the geographic environment based on the fused images and processed DSM data using 3D texture mapping technology.
9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Board identification method based on geometric figure range discrimination
CN119991640A
DSM (Digital Subscriber Model) construction method based on consideration of topographic features
CN121527330A
