Aerial image reverse mapping rendering method and system

Abstract

The invention relates to the technical field of aerial image processing, and discloses an aerial image reverse mapping rendering method and system, and the method comprises the following steps: selecting an aerial target region, and taking a parameterized plane of the target region as a ground surface model of a terrain; coordinates corresponding to four vertexes of each aerial image of the target area are calculated, then geometries are created in the three-dimensional engine, and the three-dimensional vertex position of the geometry corresponding to each aerial image is obtained; and transmitting the vertex coordinates to a fragment shader of the three-dimensional engine through a GPU by using a vertex shader of the three-dimensional engine, reversely mapping the vertex coordinates to UV texture coordinates of the aerial image by using the fragment shader, and sampling the UV texture coordinates of the aerial image to obtain color values of pixels of the aerial image. The problems that in the prior art, the processing flow is heavy, the calculation efficiency is low, and the real-time mapping requirement is difficult to meet are solved.

Description

Technical Field

[0001] This invention relates to the field of aerial image processing technology, specifically to an aerial image reverse mapping rendering method and system. Background Technology

[0002] In recent years, UAV remote sensing technology has been widely applied in disaster emergency response, urban governance, and agricultural monitoring, but common technical bottlenecks still exist in real-time mapping. These technical difficulties mainly manifest in three aspects: First, there are stringent time requirements; scenarios such as disaster emergencies require the entire process to be completed in a short period of time. Secondly, computing resources are limited, and edge computing environments often face the dilemma of limited GPU resources; Third, the quality of the results must meet the standards for professional application.

[0003] In existing drone-based aerial image rendering technologies, orthophotos are often generated through "feature matching and aerial triangulation" followed by "3D geometric modeling" or "2D image stitching." This existing approach can be divided into two stages, and its drawbacks are as follows: Phase 1: Feature Matching and Aerial Triangulation The disadvantages are as follows: High computational resource dependence: Feature matching has high hardware computing power requirements; Limitations of aerial triangulation: It is difficult to obtain effective points in areas with weak texture (such as water surface), and geometric distortion occurs in the edge areas of aerial images due to the lack of matching points.

[0004] Phase Two: Aerial Image Generation and Processing This phase includes two main technical approaches: 1. Aerial image generation based on 3D geometric modeling: Construct a sparse 3D point cloud by matching feature points → Generate a surface triangular mesh model → Apply the texture of the UAV aerial image to the 3D mesh surface → Set up a virtual orthophoto camera to perform projection transformation to generate a 2D orthophoto aerial image. The disadvantages are as follows: Hole problem: In weak texture areas, the lack of feature points prevents the generation of meshes, resulting in holes in aerial images after texture mapping; Aerial image edge distortion: The matching points in the edge region of the aerial image have large geometric distortion and are therefore unusable. High resource requirements: It has high requirements for computing resources and storage space, and the completion speed is slow.

[0005] 2. Two-dimensional aerial image stitching process: Calculate the width and height of the orthophoto image to be generated → fill the orthophoto image frame with pixels according to the pixel coordinate mapping relationship; The disadvantages are as follows: Parallelization drawbacks: The native algorithm uses a sequential processing mode, which cannot be directly adapted to parallel computing architectures; Development cost issues: Parallelization requires restructuring the algorithm process, which significantly increases R&D investment.

[0006] In summary, existing technologies suffer from problems such as cumbersome processing procedures, low computational efficiency, and difficulty in meeting real-time mapping requirements. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a method and system for reverse mapping and rendering of aerial images, which solves the problems of cumbersome processing flow, low computational efficiency, and difficulty in meeting real-time image generation requirements in existing technologies.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for reverse mapping and rendering of aerial images includes the following steps: Select the target area for aerial photography and use the parametric plane of the target area as the surface model of the terrain; Calculate the coordinates of the four vertices of each aerial image of the target area, then create a geometry in the 3D engine, and combine the camera extrinsic parameters to assign the coordinates of the four vertices of each aerial image on the parametric plane to the vertices of the geometry, thus obtaining the 3D vertex positions of the geometry corresponding to each aerial image. The vertex coordinates are passed from the GPU to the fragment shader of the 3D engine using the vertex shader of the 3D engine. The fragment shader then uses the camera's intrinsic parameters, extrinsic parameters, and pixel size of the aerial image to reverse map the vertex coordinates to the UV texture coordinates of the aerial image. The color values ​​of the aerial image pixels are obtained by sampling the UV texture coordinates of the aerial image.

[0009] The beneficial effects of this invention are: This invention simplifies the Earth's surface into a planar geometric model by abandoning traditional feature matching and aerial triangulation methods, and constructs a novel processing framework using inverse mapping rendering technology. Addressing the shortcomings of existing feature matching / aerial triangulation techniques, this invention directly utilizes UAV POS data and a simplified Earth surface planar model, eliminating the need for feature matching and aerial triangulation. This fundamentally avoids problems such as high computational consumption, failure to process weak texture areas, and edge distortion. Addressing the shortcomings of existing image generation techniques, this invention does not generate 3D meshes or perform 2D stitching, but instead utilizes the GPU rendering pipeline to directly calculate each pixel of the final image through inverse mapping. This method does not produce holes and naturally supports... Parallel computing ensures high efficiency. This invention uses planar terrain fitting, and the visualization result is equivalent to projecting aerial images onto a plane. The fitted parameterized plane itself already contains real coordinate information and can be directly used for GIS base map generation. This invention utilizes the shader technology of a 3D engine to complete the reverse mapping from screen coordinates to aerial image texture coordinates. Development is simple, and it directly utilizes a 3D engine, which is inherently parallel computing. The number of pixels calculated depends on the number of pixels on the screen rather than the sum of the number of pixels in all aerial images, which has a significant advantage in reducing the amount of computation when generating aerial images with a large field of view. It does not rely on feature matching between multiple views, but processes each aerial image as an independent texture unit.

[0010] Based on the above technical solution, the present invention can be further improved as follows.

[0011] As a preferred technical solution, the method of using the vertex shader of the 3D engine to pass vertex coordinates to the fragment shader of the 3D engine via the GPU includes the following steps: Declare a varying variable to pass vertex coordinates; Assign the vertex coordinates to the varying variable; During the rasterization stage, the GPU smoothly interpolates the varying values ​​based on the position of each pixel in the aerial image on the surface of the geometry, and then inputs the interpolated vertex coordinates into the fragment shader.

[0012] The beneficial effects of adopting the above-mentioned preferred technical solution are: This makes it easy to input the interpolated, pixel-accurate coordinates into the fragment shader.

[0013] As a preferred technical solution, the method of using a fragment shader to inversely map vertex coordinates to UV texture coordinates of the aerial image based on the camera's intrinsic parameters, extrinsic parameters, and pixel dimensions of the aerial image includes the following steps: Using the camera extrinsic parameters corresponding to the aerial image, the input vertices obtained by GPU interpolation are transformed into the camera coordinate system with the camera optical center as the origin; Using the camera's intrinsic parameters, the vertex coordinates in the camera coordinate system are projected onto a two-dimensional aerial image, and the two-dimensional pixel coordinates corresponding to the vertex are calculated. Divide the two-dimensional pixel coordinates corresponding to the vertex by the width and height of the aerial image, and then normalize them to obtain the UV texture coordinates of the aerial image; the range of the UV texture coordinates of the aerial image is [0,1].

[0014] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the accurate acquisition of UV texture coordinates from aerial images through reverse mapping.

[0015] As a preferred technical solution, if the vertex coordinates in the camera coordinate system have a Z-axis value ≤ 0 or the UV texture coordinates of the obtained aerial image are not in the range of [0,1], then the vertex coordinates obtained after interpolation are discarded.

[0016] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the validity determination of vertex coordinates, resulting in more accurate rendering results.

[0017] As a preferred technical solution, the method for calculating the parameterized plane of the target region is as follows: Calculate the parameterized plane of the target region of , , , The value, using , , , The value determines the parameterized plane: in, The x-axis component represents the plane normal vector. The Y-axis component represents the plane normal vector. The Z-axis component represents the plane normal vector. Represents the plane offset constant. This represents the X-axis coordinate of a point in three-dimensional space. This represents the Y-axis coordinate of a point in three-dimensional space. It represents the Z-axis coordinate of a point in three-dimensional space.

[0018] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the spatial position and orientation of the parameterized plane through calculation parameters, thereby using the parameterized plane as a defined surface model.

[0019] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: If laser point cloud data for the target area already exists, the RANSAC method is used to fit the laser point cloud data. The fitting steps include: Three points are randomly selected from all the laser point cloud data, and these three points are used to determine a unique candidate parameterized plane; Traverse all laser point cloud data, calculate the distance of each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value of .

[0020] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parameterized plane when laser point cloud data of the target area is already available.

[0021] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: If 3D reconstruction data of the target area already exists, obtain a dense point cloud and fit it using the ransac method. The fitting steps include: Three points are randomly selected from all dense point cloud data, and these three points are used to determine a unique candidate parameterized plane; Iterate through all dense point cloud data, calculate the distance from each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value; The criteria for determining dense point clouds are: the number of points in the 3D reconstruction data is greater than a set number threshold and the distance between adjacent points is less than a set distance threshold.

[0022] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parametric plane when 3D reconstruction data of the target area is already available.

[0023] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: Manually input and adjust , , , The value of the parameterized plane has a good visual effect, and the visual effect is good when... , , , The value is the parameterized plane of the target region. , , , The value; Among them, the good parametric planar visual effect is judged by human observation of the continuity and geometric shape of ground features.

[0024] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parameterized plane when parameters are manually entered.

[0025] As a preferred technical solution, the terrain is flat terrain; where flat terrain refers to terrain where the vertical difference between the highest and lowest points of the ground is less than a set threshold.

[0026] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the selection of the specific area for flat terrain.

[0027] Based on the above technical solutions, the present invention also provides an aerial image reverse mapping rendering system.

[0028] An aerial image reverse mapping rendering system includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the aerial image reverse mapping rendering method.

[0029] Compared with the prior art, the present invention has the following advantages: (1) This invention simplifies the Earth's surface into a planar geometric model by abandoning traditional feature matching and aerial triangulation methods, and constructs a new processing framework by combining reverse mapping rendering technology; (2) In view of the shortcomings of feature matching / aerial triangulation in the prior art, the present invention does not perform feature matching and aerial triangulation, but directly uses the POS data of the UAV and a simplified ground plane model; this avoids the problems of high computing power consumption, failure to process weak texture areas and edge distortion from the root. (3) In view of the shortcomings of existing image generation technology, the present invention does not generate three-dimensional mesh or perform two-dimensional stitching, but uses the GPU rendering pipeline to directly calculate each pixel of the final image through reverse mapping; this method does not produce holes and naturally supports parallel computing, which is highly efficient. (4) The planar fitting terrain of the present invention will be visualized as if the aerial image were projected onto the plane, so that the fitted parameterized plane itself contains real coordinate information and can be directly used for GIS base map generation. (5) This invention utilizes the shader technology of the 3D engine to complete the reverse mapping from screen coordinates to aerial image texture coordinates. It is simple to develop and directly utilizes the 3D engine, which is naturally parallel computing. The number of pixels calculated depends on the number of pixels on the screen rather than the sum of the number of pixels in all aerial images. It has a huge advantage in reducing the amount of computation when generating aerial images with a large field of view. It does not rely on feature matching between multiple viewpoints, but processes each aerial image as an independent texture unit. Attached Figure Description

[0030] Figure 1 This is a schematic diagram illustrating the steps of the aerial image reverse mapping rendering method described in this invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0032] The principles and features of the present invention are described below. The embodiments given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0033] Example 1 like Figure 1 As shown, an aerial image reverse mapping rendering method includes the following steps: Select the target area for aerial photography and use the parametric plane of the target area as the surface model of the terrain; Calculate the coordinates of the four vertices of each aerial image of the target area, then create a geometry in the 3D engine, and combine the camera extrinsic parameters to assign the coordinates of the four vertices of each aerial image on the parametric plane to the vertices of the geometry, thus obtaining the 3D vertex positions of the geometry corresponding to each aerial image. The vertex coordinates are passed from the GPU to the fragment shader of the 3D engine using the vertex shader of the 3D engine. The fragment shader then uses the camera's intrinsic parameters, extrinsic parameters, and pixel size of the aerial image to reverse map the vertex coordinates to the UV texture coordinates of the aerial image. The color values ​​of the aerial image pixels are obtained by sampling the UV texture coordinates of the aerial image.

[0034] The beneficial effects of this invention are: This invention simplifies the Earth's surface into a planar geometric model by abandoning traditional feature matching and aerial triangulation methods, and constructs a novel processing framework using inverse mapping rendering technology. Addressing the shortcomings of existing feature matching / aerial triangulation techniques, this invention directly utilizes UAV POS data and a simplified Earth surface planar model, eliminating the need for feature matching and aerial triangulation. This fundamentally avoids problems such as high computational consumption, failure to process weak texture areas, and edge distortion. Addressing the shortcomings of existing image generation techniques, this invention does not generate 3D meshes or perform 2D stitching, but instead utilizes the GPU rendering pipeline to directly calculate each pixel of the final image through inverse mapping. This method does not produce holes and naturally supports... Parallel computing ensures high efficiency. This invention uses planar terrain fitting, and the visualization result is equivalent to projecting aerial images onto a plane. The fitted parameterized plane itself already contains real coordinate information and can be directly used for GIS base map generation. This invention utilizes the shader technology of a 3D engine to complete the reverse mapping from screen coordinates to aerial image texture coordinates. Development is simple, and it directly utilizes a 3D engine, which is inherently parallel computing. The number of pixels calculated depends on the number of pixels on the screen rather than the sum of the number of pixels in all aerial images, which has a significant advantage in reducing the amount of computation when generating aerial images with a large field of view. It does not rely on feature matching between multiple views, but processes each aerial image as an independent texture unit.

[0035] Based on the above technical solution, the present invention can be further improved as follows.

[0036] As a preferred technical solution, the method of using the vertex shader of the 3D engine to pass vertex coordinates to the fragment shader of the 3D engine via the GPU includes the following steps: Declare a varying variable to pass vertex coordinates; Assign the vertex coordinates to the varying variable; During the rasterization stage, the GPU smoothly interpolates the varying values ​​based on the position of each pixel in the aerial image on the surface of the geometry, and then inputs the interpolated vertex coordinates into the fragment shader.

[0037] The beneficial effects of adopting the above-mentioned preferred technical solution are: This makes it easy to input the interpolated, pixel-accurate coordinates into the fragment shader.

[0038] As a preferred technical solution, the method of using a fragment shader to inversely map vertex coordinates to UV texture coordinates of the aerial image based on the camera's intrinsic parameters, extrinsic parameters, and pixel dimensions of the aerial image includes the following steps: Using the camera extrinsic parameters corresponding to the aerial image, the input vertices obtained by GPU interpolation are transformed into the camera coordinate system with the camera optical center as the origin; Using the camera's intrinsic parameters, the vertex coordinates in the camera coordinate system are projected onto a two-dimensional aerial image, and the two-dimensional pixel coordinates corresponding to the vertex are calculated. Divide the two-dimensional pixel coordinates corresponding to the vertex by the width and height of the aerial image, and then normalize them to obtain the UV texture coordinates of the aerial image; the range of the UV texture coordinates of the aerial image is [0,1].

[0039] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the accurate acquisition of UV texture coordinates from aerial images through reverse mapping.

[0040] As a preferred technical solution, if the vertex coordinates in the camera coordinate system have a Z-axis value ≤ 0 or the UV texture coordinates of the obtained aerial image are not in the range of [0,1], then the vertex coordinates obtained after interpolation are discarded.

[0041] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the validity determination of vertex coordinates, resulting in more accurate rendering results.

[0042] As a preferred technical solution, the method for calculating the parameterized plane of the target region is as follows: Calculate the parameterized plane of the target region of , , , The value, using , , , The value determines the parameterized plane: in, The x-axis component represents the plane normal vector. The Y-axis component represents the plane normal vector. The Z-axis component represents the plane normal vector. Represents the plane offset constant. This represents the X-axis coordinate of a point in three-dimensional space. This represents the Y-axis coordinate of a point in three-dimensional space. It represents the Z-axis coordinate of a point in three-dimensional space.

[0043] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the spatial position and orientation of the parameterized plane through calculation parameters, thereby using the parameterized plane as a defined surface model.

[0044] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: If laser point cloud data for the target area already exists, the RANSAC method is used to fit the laser point cloud data. The fitting steps include: Three points are randomly selected from all the laser point cloud data, and these three points are used to determine a unique candidate parameterized plane; Traverse all laser point cloud data, calculate the distance of each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value of .

[0045] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parameterized plane when laser point cloud data of the target area is already available.

[0046] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: If 3D reconstruction data of the target area already exists, obtain a dense point cloud and fit it using the ransac method. The fitting steps include: Three points are randomly selected from all dense point cloud data, and these three points are used to determine a unique candidate parameterized plane; Iterate through all dense point cloud data, calculate the distance from each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value; The criteria for determining dense point clouds are: the number of points in the 3D reconstruction data is greater than a set number threshold and the distance between adjacent points is less than a set distance threshold.

[0047] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parametric plane when 3D reconstruction data of the target area is already available.

[0048] As a preferred technical solution, the parameterized plane of the calculation target region of , , , The value is determined by the following steps: Manually input and adjust , , , The value of the parameterized plane has a good visual effect, and the visual effect is good when... , , , The value is the parameterized plane of the target region. , , , The value; Among them, the good parametric planar visual effect is judged by human observation of the continuity and geometric shape of ground features.

[0049] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the determination of the parameterized plane when parameters are manually entered.

[0050] As a preferred technical solution, the terrain is flat terrain; where flat terrain refers to terrain where the vertical difference between the highest and lowest points of the ground is less than a set threshold.

[0051] The beneficial effects of adopting the above-mentioned preferred technical solution are: This facilitates the selection of the specific area for flat terrain.

[0052] Based on the above technical solutions, the present invention also provides an aerial image reverse mapping rendering system.

[0053] An aerial image reverse mapping rendering system includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the aerial image reverse mapping rendering method.

[0054] Example 2 like Figure 1 As shown, based on Example 1, this example provides a more detailed implementation method.

[0055] This invention proposes an innovative technical approach: By abandoning traditional feature matching and aerial triangulation methods, this paper simplifies the terrain into a planar geometric model for flat terrain features and constructs a new processing framework using inverse mapping rendering technology. Flat terrain refers to terrain where the vertical difference between the highest and lowest points is less than a set threshold; in this embodiment, the threshold is 5% of the flight altitude. Flight altitude (relative flight altitude) is a key parameter in aerial photogrammetry, referring to the vertical distance from the camera lens center to the ground reference plane, directly affecting the image scale and resolution, and can be directly measured using sensors.

[0056] The technical approach adopted in this invention differs from that of existing technologies. It avoids the shortcomings of existing technologies by circumventing rather than repairing them.

[0057] To address the shortcomings of feature matching / aerial triangulation: This invention does not perform feature matching and aerial triangulation, but directly utilizes the POS data of the UAV and a simplified ground plane model; this fundamentally avoids the problems of high computing power consumption, failure to process weak texture areas, and edge distortion. To address the shortcomings of image generation: This invention does not generate 3D meshes or perform 2D stitching, but instead utilizes the GPU rendering pipeline to directly calculate each pixel of the final image through reverse mapping; this method does not produce holes and naturally supports parallel computing, resulting in high efficiency.

[0058] The main steps of this plan are: Step A: Calculate the parametric plane of the target region The parameters are used to parameterize the surface model representing the terrain: in, The x-axis component represents the plane normal vector. The Y-axis component represents the plane normal vector. The Z-axis component represents the plane normal vector. , , Together they determine the spatial orientation of the plane. This represents the offset constant of the plane (which determines the distance between the plane and the origin of the coordinate system). This represents the X-axis coordinate of a point in three-dimensional space. This represents the Y-axis coordinate of a point in three-dimensional space. This represents the Z-axis coordinate of a point in three-dimensional space. A parameterized plane refers to a plane defined by mathematical equations (in this embodiment, it is...). A precisely defined ideal plane, in which , , , These four values ​​are its "parameters." Once the "parameters" are determined, the spatial position and orientation of the plane are uniquely determined. In this embodiment, the parameterized plane is treated as a simplified surface model.

[0059] For example: a) Given high-precision laser point cloud data for this area, fit it using the ransac method. The fitting process is an iterative optimization process, and the steps are as follows: First, three points are randomly selected from all the laser point cloud data. These three points can be used to determine a unique candidate plane. Next, all laser point cloud data are traversed, the distance from each point to the candidate plane is calculated, and the total number of points whose distance is less than a preset threshold is counted. These points are called "inliers". The above process is repeated a preset number of times, with 3 points randomly selected each time; Ultimately, the candidate plane that receives the most "interior point" support is considered the best fit, and its parameters will be adopted. b) Given the existing 3D reconstruction data of the region, obtain dense point clouds and fit them using the ransac method; the criteria for judging dense point clouds are: the number of points in the 3D reconstruction data is greater than a set number threshold and the distance between adjacent points is less than a set distance threshold.

[0060] c) The user manually inputs planar parameters and adjusts them until the visual effect is satisfactory. The satisfactory visual effect is mainly judged by observing the continuity and geometric shape of the features in the rendering result: the user adjusts the parameters until linear features (such as roads, rivers, and building edges) in the image can be smoothly and straightly pieced together without obvious misalignment, breakage, or bending.

[0061] Step B: Create the corresponding geometry in the 3D rendering engine for each aerial image; Required inputs (input to the 3D rendering engine): camera intrinsic and extrinsic parameters + pixel dimensions of aerial images; more specifically, camera extrinsic parameters are used in step B to calculate the 3D vertex positions of the geometry corresponding to each aerial image; while camera intrinsic, extrinsic, and pixel dimensions are used as parameters by the shader in step C to perform the core inverse mapping calculation. Calculate the coordinates of the four vertices of each aerial image; where the four vertices are on the parameterized plane and are the intersections of the four rays of the view frustum and the parameterized plane; Create a geometry in the 3D engine and assign the coordinates obtained in the previous step to the vertices of the geometry (each aerial image corresponds to one geometry).

[0062] Step C: 3D engine shaders implement reverse mapping rendering: C1: Vertex shader implementation: Declare a varying variable to pass vertex coordinates; The vertex coordinates are assigned to the varying variable, and then the varying variable is passed to the fragment shader. More specifically, this "passing" is done automatically by the GPU's rasterization stage: the rasterizer smoothly interpolates the varying values ​​from each vertex based on the specific position of each pixel on the surface of the geometry, and then inputs the interpolated coordinates, which are accurate to the pixel level, into the fragment shader. C2: Fragment shader implementation: Receive varying world coordinates (i.e., 3D world coordinates) from the vertex shader. Varying means that it is a varying variable, which is a concept in the GLSL language, meaning that it is calculated by the vertex shader and passed to the fragment shader, and its value is the 3D world coordinate. Receive the camera's intrinsic and extrinsic parameters; Perform inverse mapping calculation: Inversely map the 3D world coordinates to the UV texture coordinates of the aerial image; more specifically, see sub-steps 1 through 3 below: Sub-step 1: World coordinate system → Camera coordinate system: Using the camera extrinsic parameters (i.e., view matrix) corresponding to the aerial image, the input three-dimensional world coordinates, which are interpolated by the GPU, are transformed into the camera coordinate system with the camera optical center as the origin; Sub-step 2: Camera coordinate system → Pixel coordinate system: Using the camera's intrinsic parameters (focal length fx, fy and principal point coordinates cx, cy), and applying the pinhole camera model formula, the three-dimensional camera coordinates obtained in the previous step are projected onto the two-dimensional aerial image plane, and their corresponding pixel coordinates are directly calculated. Sub-step 3: Pixel coordinate system → UV texture coordinate system: Divide the pixel coordinates obtained in the previous step by the width and height of the aerial image respectively, and normalize them to obtain the final UV texture coordinates in the range of [0,1]. It should be noted that in the above process, a validity judgment is required: if the calculated camera coordinate Z value is less than or equal to 0 (i.e. the 3D point is behind the camera), or the calculated UV coordinate exceeds the range of [0,1], it means that the 3D point is not within the imaging range of this aerial image and the current calculation should be discarded. The color value of a pixel is obtained through texture sampling (i.e., sampling the UV texture coordinates of the aerial image to obtain the color value of the corresponding pixel) (referring to assigning a value to the pixel on the screen during rasterization). Suppose there are M aerial images, each with N pixels. For example, if the screen has S pixels, where S is generally less than M*N, the number of times the pixel value needs to be queried = S × a constant. This "constant" refers to the average overlap of the aerial images. Because aerial images from the front, back, and sides overlap, the same location on the ground usually exists in multiple aerial images. In the rendering process, this means that the same pixel in the final output aerial image will be covered and drawn by the geometry of multiple different source aerial images. Therefore, the shader program for that pixel will be executed multiple times, performing a query and assignment each time. This "constant" refers to the average number of overlaps.

[0063] This invention possesses parallel computing capabilities, significantly improving computational efficiency under general-purpose computing architectures. The parallel computing capability stems from its implementation in the GPU shader (step C2). The GPU provides each pixel of the final aerial image with 3D world coordinates derived from vertex interpolation. A key technical feature is that, in the fragment shader, an inverse mapping mathematical transformation is performed on these coordinates to directly calculate their precise texture coordinates (UV) on the original aerial image. This calculation process is simple and closed, requiring no information from any other pixels. Subsequent UV coordinate color sampling and texture interpolation are also automatically and efficiently performed by the GPU hardware. Therefore, the entire image generation process is decomposed into a massive number of independent computational tasks that can be executed simultaneously on the GPU cores, thus achieving efficient parallel processing.

[0064] D: Repeat steps B and C for each aerial image until all aerial data (aerial data includes aerial images, intrinsic parameters of aerial images, and extrinsic parameters of aerial images) has been processed.

[0065] This invention has the following features: 1. Using a local Cartesian coordinate system (a local Cartesian coordinate system refers to a plane rectangular coordinate system obtained by transforming the latitude and longitude coordinates of a small area of ​​the Earth's surface, such as WGS84, through projection methods such as UTM; in this coordinate system, the ground can be approximated as a plane, facilitating geometric calculations), the terrain is fitted onto a plane. The visualization result is equivalent to projecting the aerial image onto a plane (the plane parameters can be fitted using 3D reconstruction results or laser point clouds, or directly input by the user). This method ensures that the fitted parametric plane itself contains real coordinate information and can be directly used for GIS base map generation (because the parametric plane is defined in a local Cartesian coordinate system containing real coordinate information, any point on the parametric plane corresponds to a real geographic coordinate; therefore, the final rendered aerial image inherently contains geographic reference information).

[0066] 2. Using the shader technology of the 3D engine, the reverse mapping from screen coordinates to aerial image texture coordinates is completed. The advantages include: (1) simple development, directly using the 3D engine, which is naturally parallel computing; (2) the number of pixels calculated depends on the number of pixels on the screen rather than the sum of the number of pixels in all aerial images, which has a huge advantage in reducing the amount of computation when generating aerial images with a large field of view; (3) it does not rely on feature matching between multiple viewpoints, but processes each aerial image as an independent texture unit.

[0067] As described above, the present invention can be implemented well.

[0068] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0069] In the description of this invention, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0070] In the description of this invention, although embodiments of the invention have been shown and described herein, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this invention.

[0071] In the description of this invention, all features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for reverse mapping and rendering of aerial images, characterized in that, Includes the following steps: Select the target area for aerial photography and use the parametric plane of the target area as the surface model of the terrain; Calculate the coordinates of the four vertices of each aerial image of the target area, then create a geometry in the 3D engine, and combine the camera extrinsic parameters to assign the coordinates of the four vertices of each aerial image on the parametric plane to the vertices of the geometry, thus obtaining the 3D vertex positions of the geometry corresponding to each aerial image. The vertex coordinates are passed from the GPU to the fragment shader of the 3D engine using the vertex shader of the 3D engine. The fragment shader then uses the camera's intrinsic parameters, extrinsic parameters, and pixel size of the aerial image to reverse map the vertex coordinates to the UV texture coordinates of the aerial image. The color values ​​of the aerial image pixels are obtained by sampling the UV texture coordinates of the aerial image.

2. The aerial image reverse mapping rendering method according to claim 1, characterized in that, The process of using the vertex shader of the 3D engine to pass vertex coordinates to the fragment shader of the 3D engine via the GPU includes the following steps: Declare a varying variable to pass vertex coordinates; Assign the vertex coordinates to the varying variable; During the rasterization stage, the GPU smoothly interpolates the varying values ​​based on the position of each pixel in the aerial image on the surface of the geometry, and then inputs the interpolated vertex coordinates into the fragment shader.

3. The aerial image reverse mapping rendering method according to claim 2, characterized in that, The method of using a fragment shader to inversely map vertex coordinates to UV texture coordinates of the aerial image based on the camera's intrinsic parameters, extrinsic parameters, and pixel dimensions of the aerial image includes the following steps: Using the camera extrinsic parameters corresponding to the aerial image, the input vertices obtained by GPU interpolation are transformed into the camera coordinate system with the camera optical center as the origin; Using the camera's intrinsic parameters, the vertex coordinates in the camera coordinate system are projected onto a two-dimensional aerial image, and the two-dimensional pixel coordinates corresponding to the vertex are calculated. Divide the two-dimensional pixel coordinates corresponding to the vertex by the width and height of the aerial image, and then normalize them to obtain the UV texture coordinates of the aerial image; the range of the UV texture coordinates of the aerial image is [0,1].

4. The aerial image reverse mapping rendering method according to claim 3, characterized in that, If the vertex coordinates in the camera coordinate system have a Z-axis value ≤ 0 or the UV texture coordinates of the obtained aerial image are not in the range [0,1], then the vertex coordinates obtained after interpolation are discarded.

5. The aerial image reverse mapping rendering method according to claim 1, characterized in that, The method for calculating the parametric plane of the target region is as follows: Calculate the parameterized plane of the target region of , , , The value, using , , , The value determines the parameterized plane: in, The x-axis component represents the plane normal vector. The Y-axis component represents the plane normal vector. The Z-axis component represents the plane normal vector. Represents the plane offset constant. This represents the X-axis coordinate of a point in three-dimensional space. This represents the Y-axis coordinate of a point in three-dimensional space. It represents the Z-axis coordinate of a point in three-dimensional space.

6. The aerial image reverse mapping rendering method according to claim 5, characterized in that, The parameterized plane of the calculation target region of , , , The value is determined by the following steps: If laser point cloud data for the target area already exists, the RANSAC method is used to fit the laser point cloud data. The fitting steps include: Three points are randomly selected from all the laser point cloud data, and these three points are used to determine a unique candidate parameterized plane; Traverse all laser point cloud data, calculate the distance of each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value of .

7. The aerial image reverse mapping rendering method according to claim 5, characterized in that, The parameterized plane of the calculation target region of , , , The value is determined by the following steps: If 3D reconstruction data of the target area already exists, obtain a dense point cloud and fit it using the ransac method. The fitting steps include: Three points are randomly selected from all dense point cloud data, and these three points are used to determine a unique candidate parameterized plane; Iterate through all dense point cloud data, calculate the distance from each point to the candidate parameterized plane, and count the total number of points whose distance is less than a preset threshold, which is recorded as the total number of interior points; Select 3 points randomly again and repeat the above steps a preset number of times; The candidate parameterized plane with the largest number of interior points , , , The value is the parameterized plane of the target region. , , , The value; The criteria for determining dense point clouds are: the number of points in the 3D reconstruction data is greater than a set number threshold and the distance between adjacent points is less than a set distance threshold.

8. The aerial image reverse mapping rendering method according to claim 5, characterized in that, The parameterized plane of the calculation target region of , , , The value is determined by the following steps: Manually input and adjust , , , The value of the parameterized plane has a good visual effect, and the visual effect is good when... , , , The value is the parameterized plane of the target region. , , , The value; Among them, the good parametric planar visual effect is judged by human observation of the continuity and geometric shape of ground features.

9. A method for reverse mapping and rendering of aerial images according to any one of claims 1 to 8, characterized in that, The terrain is flat; flat terrain refers to terrain where the vertical difference between the highest and lowest points of the ground is less than a set threshold.

10. A system for reverse mapping and rendering of aerial images, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the aerial image reverse mapping rendering method according to any one of claims 1 to 9.

Images