A geographic information data preprocessing method and system
By adopting a GPU parallel computing framework and domestic optimization in the geographic information preprocessing system, the problems of insufficient lighting realism and processing performance were solved, realizing the automated conversion of high-quality data and efficient adaptation to domestic hardware.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LINGMEIXIN (BEIJING) TECH CO LTD
- Filing Date
- 2026-03-09
- Publication Date
- 2026-06-12
AI Technical Summary
Existing geographic information preprocessing systems are inadequate in terms of lighting realism, processing performance, and the degree of automation in data fusion. In particular, they cannot fully utilize their performance in a domestically produced environment and require a large amount of manual intervention.
The GPU computing framework is used to execute tile cutting, resampling, normal calculation and texture compression tasks in parallel. Through MUSA or CUDA architecture optimization, data is transferred to GPU memory block by block and encapsulated into data packets supported by OSG engine to achieve end-to-end adaptation and optimization.
It enhances the realism of lighting, eliminates lighting discontinuity issues, improves processing performance, enables automated conversion of high-quality data, reduces operation and maintenance costs, and supports the efficient operation of domestic hardware platforms.
Smart Images
Figure CN122199766A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of geographic information system technology, and more specifically, to a geographic information data preprocessing method and system. Background Technology
[0002] In existing technologies, geographic information preprocessing mainly employs open-source tools such as VPB (Virtual PlanetBuilder) and QGIS. The main function of these systems is to convert massive, multi-source geospatial data (such as Digital Elevation Models (DEMs) and Digital Orthophotos (DOMs)) into multi-resolution pyramid tile data suitable for real-time 3D rendering engines (such as OpenSceneGraph, OSG). However, existing technologies have many shortcomings: insufficient realism in terrain lighting and lighting tearing severely affect the realism of the scene, especially in applications like aerial visualization where the perception of terrain undulations is extremely important; significant performance bottlenecks make them difficult to adapt to domestic environments, as these systems are typically optimized for x86 architecture and general-purpose GPU computing ecosystems such as CUDA, and cannot run directly on domestic CPUs and GPUs or utilize their full performance; and low automation in data fusion and quality assurance, requiring extensive manual intervention and post-processing checks, making it difficult to achieve high-quality output with a single click. Summary of the Invention
[0003] To enhance the realism of lighting, improve data processing performance, achieve end-to-end adaptation and optimization across multiple platforms, and ensure high-quality and automated data output, this invention proposes a geographic information data preprocessing method, which includes: The raw data is standardized by unifying the coordinates of the raw data and dividing it into multiple macro task blocks. The first kernel function, the second kernel function, and the third kernel function optimized for the GPU are loaded into the GPU, and the data corresponding to each macro task block is transferred to the GPU memory block by block. For each corresponding data point, the following steps are executed in parallel on the GPU using a GPU computing framework: The first kernel function is configured to perform tile cutting and resampling tasks. The GPU performs the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and converts the multiple corresponding tiles into geometric meshes and diffuse textures. The second kernel function is configured to perform a normal calculation task, and the GPU performs the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; The third kernel function is configured to perform a texture compression task, and the GPU performs the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; The compressed diffuse texture and normal map are encapsulated together with the geometric mesh into a data package supported by the OSG engine.
[0004] As a further improvement of the present invention, the step of transmitting the data corresponding to each macro task block to the GPU memory block by block includes: transmitting the corresponding data to the GPU memory via the PCIe bus.
[0005] As a further improvement of the present invention, the original data coordinates are unified and divided into multiple macro-task blocks, including: The original data is concatenated into a first virtual dataset, the first virtual dataset is dynamically reprojected onto the target coordinate system, a second virtual dataset is output, and the second virtual dataset is divided into multiple macro task blocks according to the first partitioning rule. The original data includes DEM data and DOM data.
[0006] As a further improvement of the present invention, the step of encapsulating the compressed diffuse texture and normal map with the geometric mesh into a data package supported by the OSG engine includes: The compressed diffuse texture and normal map are combined with the geometric mesh to form an OSG scene graph, which is then serialized into a binary file. Generate a metadata file based on the binary file to serve as a description of the binary file; The data packet includes the binary file and the metadata file.
[0007] As a further improvement of the present invention, the standardization processing of the raw data and the encapsulated data are executed in the CPU.
[0008] As a further improvement of the present invention, the GPU adopts the MUSA architecture or NVIDIA's CUDA architecture, and the GPU computing framework supports OpenCL or Vulkan as the interface language.
[0009] As a further improvement of the present invention, the step of enabling the GPU to perform the tile cutting and resampling task in parallel for the corresponding data to generate multiple corresponding tiles includes: The elevation values of the corresponding grids are calculated from the DEM data to generate geometric tiles; Generate texture tiles for the corresponding region from the DOM data; The plurality of corresponding tiles include a plurality of geometric tiles and a plurality of textured tiles.
[0010] As a further improvement of the present invention, the step of causing the GPU to perform the normal calculation task for the corresponding tile to generate the corresponding normal map includes: For each of the geometric tiles, the normal calculation task is performed to generate a unit normal. The corresponding unit normal is then encoded into RGB color values to generate a normal map with the same resolution as the geometric tile.
[0011] As a further improvement of the present invention, the step of causing the GPU to perform the texture compression task on the diffuse texture and the normal map to generate compressed diffuse texture and normal map includes: Using the texture compression function of the Vulkan interface or a dedicated texture compression algorithm unit module, the diffuse texture and the normal map are compressed into a texture format to generate the compressed diffuse texture and normal map.
[0012] This invention proposes a geographic information data preprocessing system, the system comprising: The data standardization module is used to unify the original data coordinates and divide them into multiple macro task blocks; The pyramid building module is used to load the first kernel function, the second kernel function, and the third kernel function optimized for the GPU into the GPU, and to transfer the data corresponding to each macro task block to the GPU memory block by block. For each corresponding data point, the following steps are executed in parallel on the GPU using a GPU computing framework: The tile cutting and resampling tasks are written into the first kernel function so that the GPU can execute the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and convert the multiple corresponding tiles into geometric meshes and diffuse textures; The normal calculation task is written into the second kernel function so that the GPU can execute the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; The texture compression task is written into the third kernel function so that the GPU can execute the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; The data output module is used to encapsulate data, packaging the compressed diffuse texture and normal map, along with the geometric mesh, into a data package supported by the OSG engine.
[0013] The beneficial effects of this invention are as follows: This method significantly improves the realism of lighting by processing data and completely eliminates the problem of discontinuous lighting at edges caused by real-time calculation of normals. The front-end rendering engine only needs to perform standard lighting shading to present terrain with accurate contrast, rich details, and strong three-dimensionality, greatly enhancing the immersiveness and realism of aerial visuals.
[0014] Breakthrough processing performance has been achieved by offloading the most computationally intensive tasks of tile cutting, resampling, and normal calculation to the domestic MUSA GPU and by adopting a "macro-task block fully parallel" scheduling strategy, giving full play to the many-core parallel advantages of domestic processors.
[0015] This invention features comprehensive design from algorithm and computing interface to instruction set optimization and output format, ensuring that the entire chain from data input to output can run efficiently and stably on domestic software and hardware platforms, thus strongly supporting the strategy of independent control in the field of geographic information processing.
[0016] Ensuring high-quality and automated data output, it achieves a "one-click" automated conversion from "raw data" to "render-ready, high-quality, illuminated data", significantly reducing technical barriers and operational costs. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of a geographic information data preprocessing method according to an exemplary embodiment of the present invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.
[0021] Furthermore, the terminology used in the description of this invention is for illustrative purposes only and is not intended to limit the scope of the invention. The terms "comprising" and / or "including" are used to specify the presence of said elements, steps, operations, and / or components, but do not exclude the presence or addition of one or more other elements, steps, operations, and / or components. The terms "first," "second," etc., may be used to describe various elements, do not represent an order, and do not limit these elements. Moreover, in the description of this invention, unless otherwise stated, "a plurality of" means two or more. These terms are used only to distinguish one element from another. These and / or other aspects become apparent in conjunction with the following drawings, and those skilled in the art will more readily understand the description of the embodiments of the invention. The drawings are used for illustrative purposes only to depict the embodiments of the invention. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods shown in the invention can be employed without departing from the principles of the invention.
[0022] like Figure 1 As shown in the embodiment of the present invention, a geographic information data preprocessing method includes: S1, standardize the raw data, unify the coordinates of the raw data and divide them into multiple macro task blocks; Optionally, the original data coordinates are unified and divided into multiple macro-task blocks, including: The original data is stitched together to form a first virtual dataset. The first virtual dataset is dynamically reprojected onto the target coordinate system to output a second virtual dataset. The second virtual dataset is then divided into multiple macro task blocks according to a first partitioning rule. The original data includes DEM data (Digital Elevation Model) and DOM data (Document Object Model).
[0023] Specifically, using the GDAL library (an open-source raster spatial data transformation library), multiple DEM data sets to be processed are stitched together into a DEM-VRT (DEM virtual dataset), and multiple DOM data sets to be processed are stitched together into a DOM-VRT (DOM virtual dataset). The DEM virtual dataset and the DOM virtual dataset serve as the first virtual dataset. The PROJ library (a core tool for geospatial coordinate transformation) is then called to dynamically reproject the first virtual dataset to the target coordinate system, outputting a VRT (second virtual dataset). Preferably, the target coordinate system is the Web Mercator projection, and the process of unifying the coordinates of the original data is executed on the CPU (Central Processing Unit).
[0024] The second virtual dataset is divided into multiple independent and parallel macro-task blocks according to the first partitioning rule. Preferably, the first partitioning rule is the TMS (Tile Map Service) rule. According to the TMS rule, the coarsest level (Level 0) of the entire region is divided into multiple macro-task blocks. The processing of each macro-task block includes tile generation, terrain construction and normal calculation of all its sub-levels. Each macro-task block is a completely independent and parallel-executable task unit.
[0025] The present invention adopts a strategy of independent parallel processing of macro task blocks, which eliminates the dependency between layers and allows all task blocks to start processing tiles of all layers below them at the same time, greatly improving the parallel scale and overall throughput.
[0026] S2, load the first kernel function, the second kernel function, and the third kernel function optimized for the GPU (graphics processing unit) into the GPU, and transfer the data corresponding to each macro task block to the GPU video memory block by block; For each corresponding data, the following steps S3-S5 are executed in parallel on the GPU using a GPU computing framework; Preferably, the GPU adopts the MUSA architecture (Meta-Computing Unified Computing Architecture) or NVIDIA's CUDA architecture (a computing platform launched by graphics card manufacturer NVIDIA). The GPU computing framework is implemented using CUDA or MUSA parallel programming architecture and supports OpenCL (Open Computing Language) or Vulkan (Graphics Application Programming Interface) as interface languages. The data transmission and download process in the CPU (Central Processing Unit) is synchronized with the GPU's working threads through means such as data locks.
[0027] It is understood that during system initialization, the first kernel function, the second kernel function, and the third kernel function optimized for the MUSA architecture are loaded into the GPU. Preferably, the step-by-step transfer of data corresponding to each macro task block to the GPU memory includes: transferring the corresponding data to the GPU memory via the PCIe bus. It is understood that the DMA engine of the PCIe bus (High-Speed Serial Computer Extended Bus Standard) is started, and the DMA engine (Direct Memory Access) transfers the data corresponding to the macro task block to the GPU memory block by block. The DMA engine can directly access the data in the CPU. For all the data corresponding to each macro task block, the following steps S3-S5 are executed in parallel on the GPU. While the GPU is processing a task block, the CPU is simultaneously preparing the data for the next task block, realizing the overlap of computation and I / O and avoiding GPU idleness.
[0028] Specifically, the DMA engine of the PCIe bus is started, and the DMA engine transfers the data corresponding to the first macro task block in the CPU to the GPU memory. For all the data corresponding to the first macro task block, the following steps S3-S5 are executed in parallel on the GPU. While the GPU is processing the data corresponding to the first macro task block, the CPU is simultaneously preparing the data corresponding to the second macro task block. When the first macro task block is completed, the GPU releases the memory and receives the data corresponding to the second macro task block, until all macro task blocks are completed.
[0029] Preferably, the system includes multiple GPUs, and a load balancer is set up to monitor the load of each GPU. When there are multiple domestically produced GPUs, tasks can be dynamically allocated to achieve linear acceleration. Specifically, for example, when there are two GPUs, the CPU transfers all the data corresponding to the first macro task block to the first GPU, the CPU transfers all the data corresponding to the second macro task block to the second GPU, and after the first GPU finishes processing the first macro task block, the first GPU releases its video memory, and the CPU transfers the data corresponding to the third macro task block to the first GPU, and so on, until all macro task blocks are processed.
[0030] S3, the first kernel function is configured to perform tile cutting and resampling tasks. The GPU performs the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and converts the multiple corresponding tiles into geometric meshes and diffuse textures. It is understood that after the GPU receives the corresponding data and loads the first kernel function, it executes the tasks configured in the first kernel function in parallel on all the corresponding data. The first kernel function is configured to perform tile cutting and resampling tasks, that is, to perform tile cutting and resampling tasks in parallel on the corresponding data, specifically including: The elevation values of the corresponding grids are calculated from the DEM data to generate geometric tiles; Generate texture tiles for the corresponding region from the DOM data; The plurality of corresponding tiles include a plurality of geometric tiles and a plurality of textured tiles.
[0031] The specific process of converting the multiple geometric tiles into a geometric mesh is as follows: a. Extract elevation values: Read the grayscale value of each pixel in the geometric tile and convert it into a specific height value; b. Calculate the three-dimensional coordinates and assign complete three-dimensional spatial coordinates to each pixel. The row and column position of the pixel determines the X and Z coordinates, and the height value determines the Y coordinate. c. Connect the X, Y, and Z coordinates according to the adjacent relationship of pixels to form a geometric grid.
[0032] The diffuse texture contains pixel data of the texture tiles and configuration information on how to sample this data.
[0033] Compared to performing tile cutting and resampling tasks serially on the CPU, the many-core parallelism of the MUSA architecture allows for the simultaneous processing of resampling calculations for tens of thousands of tiles, increasing the speed of this step by tens of times.
[0034] S4, the second kernel function is configured to perform a normal calculation task, and the GPU performs the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; It can be understood that the GPU executes the tasks configured in the second kernel function in parallel on multiple geometric tiles. The second kernel function is configured to perform normal calculation tasks, that is, to perform normal calculation tasks in parallel on multiple geometric tiles.
[0035] For each of the geometric tiles, the normal calculation task is performed to generate a unit normal. Specifically, for each pixel in the geometric tile, based on the elevation difference between its neighboring pixels, the surface normal vector (X, Y, Z) of that point in geographic space is calculated, and the elevation change rate along the X direction is expressed as... The rate of change of elevation along the Y direction is expressed as The elevation change rate along the Z direction is For vectors ( , , Normalization is performed to obtain the unit normal; preferably, a 3x3 Sobel filter is used to calculate the elevation difference between adjacent pixels.
[0036] Then, the corresponding unit normal is encoded into RGB color values to generate a normal map with the same resolution as the geometric tile.
[0037] This invention calculates high-precision terrain normals in a single step during the preprocessing stage, ensuring the consistency and high quality of the normals. It shifts the computational cost from the real-time stage to the preprocessing stage and utilizes the parallel computing capabilities of the MUSA GPU, so that normal calculations add almost no increase to the overall processing time. By performing normal calculations through geometric tiles, the normal information is solidified as an inherent attribute into the output tiles, completely eliminating the problems of discontinuous lighting and poor realism caused by real-time normal calculations. The front-end rendering engine only needs to perform standard lighting shading to present terrain with accurate contrast, rich details, and strong three-dimensionality, greatly enhancing the immersion and realism of aerial views and improving the overall realism of terrain rendering.
[0038] S5, the third kernel function is configured to perform a texture compression task, and the GPU performs the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; Preferably, the diffuse texture and the normal map are compressed in real time into an efficient texture format using the texture compression function of the Vulkan interface supported by the MUSA architecture or a dedicated texture compression algorithm unit module, generating the compressed diffuse texture and normal map; preferably, the real-time compression is to the ASTC (Adaptive Scalable Texture Compression) format, which can achieve fast decoding on most GPUs and reduce texture memory usage by more than 70%.
[0039] The dedicated texture compression algorithm unit addresses the different GPU types and the support levels of domestic GPUs. This unit abstracts the implementation of the compression algorithm, exposing a unified API interface to the user layer, but the underlying implementation is encapsulated according to the version of the OpenGL or Vulkan graphics interface and the version of the CUDA or MUSA parallel interface. For older GPUs, this unit automatically outputs uncompressed raw texture formats or earlier compressed formats (such as BC1-BC5) to avoid recognition issues during actual rendering. For newer GPUs, this unit outputs compressed texture formats with better data quality and processing performance (such as BC7), and further accelerates the processing through parallel algorithms.
[0040] S6, encapsulate the data, and encapsulate all the compressed diffuse textures and normal maps, along with the geometric mesh, into a data package supported by the OSG engine (an open-source 3D engine written in standard C++ and OpenGL).
[0041] Specifically, on the CPU side, the compressed diffuse texture and the compressed normal map are combined with the geometric mesh processed by the GPU to form an OSG scene graph, and serialized into a binary file, namely a .osgb file. The OSG scene graph is a scene graph structure supported by the OSG engine. Generate a metadata file based on the binary file to serve as a description of the binary file; The data package includes the binary file and the metadata file. Preferably, the metadata file is a metadata description file for the entire 3D tile dataset. Depending on the user's settings, it can be a tilesset.json file (for 3dtiles format output, running in a browser) or a metadata.xml file (for osgb format output, running on a desktop).
[0042] The data package generated by the method described in this application contains a 3D scene model that has undergone quadtree hierarchical processing and has automatically mapped texture and normal information onto the model. It can be directly rendered via OpenGL, Vulkan, or the browser-side WebGL graphics interface. When the original scene data is extremely large, the data package generated by the method described in this application will be automatically divided into multiple sub-blocks for streaming loading and rendering. The simulation system can dynamically load sub-blocks entering the viewport or unload sub-blocks leaving the viewport based on the current user's field of view information to maintain a balance between memory and GPU rendering pressure. Based on this, users can directly browse, analyze, and visualize the current 3D scene in the simulation system without additional processing.
[0043] This invention ensures efficient collaboration between the CPU and GPU through the MUSA software architecture, and is compatible with a variety of GPUs.
[0044] The entire system is driven by a main controller (running on the CPU), which is responsible for parsing the processing flow, dividing macro task blocks, and putting these task blocks into a queue.
[0045] The MUSA device worker thread pool continuously retrieves task blocks from the queue and submits them to the GPU for execution through the MUSA runtime interface.
[0046] By adopting a producer-consumer model, when the GPU is processing one task block, the CPU is simultaneously preparing data for the next task block, thus achieving overlap between computation and I / O and avoiding GPU idleness.
[0047] A load balancer is configured to monitor the load on each GPU. When multiple domestically produced GPUs are used, tasks can be dynamically allocated to achieve linear acceleration. One possible implementation method involves processing DEM and DOM data covering 50,000 square kilometers with an original data volume of 1TB.
[0048] On a domestic workstation equipped with a domestically produced 8-core CPU and a Moore's Threads MTT S4000 GPU, where the MTT S4000 is an enterprise-grade GPU accelerator card from Moore's Threads designed for AI large-scale model training and push integration, the method described in this invention is used: S1. Standardize the raw data. Using the GDAL library, multiple DEM data to be processed are concatenated into a DEM-VRT, and multiple DOM data to be processed are concatenated into a DOM-VRT. The DEM virtual dataset and the DOM virtual dataset are used as the first virtual dataset. The PROJ library is called to dynamically reproject the first virtual dataset to the target coordinate system Web Mercator, and the output is a VRT. The process of unifying the coordinates of the raw data is executed on the CPU.
[0049] According to the TMS rule, the coarsest level (Level 0) of the entire region of the second virtual dataset is divided into 256 macro task blocks. The processing of each macro task block includes tile generation, terrain construction and normal calculation of all its sub-levels. Each macro task block is a completely independent task unit that can be executed in parallel.
[0050] S2, during system initialization, the first kernel function, the second kernel function, and the third kernel function optimized for the MUSA architecture are loaded into the GPU; the DMA engine of the PCIe bus is started, and the DMA engine transfers the data corresponding to the first macro task block in the CPU to the GPU memory; the MUSA GPU processes the data corresponding to each macro task block in parallel, and simultaneously executes the following steps S3-S5: while the MUSA GPU is processing the data corresponding to the first macro task block, the CPU is simultaneously preparing the data corresponding to the second macro task block; when the first macro task block is completed, the MUSA GPU releases the memory and receives the data corresponding to the second macro task block, until all 256 macro task blocks are processed.
[0051] S3, execute the tile cutting and resampling tasks in the first kernel function in parallel on the corresponding data, calculate the elevation value of the corresponding grid from the DEM data to generate geometric tiles, and convert the geometric tiles into geometric grids; generate texture tiles for the corresponding area from the DOM data, and convert the texture tiles into diffuse textures; S4, the normal calculation task in the second kernel function is executed in parallel on multiple geometric tiles to obtain unit normals, and then the corresponding unit normals are encoded into RGB color values to generate a normal map with the same resolution as the geometric tiles.
[0052] S5 performs texture compression tasks in the third kernel function on diffuse textures and normal maps in parallel, compressing them into ASTC format in real time.
[0053] S6. Encapsulate the data by encapsulating all the compressed diffuse textures and normal maps, along with the geometric mesh, into a data package supported by the OSG engine. The data package is a tile dataset containing 20 levels of LOD (Level of Detail), a geometric mesh, normal maps, and diffuse textures in ASTC compression format.
[0054] The total processing time has been reduced from the estimated 50 hours for traditional VPB on x86 CPUs to 4 hours. Furthermore, the generated terrain in the flight visual simulation system can display accurate, smooth, and seamless lighting effects without any real-time calculations, resulting in a qualitative leap in the three-dimensionality and realism of the terrain.
[0055] An embodiment of the present invention illustrates a geographic information data preprocessing system, the system comprising: The data standardization module is used to unify the original data coordinates and divide them into multiple macro task blocks; The pyramid building module is used to load the first kernel function, the second kernel function, and the third kernel function optimized for the GPU into the GPU, and to transfer the data corresponding to each macro task block to the GPU memory block by block. For each corresponding data point, the following steps are executed in parallel on the GPU using a GPU computing framework: The tile cutting and resampling tasks are written into the first kernel function so that the GPU can execute the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and convert the multiple corresponding tiles into geometric meshes and diffuse textures; The normal calculation task is written into the second kernel function so that the GPU can execute the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; The texture compression task is written into the third kernel function so that the GPU can execute the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; The data output module is used to encapsulate data, packaging the compressed diffuse texture and normal map, along with the geometric mesh, into a data package supported by the OSG engine.
[0056] The first kernel function, the second kernel function, and the third kernel function mentioned in this application all refer to different kernel functions and do not have any semantic order.
[0057] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0058] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.
[0059] Those skilled in the art will understand that although the invention has been described with reference to exemplary embodiments, various changes may be made and its elements may be substituted with equivalents without departing from the scope of the invention. Furthermore, many modifications may be made to adapt particular situations or materials to the teachings of the invention without departing from the essential scope of the invention. Therefore, the invention is not limited to the specific embodiments disclosed, but rather the invention will include all embodiments falling within the scope of the appended claims.
Claims
1. A geographic information data preprocessing method, characterized in that, The method includes: The raw data is standardized by unifying the coordinates of the raw data and dividing it into multiple macro task blocks. The first kernel function, the second kernel function, and the third kernel function optimized for the GPU are loaded into the GPU, and the data corresponding to each macro task block is transferred to the GPU memory block by block. For each corresponding data point, the following steps are executed in parallel on the GPU using a GPU computing framework: The first kernel function is configured to perform tile cutting and resampling tasks. The GPU performs the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and converts the multiple corresponding tiles into geometric meshes and diffuse textures. The second kernel function is configured to perform a normal calculation task, and the GPU performs the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; The third kernel function is configured to perform a texture compression task, and the GPU performs the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; The compressed diffuse texture and normal map are packaged together with the geometric mesh into a data package supported by the OSG engine.
2. The geographic information data preprocessing method as described in claim 1, characterized in that, The stepwise transfer of data corresponding to each macro task block to the GPU memory includes: transferring the corresponding data to the GPU memory via the PCIe bus.
3. The geographic information data preprocessing method as described in claim 1, characterized in that, The original data coordinates are unified and divided into multiple macro-task blocks, including: The original data is concatenated into a first virtual dataset, the first virtual dataset is dynamically reprojected onto the target coordinate system, a second virtual dataset is output, and the second virtual dataset is divided into multiple macro task blocks according to the first partitioning rule. The original data includes DEM data and DOM data.
4. The geographic information data preprocessing method as described in claim 1, characterized in that, The step of encapsulating the compressed diffuse texture and normal map with the geometric mesh into a data package supported by the OSG engine includes: The compressed diffuse texture and normal map are combined with the geometric mesh to form an OSG scene graph, which is then serialized into a binary file; Generate a metadata file based on the binary file to serve as a description of the binary file; The data packet includes the binary file and the metadata file.
5. The geographic information data preprocessing method as described in claim 1, characterized in that, The standardization process for the raw data and the encapsulated data is executed in the CPU.
6. The geographic information data preprocessing method as described in claim 1, characterized in that, The GPU adopts the MUSA architecture or NVIDIA's CUDA architecture, and the GPU computing framework supports OpenCL or Vulkan as the interface language.
7. The geographic information preprocessing method as described in claim 3, characterized in that, The step of enabling the GPU to perform the tile cutting and resampling task in parallel for the corresponding data to generate multiple corresponding tiles includes: The elevation values of the corresponding grids are calculated from the DEM data to generate geometric tiles; Generate texture tiles for the corresponding region from the DOM data; The plurality of corresponding tiles include a plurality of geometric tiles and a plurality of textured tiles.
8. The geographic information data preprocessing method as described in claim 7, characterized in that, The step of enabling the GPU to perform the normal calculation task for the corresponding tile to generate the corresponding normal map includes: For each of the geometric tiles, the normal calculation task is performed to generate a unit normal. The corresponding unit normal is then encoded into RGB color values to generate a normal map with the same resolution as the geometric tile.
9. The geographic information data preprocessing method as described in claim 1, characterized in that, The step of causing the GPU to perform the texture compression task on the diffuse texture and the normal map to generate compressed diffuse texture and normal map includes: Using the texture compression function of the Vulkan interface or a dedicated texture compression algorithm unit module, the diffuse texture and the normal map are compressed into a texture format to generate the compressed diffuse texture and normal map.
10. A geographic information data preprocessing system, characterized in that, The system includes: The data standardization module is used to unify the original data coordinates and divide them into multiple macro task blocks; The pyramid building module is used to load the first kernel function, the second kernel function, and the third kernel function optimized for the GPU into the GPU, and to transfer the data corresponding to each macro task block to the GPU memory block by block. For each corresponding data point, the following steps are executed in parallel on the GPU using a GPU computing framework: The tile cutting and resampling tasks are written into the first kernel function so that the GPU can execute the tile cutting and resampling tasks in parallel for the corresponding data to generate multiple corresponding tiles, and convert the multiple corresponding tiles into geometric meshes and diffuse textures; The normal calculation task is written into the second kernel function so that the GPU can execute the normal calculation task in parallel for the corresponding tile to generate the corresponding normal map; The texture compression task is written into the third kernel function so that the GPU can execute the texture compression task in parallel for the diffuse texture and the normal map to generate compressed diffuse texture and normal map; The data output module is used to encapsulate data, packaging the compressed diffuse texture and normal map with the geometric mesh into a data package supported by the OSG engine.