Low-altitude digital twin multi-mode layer fusion rendering method and system, terminal and medium

Abstract

The invention belongs to the technical field of layer fusion rendering, and particularly discloses a low-altitude digital twin multi-mode layer fusion rendering method and system, a terminal and a medium. Comprising the following steps: acquiring multi-source spatial data required by a low-altitude digital twinborn scene, performing format analysis and coordinate conversion on the multi-source spatial data, establishing a unified spatial coordinate reference, and generating multi-modal basic data with a unified resolution; the fused multi-modal scene data is formed; establishing a hierarchical index and spatial buffer system; and constructing a hierarchical rendering pipeline according to the spatial index system, and outputting a three-dimensional visualization result after fusion rendering, so that high-precision registration and fusion expression of multi-source heterogeneous spatial data can be realized, and the problem that multi-modal data in an existing low-altitude scene cannot be managed in a unified manner and cannot be dynamically visualized is solved. The method can be widely applied to the fields of urban low-altitude traffic supervision, unmanned aerial vehicle track display, meteorological situation awareness, three-dimensional geographic information visualization and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of layer fusion rendering, and particularly relates to a low-altitude digital twin multi-modal layer fusion rendering method, system, terminal and medium. BACKGROUND

[0002] With the rapid development of low-altitude economy, unmanned aerial vehicle supervision, urban digital management and three-dimensional geographic information system, low-altitude digital twin scenarios gradually become an important supporting technology for urban airspace visualization and intelligent analysis. By constructing a virtual twin environment corresponding to the real airspace, dynamic monitoring and simulation analysis of urban terrain, building groups, flight tracks, weather changes and other multi-source elements can be realized.

[0003] Among them, multi-modal spatial data such as terrain models, oblique photography models, point cloud models and vector image data constitute the basic information source of low-altitude twin scenarios. There are significant differences between different data types in spatial resolution, coordinate reference and format structure. How to efficiently fuse and real-time render them is the key to realizing accurate visualization and real-time interaction of low-altitude twin systems.

[0004] The existing technology generally adopts layered loading or single-modal rendering to realize three-dimensional scene construction. The common scheme usually takes terrain data as the basic layer, superimposes oblique photography models or point cloud models to reconstruct the scene, and then combines vector data or image texture to modify the surface, so as to realize three-dimensional display of urban or geographic areas.

[0005] However, the existing technology still has the following deficiencies: The existing scheme usually renders terrain, oblique photography and point cloud data independently, and cannot realize unified scheduling and dynamic fusion of multi-modal data in the same rendering pipeline, especially lacking a collaborative mechanism in terms of LOD hierarchical loading and precision adaptation, resulting in problems such as texture misalignment, resolution mismatch and rendering delay between different layers.

[0006] The existing three-dimensional visualization system mostly takes Cartesian coordinates or tile indexes as the core, lacks a unified spatial grid coding model, and is difficult to realize spatial relationship calculation and hierarchical retrieval of point, line, surface and volume multi-dimensional objects. Especially in low-altitude complex airspace, it is impossible to complete airspace buffer zone modeling and distance calculation through a standardized grid system, which limits the spatial analysis accuracy and real-time rendering efficiency of the twin scenario. SUMMARY

[0007] The present application provides a low-altitude digital twin multi-modal layer fusion rendering method, system, terminal and medium to solve the problems of texture misplacement, resolution mismatch and rendering delay between different layers in the background art, and to solve the problem of inability to complete airspace buffer modeling and distance calculation through a standardized grid system in a low-altitude complex airspace, which limits the spatial analysis accuracy and real-time rendering efficiency of the twin scene.

[0008] The technical scheme adopted by the present application is as follows: In a first aspect, the present application provides a low-altitude digital twin multi-modal layer fusion rendering method, which comprises the following steps: Step S1, obtaining multi-source spatial data required by a low-altitude digital twin scene, the multi-source spatial data comprising terrain data, oblique photography data, point cloud data, vector data, image data, weather service data and flight situation data, wherein the terrain data stores elevation information in a terrain format, the oblique photography data and the point cloud data store three-dimensional geometric information in a 3dtiles format, the vector data stores spatial boundary information in a GeoJSON format, and the image data stores texture information in a WMS protocol; Step S2, performing format analysis and coordinate conversion on the multi-source spatial data, establishing a unified spatial coordinate reference, converting the original coordinate systems of different types of data into a unified geographic coordinate system, performing spatial alignment on the oblique photography data and the point cloud data according to the elevation information of the terrain data, and obtaining multi-modal basic data of a unified spatial resolution; Step S3, based on the multi-modal basic data, taking the terrain data as a base layer, superimposing and registering the oblique photography data and the point cloud data in the same three-dimensional space, limiting the scene boundary by using the vector data, and performing surface texture mapping by combining the image data, to form fused multi-modal scene data; Step S4, based on the GeoSOT spatial grid encoding rule, performing spatial encoding on the multi-modal scene data, mapping point, line, surface and body objects into corresponding grid cells, calculating the spatial buffer range and hierarchical index of the objects according to the grid cells, and establishing a spatial index system of the multi-modal scene data; Step S5, establishing a hierarchical rendering pipeline according to the spatial index system, loading the terrain, oblique photography, point cloud and image texture as different levels of rendering nodes, executing a LOD hierarchical loading strategy based on the hierarchical index, adaptively adjusting the rendering accuracy of the scene at different viewing distances, and taking the weather service data and the flight situation data as dynamic superimposed layers, and synchronously updating the multi-modal scene data; Step S6, outputting the low-altitude digital twin three-dimensional visualization result after fusion and rendering, the result comprising a fused and displayed terrain surface, an oblique photography model, point cloud details, image texture, weather information and flight situation information.

[0009] Furthermore, in step S3, the fusion of multimodal base data adopts a spatial continuity model based on the balance between normal vector consistency and depth gradient, and the fusion function... Defined as:

[0010] in, Let be the direction of the normal vector of the i-th point in the point cloud. The direction of the local normal vector of the terrain surface. Let be the perpendicular distance between point i and the terrain surface. This represents the elevation value of the corresponding pixel in the oblique photography. For the weight of the sampling points in the region, This is the distance attenuation coefficient. It is the weighted exponent of the angle between the normal vectors.

[0011] Furthermore, in step S4, the spatial encoding of the multimodal scene data is based on the three-dimensional GeoSOT hierarchical mapping function. Mapping function Defined as:

[0012] in, Let L be a point in three-dimensional space, and L be the depth of the encoding level. For the scene boundary range, The rounding operation is represented by a mapping function. Each spatial point in the multimodal scene data is mapped to a unique three-dimensional integer index.

[0013] Furthermore, based on mapping functions On the established spatial coding structure, for the fusion function Adjacency weight modeling and buffer calculation are performed on the generated multimodal scene surface, and a spatial adjacency weight matrix is ​​defined. With dynamic buffer function for:

[0014]

[0015] in, For the The coordinates of the center points of adjacent grid cells are encoded. The distance between the centers of the two units is the Euclidean distance. This is the spatial scale coefficient. For fusion function exist The gradient vector at that point, The gradient sensitivity coefficient of the normal vector. Based on the buffer radius, For adaptive expansion coefficients, To avoid small constants that divide by zero.

[0016] Furthermore, based on the adjacency matrix With buffer function Combined with spatial coding Establish multi-level index functions Its definition is:

[0017] in, Encoding for space Two-dimensional components, This is the smoothing adjustment coefficient in the horizontal and vertical directions. This is a hierarchical hash mapping function used for index mapping at the k-th level.

[0018] Furthermore, in step S5, for the spatial index unit The LOD level adopts a linearly additive adaptive model:

[0019] in, Location of the observation point. The distance from the observation point to the center of the unit. For the maximum visible radius, The angle between the line of sight and the normal to the element surface. For the fusion function in gradient magnitude at that point For the maximum elevation difference across the entire scene, For the neighborhood Inside The mean, This is the mean of that level. The normalization constant is Non-negative weights; according to The size selection corresponds to the detail level of the rendering node.

[0020] Furthermore, during the hierarchical rendering process, spatial index units are... Establish a local refresh scoring function A local redraw is triggered when a threshold condition is met:

[0021] like Then for unit The area in question will be partially redrawn; in, For changes in meteorological elements, This refers to changes in flight status. The change in LOD. To normalize the scale for each channel, Non-negative weights As the trigger threshold, This is the refresh cycle.

[0022] Secondly, this application provides a low-altitude digital twin multimodal layer fusion rendering system for implementing the low-altitude digital twin multimodal layer fusion rendering method as described in the first aspect. The system includes: The data acquisition module is used to acquire terrain data, oblique photogrammetry data, point cloud data, vector data, image data, meteorological service data, and flight status data from multi-source spatial data interfaces. Among them, the terrain data uses the terrain format to store elevation information, the oblique photogrammetry data and point cloud data use the 3dtiles format to store three-dimensional geometric information, the vector data uses the GeoJSON format to store spatial boundary information, and the image data uses the WMS protocol to store texture information. The data parsing module is used to perform format parsing and coordinate transformation on the multi-source spatial data, establish a unified spatial coordinate benchmark, and spatially align the oblique photography data and point cloud data according to the elevation information of the terrain data to generate multimodal basic data with unified resolution. The data fusion module is used to overlay and register oblique photogrammetry data and point cloud data with terrain data as the base layer, use vector data to define scene boundaries, perform texture mapping in combination with image data, and generate multimodal scene data based on the normal vector consistency and depth gradient balance model. The spatial encoding module is used to perform spatial grid encoding based on GeoSOT rules on the multimodal scene data, calculate the hierarchical index and buffer range of each spatial object, and construct a spatial index system. The index optimization module is used to calculate the adjacency weight matrix and dynamic buffer function in the spatial encoding structure, construct a multi-layer index structure based on the adjacency relationship and buffer radius, and generate index functions to achieve efficient spatial retrieval and rendering scheduling. The rendering control module is used to establish a hierarchical rendering pipeline based on the spatial indexing system. It adopts a linearly additive adaptive LOD hierarchical model to control the loading accuracy of rendering nodes and realize multi-parameter layered rendering based on distance, angle, terrain gradient, buffer radius and index difference. The dynamic refresh module is used to calculate the local refresh scoring function. When the change in meteorological service data, flight status data or LOD level exceeds the set threshold, it triggers local redrawing of the corresponding area to achieve low-latency dynamic visualization updates. The results output module is used to output the low-altitude digital twin 3D visualization results after fusion and rendering. The results include the fused terrain surface, oblique photogrammetry model, point cloud details, image texture, meteorological information and flight status information.

[0023] Thirdly, this application provides a terminal, including: Memory for storing low-altitude digital twin multimodal layer fusion rendering programs; A processor is configured to implement the steps of the low-altitude digital twin multimodal layer fusion rendering method as described in the first aspect when executing the low-altitude digital twin multimodal layer fusion rendering apparatus.

[0024] Fourthly, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the low-altitude digital twin multimodal layer fusion rendering method as described in the first aspect.

[0025] As can be seen from the above technical solutions, the advantages of the present invention are: By establishing a unified processing workflow for multi-source data, including terrain, oblique photography, point clouds, vector data, imagery, meteorological data, and flight status data, this method achieves the collaborative fusion of spatial information from different sources and in different formats under a unified coordinate system. This enables low-altitude scenes to possess high-precision geometric shapes, realistic texture representation, and real-time situational awareness. The method automates the entire process from data parsing, fusion, encoding, rendering to output, effectively reducing the errors and time costs associated with traditional manual modeling and multi-source data registration, and significantly improving the real-time performance and scalability of digital twin models.

[0026] By introducing a spatial continuity fusion function based on normal vector consistency and depth gradient balance, the oblique photogrammetry data and point cloud data are spatially registered, taking into account not only geometric position but also surface orientation and depth variations, achieving a smooth transition and normal vector consistency at model stitching points. This fusion algorithm eliminates the "step effect" and "gap layering" problems between different data sources, resulting in a more realistic continuous 3D surface that significantly improves the model's visual realism and geometric accuracy.

[0027] By employing a 3D GeoSOT hierarchical mapping function, multimodal scene data is mapped from physical space to a unified integer grid index system, achieving spatial hierarchical encoding across data sources and resolutions. This method supports hierarchical organization and rapid retrieval of large-scale 3D spatial data, providing a unified encoding foundation for subsequent buffer calculations and LOD hierarchical rendering, and significantly improving the scalability and computational efficiency of spatial indexing.

[0028] Building upon spatial encoding, this improvement combines gradient information from the fusion function to define an adjacency weight matrix and a dynamic buffer function. This allows spatial adjacency relationships to be determined not only by distance but also by the trend of surface geometric changes. This enhancement enables the buffer range to adaptively adjust based on terrain undulations, building density, or data complexity, achieving dynamic optimization of spatial region partitioning. It effectively avoids the segmentation distortion problem of traditional fixed-radius buffer algorithms in complex terrain scenes, thereby improving the stability of spatial analysis and rendering.

[0029] Based on adjacency and buffer models, a multi-layered indexing function is established by integrating spatial coding information, realizing a joint indexing mechanism of geometric features, topological relationships, and spatial location. This hierarchical indexing model not only improves the retrieval speed of multimodal data across different levels but also supports rapid localization of regionalized and segmented rendering tasks, thereby enabling efficient visualization task scheduling and memory resource management in large-scale low-altitude scenes.

[0030] By constructing a linearly additive Level of Detail (LOD) hierarchical model, multiple factors such as distance, viewpoint, terrain gradient, buffer radius, and index differences are jointly incorporated into the rendering control, enabling adaptive layered loading and dynamic resolution adjustment of the scene. This model can determine the rendering accuracy in real time based on the viewpoint position and spatial complexity, reducing unnecessary computation while ensuring visual quality, and significantly improving frame rate stability and resource utilization in large-scene visualization.

[0031] By establishing a local refresh scoring function, changes in meteorological data, flight status, LOD changes, spatial buffering, and index deviation are comprehensively incorporated into a dynamic judgment system, enabling real-time redrawing of local areas based on multi-source dynamic data. This mechanism can trigger visualization updates instantly when local scene states change due to meteorological changes or the movement of flying targets, without requiring overall re-rendering, thus achieving a low-latency, high-response dynamic display effect for twin scenes. Through these improvements, this invention not only achieves the fusion rendering of static spatial information and dynamic situational data but also possesses high spatiotemporal adaptability and computational efficiency, supporting real-time 3D twin visualization applications in complex low-altitude environments. Attached Figure Description

[0032] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a step diagram of the low-altitude digital twin multimodal layer fusion rendering method in the embodiment. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 As shown, this invention provides a low-altitude digital twin multimodal layer fusion rendering method, comprising the following steps: Step S1: Obtain multi-source spatial data required for the low-altitude digital twin scene. The multi-source spatial data includes terrain data, oblique photogrammetry data, point cloud data, vector data, image data, meteorological service data, and flight status data. Among them, the terrain data uses the terrain format to store elevation information, the oblique photogrammetry data and point cloud data use the 3dtiles format to store three-dimensional geometric information, the vector data uses the GeoJSON format to store spatial boundary information, and the image data uses the WMS protocol to store texture information. In practical implementation, terrain and imagery data can be accessed through a geographic information system platform or open geographic data interfaces. Terrain data can originate from satellite mapping or ground-based laser altimetry results, while oblique photogrammetry data can be acquired by UAV oblique cameras at multiple angles of 45°. Point cloud data can be obtained through LiDAR scanning. Vector data can consist of geographic elements such as building boundaries and road centerlines provided by urban planning departments, and imagery data can be generated from remote sensing imagery or aerial orthophotos. Meteorological service data and flight status data can be accessed in real time via HTTP or MQTT interfaces. The former includes elements such as temperature, wind speed, and humidity, while the latter includes the coordinates, speed, trajectory, and attitude information of flying targets. To improve system stability, a caching mechanism can be set up at the data acquisition end, supporting data block loading and time synchronization markers, enabling static geographic information and dynamic status data to be processed in parallel in subsequent steps.

[0036] Step S2: Perform format parsing and coordinate transformation on multi-source spatial data, establish a unified spatial coordinate benchmark, convert the original coordinate system of different types of data into a unified geographic coordinate system, and spatially align the oblique photography data and point cloud data based on the elevation information of the terrain data to obtain multimodal basic data with unified spatial resolution. In this step, EPSG:4326 (WGS84) or EPSG:4490 (CGCS2000) can be used as a unified geographic coordinate system. Data in various formats can be read and parsed using GDAL, Cesium, or open-source spatial data parsing libraries. For example, terrain format data contains elevation raster information and can be used as a vertical reference; 3dtiles format oblique photogrammetry and point cloud data undergo coordinate calculation and scaling using the tile transform field in the metadata file; spatial boundaries in GeoJSON data can be directly mapped to the target coordinate system. After parsing, the system performs spatial registration of the point cloud and oblique photogrammetry model based on the terrain elevation grid, ensuring complete alignment of the three in both the vertical and horizontal directions. To ensure accuracy, a spatial weighted average method can be used to smooth the coordinates of overlapping areas, thereby eliminating spatial deviations caused by acquisition errors. Through this step, multimodal base data with unified spatial resolution and geographic accuracy is obtained, laying a data consistency foundation for subsequent fusion and rendering.

[0037] Step S3: Based on multimodal basic data, with terrain data as the base layer, oblique photogrammetry data and point cloud data are superimposed and registered in the same three-dimensional space. Vector data is used to define the scene boundary, and surface texture mapping is performed in combination with image data to form fused multimodal scene data. In practical implementation, the system uses a terrain elevation raster as the base grid and spatially overlays tiles from the oblique photogrammetry model with point cloud blocks. During the overlay process, the system first detects the spatial overlap range of each data source and uses a local plane fitting method to calculate the vertical deviation between them, achieving accurate registration through interpolation correction. Boundary information from the vector data is used to limit the scene display range; for example, airport areas, city blocks, or specified terrain segments can be selected as the rendering range, thereby reducing redundant data loading. Subsequently, the image data is mapped onto the surface of the fused model, and texture coordinates are obtained through geographic projection calculation, so that the final fused model simultaneously possesses realistic surface texture and high-precision geometry. In practical applications, if the terrain has significant undulations, the point cloud data can be locally simplified to prevent stretching or folding during texture mapping. After completing this step, multimodal scene data with high-fidelity geometry, continuous surface features, and realistic texture can be generated.

[0038] The fusion of multimodal base data adopts a spatial continuity model based on normal vector consistency and depth gradient balance, and the fusion function... Defined as:

[0039] in, Let be the direction of the normal vector of the i-th point in the point cloud. The direction of the local normal vector of the terrain surface. Let be the perpendicular distance between point i and the terrain surface. This represents the elevation value of the corresponding pixel in the oblique photography. For the weight of the sampling points in the region, This is the distance attenuation coefficient. The weighted index of the angle between the normal vectors; Step S4: Based on the GeoSOT spatial grid coding rules, perform spatial coding on the multimodal scene data, map point, line, surface and volume objects to corresponding grid cells, calculate the spatial buffer range and hierarchical index of the objects according to the grid cells, and establish a spatial indexing system for the multimodal scene data. In practical implementation, the system determines the GeoSOT encoding level depth based on the scene's spatial range and resolution, typically selecting 8 to 16 levels to balance accuracy and storage efficiency. Terrain surfaces, buildings, and flying targets are discretized into several cubic units, each corresponding to a unique 3D index value. The system further calculates the spatial adjacency matrix and buffer range of each unit based on its distance relationship with surrounding units to enable neighborhood retrieval and collision detection. For densely built areas, the buffer radius is automatically reduced to improve spatial resolution; for flat terrain areas, the buffer radius is appropriately expanded to reduce data redundancy. The spatial indexing system constructed through this step significantly improves the retrieval speed of 3D objects, enhancing the efficiency of subsequent rendering loading, spatial querying, and dynamic updates.

[0040] Spatial encoding of multimodal scene data is based on a 3D GeoSOT hierarchical mapping function. Mapping function Defined as:

[0041] in, Let L be a point in three-dimensional space, and L be the depth of the encoding level. For the scene boundary range, The rounding operation is represented by a mapping function. Map each spatial point in the multimodal scene data to a unique three-dimensional integer index; Based on mapping function On the established spatial coding structure, for the fusion function Adjacency weight modeling and buffer calculation are performed on the generated multimodal scene surface, and a spatial adjacency weight matrix is ​​defined. With dynamic buffer function for:

[0042]

[0043] in, For the The coordinates of the center points of adjacent grid cells are encoded. The distance between the centers of the two units is the Euclidean distance. This is the spatial scale coefficient. For fusion function exist The gradient vector at that point, The gradient sensitivity coefficient of the normal vector. Based on the buffer radius, For adaptive expansion coefficients, To avoid small constants that divide by zero; Based on adjacency matrix With buffer function Combined with spatial coding Establish multi-level index functions Its definition is:

[0044] in, Encoding for space Two-dimensional components, This is the smoothing adjustment coefficient in the horizontal and vertical directions. This is a hierarchical hash mapping function used for index mapping at the k-th level; Step S5: Establish a hierarchical rendering pipeline based on the spatial indexing system, load terrain, oblique photography, point cloud and image texture as rendering nodes of different levels, execute the LOD hierarchical loading strategy based on the hierarchical index, so that the scene can adaptively adjust the rendering accuracy under different viewing distances, and use meteorological service data and flight status data as dynamic overlay layers, and update synchronously with multimodal scene data. In practical implementation, the system uses the hierarchical information of the GeoSOT index to divide terrain data, oblique photogrammetry models, and point cloud data into multiple rendering nodes, each corresponding to a Level of Detail (LOD) level. When the observer is far from a certain area, only low-resolution terrain and image textures are loaded; when the observer moves closer, higher-resolution oblique photogrammetry and point cloud detail models are automatically loaded, thus balancing rendering quality and computational load. Meteorological service data and flight situation data are accessed through a real-time API interface and displayed in the multimodal scene as dynamic overlays, such as rendering cloud movement, wind direction, or aircraft trajectories in real time. To ensure smoothness, the system adopts a multi-threaded rendering and asynchronous data stream loading mechanism to ensure that the loading of different data types does not block each other. Through this step, the scene rendering accuracy can be adaptively adjusted according to the viewing distance, angle, and data complexity, achieving unified rendering and dynamic linkage of multi-source data.

[0045] For spatial index units The LOD level adopts a linearly additive adaptive model:

[0046] in, Location of the observation point. The distance from the observation point to the center of the unit. For the maximum visible radius, The angle between the line of sight and the normal to the element surface. For the fusion function in gradient magnitude at that point For the maximum elevation difference across the entire scene, For the neighborhood Inside The mean, This is the mean of that level. The normalization constant is Non-negative weights; according to The size selection corresponds to the detail level of the rendering node; During the hierarchical rendering process, spatial index units Establish a local refresh scoring function A local redraw is triggered when a threshold condition is met:

[0047] like Then for unit The area in question will be partially redrawn; in, For changes in meteorological elements, This refers to changes in flight status. The change in LOD. To normalize the scale for each channel, Non-negative weights As the trigger threshold, For refresh cycle; Step S6: Output the low-altitude digital twin 3D visualization results after fusion and rendering. The results include the fused terrain surface, oblique photogrammetry model, point cloud details, image texture, meteorological information and flight status information. In practical applications, the system outputs the rendered results in a 3D interactive interface, allowing users to view the low-altitude digital twin scene in real time through rotation, zoom, and panning. The visualization results show continuous elevation undulations on the terrain surface, realistic building forms displayed by the oblique photogrammetry model, enhanced spatial depth through point cloud details, and realistic color information provided by image textures. Simultaneously, meteorological data drives dynamic changes in cloud cover, wind direction, and visibility, while flight status data updates the aircraft's position, speed, and trajectory in real time, enabling the digital twin scene to possess dynamic perception and interactivity. The system can further export the visualization results as a web-based 3D model or video presentation file for applications such as flight control, low-altitude safety monitoring, or urban low-altitude traffic planning, achieving a comprehensive, multi-dimensional, and real-time visualization of the low-altitude environment.

[0048] In some embodiments, this application provides a low-altitude digital twin multimodal layer fusion rendering system, the system comprising: The data acquisition module is used to acquire terrain data, oblique photogrammetry data, point cloud data, vector data, image data, meteorological service data, and flight status data from multi-source spatial data interfaces. Among them, the terrain data uses the terrain format to store elevation information, the oblique photogrammetry data and point cloud data use the 3dtiles format to store three-dimensional geometric information, the vector data uses the GeoJSON format to store spatial boundary information, and the image data uses the WMS protocol to store texture information. The data parsing module is used to perform format parsing and coordinate transformation on the multi-source spatial data, establish a unified spatial coordinate benchmark, and spatially align the oblique photography data and point cloud data according to the elevation information of the terrain data to generate multimodal basic data with unified resolution. The data fusion module is used to overlay and register oblique photogrammetry data and point cloud data with terrain data as the base layer, use vector data to define scene boundaries, perform texture mapping in combination with image data, and generate multimodal scene data based on the normal vector consistency and depth gradient balance model. The spatial encoding module is used to perform spatial grid encoding based on GeoSOT rules on the multimodal scene data, calculate the hierarchical index and buffer range of each spatial object, and construct a spatial index system. The index optimization module is used to calculate the adjacency weight matrix and dynamic buffer function in the spatial encoding structure, construct a multi-layer index structure based on the adjacency relationship and buffer radius, and generate index functions to achieve efficient spatial retrieval and rendering scheduling. The rendering control module is used to establish a hierarchical rendering pipeline based on the spatial indexing system. It adopts a linearly additive adaptive LOD hierarchical model to control the loading accuracy of rendering nodes and realize multi-parameter layered rendering based on distance, angle, terrain gradient, buffer radius and index difference. The dynamic refresh module is used to calculate the local refresh scoring function. When the change in meteorological service data, flight status data or LOD level exceeds the set threshold, it triggers local redrawing of the corresponding area to achieve low-latency dynamic visualization updates. The results output module is used to output the low-altitude digital twin 3D visualization results after fusion and rendering. The results include the fused terrain surface, oblique photogrammetry model, point cloud details, image texture, meteorological information and flight status information.

[0049] In some embodiments, this application provides a terminal, including: Memory for storing low-altitude digital twin multimodal layer fusion rendering programs; A processor is configured to execute the steps of the low-altitude digital twin multimodal layer fusion rendering method when implementing the low-altitude digital twin multimodal layer fusion rendering system.

[0050] In some embodiments, this application provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the low-altitude digital twin multimodal layer fusion rendering method.

[0051] It is understood that the systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can be a personal computer, a laptop computer, a personal digital assistant, a tablet computer, a wearable device, or any combination of these devices.

[0052] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0053] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0054] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0055] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0056] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0057] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A low-altitude digital twin multimodal layer fusion rendering method, characterized in that, Includes the following steps: Step S1: Obtain multi-source spatial data required for the low-altitude digital twin scene. The multi-source spatial data includes terrain data, oblique photogrammetry data, point cloud data, vector data, image data, meteorological service data, and flight status data. Among them, the terrain data uses the terrain format to store elevation information, the oblique photogrammetry data and point cloud data use the 3dtiles format to store three-dimensional geometric information, the vector data uses the GeoJSON format to store spatial boundary information, and the image data uses the WMS protocol to store texture information. Step S2: Perform format parsing and coordinate transformation on multi-source spatial data, establish a unified spatial coordinate benchmark, convert the original coordinate system of different types of data into a unified geographic coordinate system, and spatially align the oblique photography data and point cloud data based on the elevation information of the terrain data to obtain multimodal basic data with unified spatial resolution. Step S3: Based on multimodal basic data, with terrain data as the base layer, oblique photogrammetry data and point cloud data are superimposed and registered in the same three-dimensional space. Vector data is used to define the scene boundary, and surface texture mapping is performed in combination with image data to form fused multimodal scene data. Step S4: Based on the GeoSOT spatial grid coding rules, perform spatial coding on the multimodal scene data, map point, line, surface and volume objects to corresponding grid cells, calculate the spatial buffer range and hierarchical index of the objects according to the grid cells, and establish a spatial indexing system for the multimodal scene data. Step S5: Establish a hierarchical rendering pipeline based on the spatial indexing system, load terrain, oblique photography, point cloud and image texture as rendering nodes of different levels, execute the LOD hierarchical loading strategy based on the hierarchical index, so that the scene can adaptively adjust the rendering accuracy under different viewing distances, and use meteorological service data and flight status data as dynamic overlay layers, and update synchronously with multimodal scene data. Step S6: Output the fused and rendered low-altitude digital twin 3D visualization results, including the fused terrain surface, oblique photogrammetry model, point cloud details, image texture, meteorological information, and flight status information.

2. The low-altitude digital twin multimodal layer fusion rendering method according to claim 1, characterized in that, In step S3, the fusion of multimodal base data adopts a spatial continuity model based on normal vector consistency and depth gradient balance, and the fusion function... Defined as: in, Let be the direction of the normal vector of the i-th point in the point cloud. The direction of the local normal vector of the terrain surface. Let be the perpendicular distance between point i and the terrain surface. This represents the elevation value of the corresponding pixel in the oblique photography. For the weight of the sampling points in the region, This is the distance attenuation coefficient. It is the weighted exponent of the angle between the normal vectors.

3. The low-altitude digital twin multimodal layer fusion rendering method according to claim 2, characterized in that, In step S4, the spatial encoding of the multimodal scene data is based on the three-dimensional GeoSOT hierarchical mapping function. Mapping function Defined as: in, Let L be a point in three-dimensional space, and L be the depth of the encoding level. For the scene boundary range, The rounding operation is represented by a mapping function. Each spatial point in the multimodal scene data is mapped to a unique three-dimensional integer index.

4. The low-altitude digital twin multimodal layer fusion rendering method according to claim 3, characterized in that, Based on mapping function On the established spatial coding structure, for the fusion function Adjacency weight modeling and buffer calculation are performed on the generated multimodal scene surface, and a spatial adjacency weight matrix is ​​defined. With dynamic buffer function for: in, For the The coordinates of the center points of adjacent grid cells are encoded. The distance between the centers of the two units is expressed in Euclidean form. This is the spatial scale coefficient. For fusion function exist The gradient vector at that point, The gradient sensitivity coefficient of the normal vector. Based on the buffer radius, For adaptive expansion coefficients, To avoid small constants that divide by zero.

5. The low-altitude digital twin multimodal layer fusion rendering method according to claim 4, characterized in that, Based on adjacency matrix With buffer function Combined with spatial coding Establish multi-level index functions Its definition is: in, Encoding for space Two-dimensional components, This is the smoothing adjustment coefficient in the horizontal and vertical directions. This is a hierarchical hash mapping function used for index mapping at the k-th level.

6. The low-altitude digital twin multimodal layer fusion rendering method according to claim 5, characterized in that, In step S5, for the spatial index unit The LOD level adopts a linearly additive adaptive model: in, Location of the observation point. The distance from the observation point to the center of the unit. For the maximum visible radius, The angle between the line of sight and the normal to the element surface. For the fusion function in gradient magnitude at that point For the maximum elevation difference across the entire scene, For the neighborhood Inside The mean, This is the mean of that level. The normalization constant is Non-negative weights; according to The size selection corresponds to the detail level of the rendering node.

7. The low-altitude digital twin multimodal layer fusion rendering method according to claim 6, characterized in that, During the hierarchical rendering process, spatial index units Establish a local refresh scoring function A local redraw is triggered when a threshold condition is met: like Then for unit The area in question will be partially redrawn; in, For changes in meteorological elements, This refers to changes in flight status. This represents the change in LOD. To normalize the scale for each channel, Non-negative weights As the trigger threshold, This is the refresh cycle.

8. A low-altitude digital twin multimodal layer fusion rendering system, used to implement the low-altitude digital twin multimodal layer fusion rendering method as described in any one of claims 1-7, characterized in that, The system includes: The data acquisition module is used to acquire terrain data, oblique photogrammetry data, point cloud data, vector data, image data, meteorological service data, and flight status data from multi-source spatial data interfaces. Among them, the terrain data uses the terrain format to store elevation information, the oblique photogrammetry data and point cloud data use the 3dtiles format to store three-dimensional geometric information, the vector data uses the GeoJSON format to store spatial boundary information, and the image data uses the WMS protocol to store texture information. The data parsing module is used to perform format parsing and coordinate transformation on the multi-source spatial data, establish a unified spatial coordinate benchmark, and spatially align the oblique photography data and point cloud data according to the elevation information of the terrain data to generate multimodal basic data with unified resolution. The data fusion module is used to overlay and register oblique photogrammetry data and point cloud data with terrain data as the base layer, use vector data to define scene boundaries, perform texture mapping in combination with image data, and generate multimodal scene data based on the normal vector consistency and depth gradient balance model. The spatial encoding module is used to perform spatial grid encoding based on GeoSOT rules on the multimodal scene data, calculate the hierarchical index and buffer range of each spatial object, and construct a spatial index system. The index optimization module is used to calculate the adjacency weight matrix and dynamic buffer function in the spatial encoding structure, construct a multi-layer index structure based on the adjacency relationship and buffer radius, and generate index functions to achieve efficient spatial retrieval and rendering scheduling. The rendering control module is used to establish a hierarchical rendering pipeline based on the spatial indexing system. It adopts a linearly additive adaptive LOD hierarchical model to control the loading accuracy of rendering nodes and realize multi-parameter layered rendering based on distance, angle, terrain gradient, buffer radius and index difference. The dynamic refresh module is used to calculate the local refresh scoring function. When the change in meteorological service data, flight status data or LOD level exceeds the set threshold, it triggers local redrawing of the corresponding area to achieve low-latency dynamic visualization updates. The results output module is used to output the low-altitude digital twin 3D visualization results after fusion and rendering. The results include the fused terrain surface, oblique photogrammetry model, point cloud details, image texture, meteorological information and flight status information.

9. A terminal, characterized in that, include: Memory for storing low-altitude digital twin multimodal layer fusion rendering programs; A processor is configured to implement the steps of the low-altitude digital twin multimodal layer fusion rendering method as described in claim 1 when executing the low-altitude digital twin multimodal layer fusion rendering apparatus.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions. When the computer reads the computer instructions from the storage medium, the computer executes the low-altitude digital twin multimodal layer fusion rendering method as described in claim 1.

Images