Linkage display method and system based on fusion of geographic information system and game engine

Abstract

The invention discloses a linkage display method and system based on fusion of a geographic information system and a game engine, and the method comprises the steps: carrying out the standardization processing of original geographic information data and situation data, generating data resources which are respectively compatible with a first visual view and a second visual view, and loading the data resources into a corresponding rendering pipeline; constructing a main window of the first visual view, constructing browser middleware of a heterogeneous kernel as a pixel flow container, and bearing and displaying a pixel flow from the second visual view through the browser middleware; when the observation visual height is lower than a preset threshold value, activating and displaying a small micro-scale scene of the second visual view; when the observation visual height is higher than a preset threshold value, switching to a large-scale and medium-scale scene of the first visual view; and when scene switching or user view angle change occurs, synchronizing the camera view angle parameters and position coordinates of the two parties through the communication link. According to the method, the problems of global and local scene expression separation, cumbersome system integration, unsmooth engine switching, poor data compatibility and the like are solved.

Description

Technical Field

[0001] This invention relates to visualization technology, specifically to a method and system for interactive display based on the fusion of geographic information systems and game engines. Background Technology

[0002] The development of information technology is constantly advancing in all areas and aspects of life. As people's understanding of the world has undergone significant changes with the help of computer science, various new concepts, ideas, applications, and possibilities are constantly emerging. The integration of Geographic Information Systems (GIS) with the metaverse enables simultaneous geospatial precision, real-world data integration, and immersive visualization. GIS data and layers can be used to create realistic digital 3D environments that include terrain, buildings, and other relevant elements, allowing users to explore three-dimensional virtual representations of city-scale scenes.

[0003] Compared to traditional Geographic Information Systems (GIS), game engines offer more features to enrich user interaction and experience. A key built-in feature of game engines is the first-person perspective (FPV). FPV allows game engine users to control their own movement, even skillfully moving or performing extreme actions such as rolling and stunts while interacting with real-world models. Game engines' support for various 3D data formats also allows for the use of diverse terrain data. GIS visualization engines struggle to realistically recreate 3D scenes, presenting significant development challenges; while game engines, such as Unreal Engine and Unity3D, can construct detailed scenes through real-time dynamic rendering technology. Unreal Engine is a popular game engine capable of creating and efficiently rendering detailed and realistic 3D virtual environments. The connection between GIS engines and game engines has given new meaning to geographic information exploration in the new era. Currently, some research has been conducted both domestically and internationally on the integration of 3D GIS engines and game engines, such as data and functional integration.

[0004] Currently, a more direct approach to data fusion is to integrate GIS data with game engines by developing GIS data processing and import plugins specifically for game engines. The technical logic can be summarized as follows: standardized GIS data (such as terrain, models, and vector data) is imported into the game engine through plugins, and the high-fidelity rendering pipeline of the game engine is used to visualize the geographic scene. Simultaneously, the real-time update mechanism of GIS data ensures the timeliness of geographic information. However, this method needs to address two key issues: first, the secondary customization development of the game engine (such as adapting to GIS data formats (such as Shapefile, GeoJSON) and expanding spatial analysis interfaces); second, the construction of the interaction logic between GIS data and the game engine (such as data synchronization and event response (such as triggering attribute queries when clicking on geographic features)).

[0005] For example, Cesium's open-source plugin, Cesium for Unreal, integrates a 3D Tiles spatial indexing mechanism into Unreal Engine, enabling real-time transmission and loading of multi-source GIS data such as models and terrain. This breaks through the spatial limitations of game engines on 3D scenes, supporting the construction of borderless 3D geographic scenes. GIS vendors such as SuperMap and ESRI have also released similar plugins or development kits (such as SuperMap for Unity and ESRI's ArcGIS for Unreal) to achieve the integration of GIS data with game engines.

[0006] Another more robust integration approach is deep integration of core functions, which involves building a unified underlying platform to organically integrate the core modules of the 3D GIS engine and the game engine. This approach relies on a newly designed integration framework to reconstruct and interface-connect key components of both engines across engines. Its core objective is to achieve bidirectional interaction between geographic data and rendered scenes, as well as collaborative processing of spatial analysis and real-time visualization, thereby overcoming the limitations of the "data-rendering" separation in the "pseudo-fusion" model and providing deeper geographic scene construction and analysis capabilities. Some vendors (Cesium, ESRI, SuperMap, VTS, etc.) are exploring full-stack geographic rendering frameworks that, through a unified spatial data model (such as GeoJSON+glTF fusion format) and rendering state management, enable real-time analysis results of GIS data to directly drive scene rendering in the game engine. At the same time, user interaction operations in the game engine (such as scene zooming and feature selection) can trigger spatial queries in the GIS engine, forming a closed loop of "analysis-rendering-interaction".

[0007] Therefore, there is an urgent need in this field for a fusion display solution that can achieve lightweight, highly adaptable, and seamless linkage while maintaining the advantages of both engines, in order to solve technical problems such as the disconnect between the overall situation and local details, cumbersome system integration, and sluggish engine switching. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for interactive display based on the integration of geographic information systems and game engines.

[0009] The technical solution to achieve the purpose of this invention is: a linkage display method based on the integration of geographic information systems and game engines, comprising the following steps:

[0010] Data preprocessing and dual-engine loading steps: The raw geographic information data and situational data are standardized to generate data resources adapted to the first and second visualization views respectively, and loaded into the corresponding rendering pipelines; wherein, the first visualization view is built based on the OSG engine and is used to display large and medium scale scenes; the second visualization view is built based on the Unreal Engine (UE) and is used to display small and micro scale scenes.

[0011] Dual-engine window construction steps: Construct the main window of the first visualization view, and construct a browser middleware with a heterogeneous kernel as a pixel stream container, and carry and display the pixel stream from the second visualization view through the browser middleware;

[0012] Seamless scene switching steps: Monitor the user's viewing height; when the viewing height is lower than a preset threshold, activate and display the small-scale scene of the second visualization view; when the viewing height is higher than the preset threshold, switch to the large-scale scene of the first visualization view.

[0013] Dual-engine interactive linkage steps: Establish a communication link between the first visualization view and the second visualization view; when scene switching or user perspective changes, synchronize the camera perspective parameters and position coordinates of both parties through the communication link.

[0014] Furthermore, in the data preprocessing and dual-engine loading steps, preprocessing is performed on the image data, elevation data, and situation vector data respectively to generate data resources adapted to the first and second visualization views, wherein:

[0015] For imagery and elevation data, use tools such as GDAL or GlobalMapper to perform format conversion and tile tiling;

[0016] The situation vector data is converted into GeoJSON format and pushed in real time via WebSocket service.

[0017] Furthermore, in the dual-engine window construction step, the browser middleware is implemented in the following way:

[0018] A customized browser application can be independently compiled and generated based on the Qt WebEngine module;

[0019] In the main application, launch the process of the customized browser application and obtain its window handle;

[0020] Using Qt's window embedding mechanism, the browser window corresponding to the window handle is embedded into a specified container control of the main window of the first visual view.

[0021] Furthermore, the customized browser application automatically loads the pixel stream service URL address of the second visualization view during initialization.

[0022] Furthermore, the seamless scene switching step also includes a resource scheduling step:

[0023] When multi-scale roaming exploration is active, the dual engines are in full-time operation, eliminating data loading delays caused by switching.

[0024] When scene switching behavior is very infrequent or the view height remains within a range that does not contain the critical point for an extended period, the background engine is paused to improve overall computing performance.

[0025] Furthermore, in the dual-engine interactive linkage step, the communication link is a bidirectional communication link based on the WebSocket protocol;

[0026] Both the first and second visualization views act as WebSocket clients, connecting to a WebSocket server to forward camera status information.

[0027] Furthermore, the camera status information is encapsulated and serialized using a general data structure before transmission. This data structure includes message type, timestamp, camera position coordinates, camera rotation attitude, and projection parameters.

[0028] Furthermore, the synchronization of camera viewpoint parameters and position coordinates between the two parties via the communication link includes a coordinate transformation step:

[0029] Convert the WGS84 latitude and longitude coordinates from the first visualization view to geocentric ECEF coordinates;

[0030] Convert the ECEF coordinates to Northeast ENU coordinates with the scene origin as a reference;

[0031] The ENU coordinates are converted to world coordinates in the second visualization view.

[0032] Furthermore, the synchronization of camera viewpoint parameters and position coordinates between the two parties via the communication link includes a parameter mapping step:

[0033] The view matrix defined by the Eye, Center, and Up vectors in the first visualization view is converted into camera parameters defined by the Location, Pitch / Yaw / Roll Euler angles, and FOV field of view in the second visualization view.

[0034] A linked display system based on the integration of geographic information system and game engine includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the method.

[0035] Compared with the prior art, the significant advantages of this invention are:

[0036] 1) Lightweight and highly adaptable integration

[0037] By using a customized browser as a dedicated container for the UE pixel stream, third-party dependencies are avoided, making it more adaptable to rendering needs. The Kylin system window handle solves the cross-process identification problem, ensuring accurate embedding of the pixel stream into the main application. Qt's embedding mechanism achieves seamless interface integration, improving user experience consistency. Encapsulated window attribute control allows the main application to flexibly adjust display effects without manipulating the underlying UE, reducing development complexity and system coupling. The core advantage is the "lightweight and highly adaptable" integration of the pixel stream and the main application, balancing display quality and development efficiency.

[0038] 2) Complete data compatibility system

[0039] A complete preprocessing system has been built, consisting of "format conversion, data slicing, hierarchical organization and loading, situational hierarchical management, and smooth situational browsing (hierarchical aggregation and detailed control)". It comprehensively covers the entire process of data from format adaptation to loading, management, and presentation, and has optimized each stage of preprocessing. It has also solved data compatibility issues and improved loading efficiency and the logic of situational presentation.

[0040] 3) Cross-engine communication compatibility

[0041] A WebSocket middleware is created to implement camera data mapping and feedback. A general data structure and serialization enable cross-engine compatibility, ensuring that the position coordinates and viewpoints are synchronized without deviation when switching scenes between OSG and UE. The middleware decouples the two engines, reduces direct dependencies, and improves scalability. Serialization optimizes transmission efficiency, and the switching process is real-time and has no delay, achieving a seamless experience when switching scenes across engines.

[0042] 4) Seamless switching between global and refined scenes

[0043] By ensuring consistent data organization hierarchy between the OSG and UE engines and unifying the window display logic, visual seamlessness and logical coherence in cross-engine scene switching are achieved. For example, the "global-regional-local" pyramid structure visualization remains consistent, and the parent-child nodes and hierarchical relationships of the scene remain unchanged after switching, avoiding the "scene fragmentation" problem caused by hierarchical confusion and ensuring consistent user understanding of the scene. When the user is in an active state of roaming and exploration (such as frequent zooming and panning), the OSG and UE engines run simultaneously, and scene switching does not require waiting for data loading or engine startup, achieving a seamless "point-and-switch" experience. When scene switching behavior is minimal (such as when the user stays at a certain scale for a long time) or the exploration viewpoint exceeds the set critical area (such as when small details do not need to be displayed), the background engine (such as the currently unused UE engine or OSG engine) automatically pauses, releasing CPU, GPU and other resources, improving the running efficiency of the foreground engine, and avoiding "dual-engine idling". Attached Figure Description

[0044] Figure 1 This is a flowchart of a linked display method based on the integration of geographic information systems and game engines;

[0045] Figure 2 It is a pixel stream carrying middleware technology route;

[0046] Figure 3 It is a technical route for integrating basic geographic data and situational data;

[0047] Figure 4 It is a synchronous linkage technology route based on the dual-engine fusion mode. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] A method for interactive display based on the integration of geographic information systems and game engines includes the following steps:

[0050] Step 100: Dual-engine integration of basic geographic information data (imagery, elevation, etc.) and situational data (moving targets, static targets, vectors, etc.);

[0051] Step 101: As Figure 2 As shown, different preprocessing schemes are performed for imagery, elevation, and situational data to preprocess the basic geographic information data and situational data that need to be loaded into the UE scene, ensuring that the data format is effectively compatible with the UE engine.

[0052] Step 102: Display the data resources on the UE engine front end using the data loading scheme on the UE engine side.

[0053] Furthermore, this invention utilizes tools such as GDAL and GlobalMapper to achieve UE engine-compatible format conversion, employs tools such as CesiumLab and ArcGIS for data slicing, and leverages the data carrying tools provided by the UE or integrates plugins such as Cesium for Unreal and SuperMap for UE to achieve hierarchical data organization and loading. For situational data, a situational hierarchical management scheme is adopted, hierarchically managing key elements and related attribute information. In terms of situational reconnaissance, smooth browsing can be achieved through hierarchical aggregation and model detail control. The specific details of preprocessing and loading are as follows:

[0054] 1) Image data preprocessing

[0055] Format conversion: Basic geographic image data is generally in GeoTIFF / JPEG2000 (raster) format. The GDAL tool can be used to convert the image data to PNG / JPG (tiled) supported by UE or retain GeoTIFF (UE5 natively supports it).

[0056] UE engine-side loading solution: For local tiles, enable the "World Partition" plugin, create a LandscapeActor, select "Import Heightmap" in the "Landscape" properties, fill in the tile path, and set the "Projection Type" to WGS84; for online services, use the Cesium for Unreal plugin, add the "Cesium Ion" component, select "Add Image Layer" (such as ESRI, Bing), or upload local tiles.

[0057] 2) Elevation data preprocessing

[0058] Format conversion: DEM / DSM are generally GeoTIFF / HGT. Global Mapper converts HGT to UE terrain system compatible RAW (unformatted binary, retaining elevation accuracy) or retains GeoTIFF containing elevation information.

[0059] UE Engine Loading Solution: Create a Landscape Actor, click "Import Heightmap," select a raw file, set the "Terrain Size" (e.g., 10km×10km), "Component Size Quads" (e.g., 63, i.e., 64×64 vertices), adjust the "Height Range" (e.g., 0-5000m), and generate the terrain. Plugin Support: Add the "CesiumTerrain" component via Cesium for Unreal, select "Cesium World Terrain" (online service) or upload a local GeoTIFF file, set the "Subdivision Level" (e.g., 15 levels to control detail); or use the SuperMap for UE plugin, select "Load DEM Data" in the "SuperMap" menu, select a local GeoTIFF file, set the "Coordinate System" to WGS84, click "Load," and the terrain will be automatically generated.

[0060] 3) Preprocessing of situation vector data

[0061] Format conversion: Vector data is generally in Shapefile / GeoJSON / OSM format. UE5 supports direct import of Shapefile / GeoJSON to ensure the integrity of situational attribute fields.

[0062] UE engine-side loading solution: Copy the Shapefile to the UE's "Content" folder, select "Import as Static Mesh," and set the "Geometry Type" to "Plane" (boundary) or "Solid." For dynamic display (e.g., moving target location), set up a WebSocket service to push real-time data in GeoJSON format. Create a blueprint in UE, add a "WebSocket Client" component, connect to the service, parse the "Coordinates" and "Status" fields of the GeoJSON (using the "Json Reader" node), and generate an Actor (e.g., an airplane model). Set the position (convert latitude and longitude to Cartesian coordinates using the "Longitude / Latitude to UE Position" function from "Cesium Georeference").

[0063] Step 200: Use QT front-end development tools to create a visualization window for large and medium-scale scene objects driven primarily by GIS.

[0064] The OSG engine is used to render GIS data, and the scene management function is developed using QT kernel tools. The OSG engine uses the QT 5.7.1 kernel version. The software test environment is shown in Table 1.

[0065] Table 1. Software testing environment for dual-engine fusion exploration mode

[0066]

[0067] Step 300: Create and activate browser middleware compatible with different versions of the QT kernel to effectively carry the pixel stream and eliminate compatibility issues caused by QT kernel versions, such as... Figure 3 As shown, by generating an independent browser application, obtaining the browser handle, and encapsulating the browser into a window object that can be embedded into the OSG engine's main application container, the transmission and expression of pixel streams are realized.

[0068] The operating system, QT kernel version, and UE version parameters for development and adaptation are shown in Table 1. The OSG engine carries GIS data and has a QT 5.7.1 kernel. The independent browser window module requires QT 5.15.2 kernel support. The browser is customized through independent compilation of QtWebEngine. The window handle of the Kylin system is generated to achieve cross-process window recognition. The browser acts as a "rendering container for UE pixel stream". The Qt window embedding mechanism is used to achieve interface integration. The window attribute control is encapsulated to indirectly manage the display effect of UE pixel stream. Through the control of the container window by the main application (OSG engine side), the pixel stream and the main application interface are seamlessly integrated.

[0069] Step 301: Build an independent browser application based on Qt5.15.2 WebEngine. In the browser application's initialization process, point to the entry URL of the UE pixel stream service.

[0070] Extract the Qt WebEngine module (including core classes such as QWebEngineView and QWebEngineProfile) from the Qt 5.15.2 source code, and compile it independently (using qmake or CMake to configure compilation options) to generate a standalone browser application with a customized interface. In the browser application's initialization process, the default loading address is preset using the QWebEngineView::load(QUrl) method, pointing to the entry URL of the UE pixel stream service (e.g., http: / / ). <pixel-stream-server-ip>:80), ensure that the browser automatically connects to the pixel stream service after startup.

[0071] Step 302: Triggering and obtaining the browser process handle in the main application (OSG engine side);

[0072] After the main application starts, a high-version browser process is launched and a handle is obtained through the QProcess class (or the fork / exec mechanism of the Kylin system) (the browser version supports the WebRTC protocol and media decoding capabilities of UE pixel stream, such as QT kernel version 5.15.2).

[0073] Step 303: The browser window object is embedded in the QT object container of the main application (OSG engine side).

[0074] The main application uses the cross-platform window embedding interface provided by Qt to convert the obtained native browser handle into a Qt-manageable window object, and encapsulates the native handle (WId) into a Qt QWindow object; the QWindow object is then embedded into a newly created container window (such as a QWidget subclass) of the main application to achieve hierarchical fusion between the browser window and the main application window.

[0075] Step 400: Use QT front-end development tools to create a visualization window for small-scale scene objects driven primarily by the user experience (UX), such as... Figure 3 As shown, the window is embedded in a Qwidget container that can be controlled by the OSG engine main application, bringing the UE engine pixel stream into the display of the small-scale scene window.

[0076] Step 500: Seamless switching between large / medium-scale and small / micro-scale scenes. Determine if the current viewport's viewing height is less than a set threshold. If not, maintain the large / medium-scale scene viewport or switch to the large / medium-scale scene viewport, and execute the traditional GIS basic geographic information and situational data visualization process, visualizing the data in a GIS-driven visualization window. If yes, maintain the small / micro-scale scene viewport or switch to the small / micro-scale scene viewport, and generate a Unreal Engine small / micro-scale scene pixel stream. Push the pixel stream to the UE-driven small / micro-scale browser visualization window via WebRTC.

[0077] Furthermore, when the roaming and exploration state is relatively active, the dual engines are in full-time operation, eliminating the data loading delay caused by switching; if scene switching behavior is very rare or the exploration viewpoint is outside the set critical area, the engine in the background is paused to improve the overall computing performance.

[0078] Step 600: Determine if there is any interactive linkage between the two engines, such as data updates and perspective changes, and if scene mode switching occurs (switching between large / medium scale scenes and small / micro scale scenes). Then, through the established dual-engine interactive control communication link, synchronize position coordinates and camera perspective parameter information, such as... Figure 4 As shown.

[0079] Step 601: Communication link established. Both the OSG engine and the UE engine act as clients and connect to the same WebSocket server (such as Node.js or Python Flask-SocketIO).

[0080] The OSG engine is implemented in the main application using libwebsockets or Boost.Beast; the UE engine can integrate the WebSocket client through a built-in solution; the OSG engine listens through the main loop, while the UE engine receives coordinates and camera parameters by binding the WebSocket message receiving callback function; the OSG engine obtains the camera update callback and current position, serializes and sends them to the WebSocket server, while the UE engine listens for camera changes through the Tick function and serializes and sends them to the WebSocket server.

[0081] 1) OSG engine implementation principle

[0082] Integrating WebSocket client

[0083] Implement a WebSocket client in the OSG engine using libwebsockets (lightweight, cross-platform) or Boost.Beast (the network module in the Boost library).

[0084] Receive UE data

[0085] In the main loop (osgViewer::Viewer::run()) on the OSG engine side, each frame checks if there are any new messages in the WebSocket: parses the JSON / binary data, extracts the camera state; converts the UE coordinates to OSG coordinates; and updates the OSG camera's position (osg::Camera::setViewMatrixAsLookAt) and pose (osg::Quat).

[0086] Send OSG camera data

[0087] In the OSG engine's camera update callback (osg::Camera::addUpdateCallback), listen for changes in camera state (such as user actions and animations): obtain the current position (osg::Vec3) and pose (osg::Quat) of the OSG camera; convert them to UE coordinate format; serialize them and send them to the server / UE via WebSocket.

[0088] 2) Implementation principle of UE engine

[0089] UE provides built-in WebSocket support (IWebSocketClient interface), which can be quickly integrated via Blueprints or C++ code:

[0090] Integrating WebSocket client

[0091] C++: Use the FWebSocketClient class and add the WebSocket module to Build.cs;

[0092] Blueprint: Use the "Create WebSocket" node, with the path Network-WebSocket.

[0093] Receive OSG data

[0094] The OnMessageReceived callback of the WebSocket is bound (triggered when an OSG message is received): parses the camera data in the message; converts the OSG coordinates to UE coordinates; updates the UE camera, sets the position using SetActorLocation, and sets the pose using SetControlRotation.

[0095] Send UE camera data

[0096] In the UE's Tick function (executed every frame), monitor changes in camera state (such as player movement or mouse rotation): obtain the UE camera's current position (GetActorLocation) and pose (GetActorRotation); convert them to OSG coordinate format; serialize them and send them to the server / OSG via WebSocket.

[0097] Step 602: Define a cross-engine common camera state data structure and implement data exchange through serialization (converting memory objects into a transmission format). The serialization structure is shown below.

[0098] {

[0099] "type": "camera_sync", / / Message type

[0100] "timestamp": 1694234567890, / / Timestamp

[0101] "camera": {

[0102] "position": [x, y, z], / / Camera world coordinates

[0103] "rotation": [pitch, yaw, roll], / / posture

[0104] "projection": { / / Projection parameters

[0105] "fov": 60.0,

[0106] "aspect_ratio": 16 / 9

[0107] }

[0108] }

[0109] }

[0110] Step 603: Synchronize position coordinates, which involves coordinate transformation. For details, please refer to the position coordinate switching scheme below. This can be implemented through algorithm logic or directly through the UE's blueprint node capabilities.

[0111] 1) Convert latitude and longitude to ECEF coordinates:

[0112] Convert latitude and longitude to radians: (latitude), (Longitude); Calculate WGS84 ellipsoid parameters: semi-major axis Meters, first eccentricity ; Calculate the radius of the circle: ECEF coordinate system (right-handed system, Earth-centered and Earth-fixed):

[0113]

[0114]

[0115]

[0116] 2) ECEF converted to UE world space coordinates:

[0117] Select the origin of the UE world space (such as the scene center) and convert its latitude and longitude to ECEF coordinates. Convert the ECEF vector to the ENU coordinate system (East-North-East-Sky, adapting to the UE axis direction), with the rotation matrix R starting from the latitude of the origin. and longitude calculate:

[0118]

[0119] Calculate the relative vector: , , ;

[0120] Convert to ENU coordinates: ;

[0121] Converted to UE world space: The UE's X-axis corresponds to E (East), Y-axis to N (North), and Z-axis to U (Day). Units are converted from meters to centimeters.

[0122]

[0123]

[0124]

[0125] 3) At the UE5 implementation level, the main technical steps are as follows:

[0126] Create a GeoReferencing System: Add this object to the UE5 level, set the ellipsoid shape to "Round Planet," and uncheck the "Origin Location in Projected CRS" option.

[0127] ‌ Enter latitude, longitude, and altitude: Enter the latitude, longitude, and altitude values ​​of the target location in the GeoReferencing System.

[0128] ‌ Coordinate transformation tool: The transformation is achieved through the "GeoGraphic to Engin" function, which is located in the Blueprint node of UE5.

[0129] Step 604: Synchronize camera view parameters. See the camera view synchronization linkage scheme for details, which involves parameter transfer and mapping.

[0130] The OSG's camera defines its view matrix using position (Eye), focus (Center), and up vector (Up); the UE's camera defines its viewpoint using location (Location), rotation (including pitch / yaw / roll), and field of view (FOV). The core of synchronization is converting the OSG's view matrix parameters into the UE's camera state parameters.

[0131] 1) Key parameter mapping

[0132]

[0133] 2) Rotation calculation from OSG to UE

[0134] The OSG's view matrix consists of a rotation matrix (controlling direction) and a translation matrix (controlling position). The rotation matrix is ​​defined by three orthogonal vectors: Right, Up, and Forward. The UE's camera rotation is derived from these three vectors.

[0135] After obtaining Eye(P), Center(C), and Up(U) from OSG, calculate:

[0136] Forward vector: (Unit vector from the camera to the focus).

[0137] Right vector: (Cross product, to obtain the camera's right direction, perpendicular to F and U).

[0138] Up vector: (Orthogonalize to ensure that U is perpendicular to R and F).

[0139] Constructing the rotation portion of the OSG view matrix:

[0140] The first 3×3 rotation matrix of the view matrix consists of R, U, and -F as column vectors (converting world coordinates to camera coordinates):

[0141]

[0142] Rotation matrix converted to UE:

[0143] The camera rotation matrix of the UE is The transpose (the inverse of an orthogonal matrix is ​​equal to its transpose, converting camera coordinates to world coordinates):

[0144]

[0145] Calculate the Euler angles (Pitch / Yaw / Roll) of the UE:

[0146] Will Convert to Euler angles in UE (following UE's coordinate system definition: Yaw around the Z-axis, Pitch around the X-axis, Roll around the Y-axis): (unit: radians).

[0147] Yaw (rotate horizontally, turn left or right):

[0148]

[0149] Pitch (rotate vertically, view from top to bottom):

[0150]

[0151] Roll (to tumble, to tilt left or right):

[0152]

[0153] (FOV) Synchronization:

[0154] The OSG uses the vertical field of view (vFOV), while the UE defaults to using the horizontal field of view (hFOV), so a conversion is required.

[0155]

[0156] in, =Screen width / Screen height.

[0157] In summary, this invention, starting from multi-scale seamless exploration, creates an SDK middleware between the GIS engine and Unreal Engine, eliminating version compatibility issues of browsers supporting pixel-stream data, and providing a loosely coupled dual-engine fusion solution, establishing a systematic framework for hierarchical, seamless expression. Specific advantages are as follows:

[0158] (1) Pixel Stream Bearer Middleware Design

[0159] By using a customized browser as a dedicated container for the UE pixel stream, third-party dependencies are avoided, making it more adaptable to rendering needs. The Kylin system window handle solves the cross-process identification problem, ensuring accurate embedding of the pixel stream into the main application. Qt's embedding mechanism achieves seamless interface integration, improving user experience consistency. Encapsulated window attribute control allows the main application to flexibly adjust display effects without manipulating the underlying UE, reducing development complexity and system coupling. The core advantage is the "lightweight and highly adaptable" integration of the pixel stream and the main application, balancing display quality and development efficiency.

[0160] (2) UE engine compatibility handling for GIS data

[0161] A complete preprocessing system has been built, consisting of "format conversion, data slicing, hierarchical organization and loading, situational hierarchical management, and smooth situational browsing (hierarchical aggregation and detailed control)," which comprehensively covers the entire process of data from format adaptation to loading, management, and presentation. It has optimized each stage of preprocessing, solved data compatibility issues, and improved loading efficiency and the logic of situational presentation.

[0162] (3) Establishment of the interaction link between OSG engine and UE engine

[0163] A WebSocket middleware is created to implement camera data mapping and feedback. A general data structure and serialization enable cross-engine compatibility, ensuring that the position coordinates and viewpoints are synchronized without deviation when switching scenes between OSG and UE. The middleware decouples the two engines, reduces direct dependencies, and improves scalability. Serialization optimizes transmission efficiency, and the switching process is real-time and has no delay, achieving a seamless experience when switching scenes across engines.

[0164] (4) Seamless switching design between global and refined scenes

[0165] By ensuring consistent data organization hierarchy between the OSG and UE engines and unifying the window display logic, visual seamlessness and logical coherence in cross-engine scene switching are achieved. For example, the "global-regional-local" pyramid structure visualization remains consistent, and the parent-child nodes and hierarchical relationships of the scene remain unchanged after switching, avoiding the "scene fragmentation" problem caused by hierarchical confusion and ensuring consistent user understanding of the scene. When the user is in an active state of roaming and exploration (such as frequent zooming and panning), the OSG and UE engines run simultaneously, and scene switching does not require waiting for data loading or engine startup, achieving a seamless "click-and-switch" experience. When scene switching behavior is minimal (such as when the user stays at a certain scale for a long time) or the exploration viewpoint exceeds the set critical area (such as when small details do not need to be displayed), the background engine (such as the currently unused UE engine or OSG engine) automatically pauses, releasing CPU, GPU and other resources, improving the running efficiency of the foreground engine, and avoiding "dual-engine idling".

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for linked display based on the integration of geographic information systems and game engines, characterized in that, Includes the following steps: Data preprocessing and dual-engine loading steps: The raw geographic information data and situational data are standardized to generate data resources compatible with the first and second visualization views respectively, and loaded into the corresponding rendering pipelines; wherein, the first visualization view is built based on the OSG engine and is used to display large and medium scale scenes; the second visualization view is built based on the Unreal Engine (UE) and is used to display small and micro scale scenes; Dual-engine window construction steps: Construct the main window of the first visualization view, and construct a browser middleware with a heterogeneous kernel as a pixel stream container, and carry and display the pixel stream from the second visualization view through the browser middleware; Seamless scene switching steps: Monitor the user's viewing height; when the viewing height is lower than a preset threshold, activate and display the small-scale scene of the second visualization view; when the viewing height is higher than the preset threshold, switch to the large-scale scene of the first visualization view. Dual-engine interactive linkage steps: Establish a communication link between the first visualization view and the second visualization view; when scene switching or user perspective changes, synchronize the camera perspective parameters and position coordinates of both parties through the communication link.

2. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, In the data preprocessing and dual-engine loading steps, image data, elevation data, and situation vector data are preprocessed respectively to generate data resources compatible with both the first and second visualization views, wherein: For image data, use the GDAL tool to convert the image data to PNG / JPG format supported by UE or retain GeoTIFF; For elevation data, convert it to UE terrain system compatible RAW or retain GeoTIFF containing elevation information using Global Mapper; Convert the situation vector data to Shapefile / GeoJSON format.

3. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, In the dual-engine window construction step, the browser middleware is implemented in the following way: A customized browser application can be independently compiled and generated based on the Qt WebEngine module; In the main application, launch the process of the customized browser application and obtain its window handle; Using Qt's window embedding mechanism, the browser window corresponding to the window handle is embedded into a specified container control of the main window of the first visual view.

4. The linked display method based on the fusion of geographic information system and game engine according to claim 3, characterized in that, The customized browser application automatically loads the pixel stream service URL address of the second visualization view during initialization.

5. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, The seamless scene switching process also includes a resource scheduling step: When multi-scale roaming exploration is active, the dual engines are in full-time operation, eliminating data loading delays caused by switching. When scene switching behavior is very infrequent or the view height remains within a range that does not contain the critical point for an extended period, the background engine is paused to improve overall computing performance.

6. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, In the dual-engine interactive linkage step, the communication link is a bidirectional communication link based on the WebSocket protocol; Both the first and second visualization views act as WebSocket clients, connecting to the WebSocket server and forwarding camera status information through the server.

7. The linked display method based on the fusion of geographic information system and game engine according to claim 6, characterized in that, The camera status information is transmitted after being encapsulated and serialized using a general data structure, which includes message type, timestamp, camera position coordinates, camera rotation attitude, and projection parameters.

8. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, The synchronization of camera viewpoint parameters and position coordinates between the two parties via the communication link includes a coordinate transformation step: Convert the WGS84 latitude and longitude coordinates from the first visualization view to geocentric ECEF coordinates; Convert the ECEF coordinates to Northeast ENU coordinates with the scene origin as a reference; The ENU coordinates are converted to world coordinates in the second visualization view.

9. The linked display method based on the fusion of geographic information system and game engine according to claim 1, characterized in that, The process of synchronizing the camera viewpoint parameters and position coordinates between the two parties via the communication link includes a parameter mapping step: The view matrix defined by the Eye, Center, and Up vectors in the first visualization view is converted into camera parameters defined by the Location, Pitch / Yaw / Roll Euler angles, and FOV field of view in the second visualization view.

10. A linked display system based on the fusion of a geographic information system and a game engine, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 9.

Images