Two-dimensional and three-dimensional fusion modeling interaction method and system based on UE engine

Abstract

The invention discloses a two-dimensional and three-dimensional fusion modeling interaction method and system based on a UE engine, and the method comprises the steps: collecting power grid GIS data, carrying out the year-on-year high-precision modeling, storing a model into a cloud model library, carrying out the matching and loading through a UE5 engine, importing the GIS data into a Cesium three-dimensional world terrain, and carrying out the tile cutting. The method comprises the following steps: performing spatial coordinate conversion on topographic data through a conversion algorithm, performing cloud rendering on the converted data by using a UE5 engine, encoding and packaging a UE5 rendering result into a media stream, generating a playable media stream environment provided by a browser, receiving and displaying a transmitted rendering picture by a front-end page, and monitoring an interaction instruction in real time. According to the method and the system, the equipment performance requirement is reduced through cloud rendering, the data processing capability and the rendering precision are remarkably improved, and digital twin visual interaction is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power grid visualization analysis, in particular to a two-three-dimensional fusion modeling interaction method and system based on a UE engine. BACKGROUND

[0002] With the accelerated advancement of digitalization, digital twin technology, as one of the core driving forces, aims to build a digital model that maps to the physical scene in virtual space using digital technology. The existing methods do not have real-time and scene accuracy, and it is difficult to achieve accurate reset and modeling for substation scenes, and it is difficult to realize real-time linkage of data and interaction, and thus it is difficult to realize the final scene twin system simulation to cope with possible equipment problems and countermeasures at any time.

[0003] The traditional Cesium technology system cannot show strong real-time rendering and scene restoration capabilities from the Web side, it lacks strong rendering engine support, and focuses more on geographic space interaction, resulting in limited scene data loading, single display function, lack of efficient spatial display and processing capabilities. It is difficult to accurately restore the layout of power stations and transmission and distribution lines in real geographic locations, lacks necessary data support and display interaction methods, and is not conducive to efficient display and accurate management of power data.

[0004] The UE engine is known for its high-quality graphics rendering capabilities, providing a powerful creative display platform for virtual reality, building and engineering visualization, game development, and other fields, and is suitable for developing complex and detailed interaction logic. However, it has weak support for native geographic space, lacks conversion of real Earth space coordinates, and does not have efficient processing capabilities for large-scale GIS data. Therefore, for large-scale multi-data scenarios, single means can only display in blocks, and the loading effect is not good.

[0005] In summary, the single UE5 engine lacks processing, analysis, and conversion capabilities for large-scale GIS data, and high-fidelity 3D content is limited by device performance, increasing the user's usage threshold. Secondly, in the existing technical system, it cannot combine geographic data for accurate device positioning and real-time state monitoring, machine inspection, and other new technology analysis, making the modernization of three-dimensional systems and real-time interaction capabilities lacking. SUMMARY

[0006] The purpose of the present application is to provide a two-three-dimensional fusion modeling interaction method based on a UE engine to solve the problems existing in the prior art and improve the display effect in the fusion interaction and modeling process. The second purpose of the present application is to provide a two-three-dimensional fusion modeling interaction system based on a UE engine.

[0007] Technical scheme: The two-three-dimensional fusion modeling interaction method based on the UE engine, comprising:

[0008] Collecting power grid related GIS data, including scene and device latitude and longitude, elevation data, and net rack structure; performing same-scale high-precision modeling on the power grid model according to the GIS data, and storing in a cloud model library; the model includes a tower, a line, a fitting, and a transformer substation; importing the model library into the UE5 engine for model matching and loading; importing the GIS data into a Cesium three-dimensional world terrain, Cesium cutting the high-precision terrain data into tiles and storing the tile data in a cloud server and loading the required tiles for rendering; converting the terrain data through a conversion algorithm, converting the converted data using the UE5 engine for cloud rendering; encoding and packaging the UE5 rendering result into a media stream, and generating a point-to-point playable media stream environment provided by the browser; the front-end page receives and displays the transmitted rendering picture, and simultaneously monitors the interactive instruction, focuses on the UE scene, simulates the dynamic effect and distance measurement, and performs interactive rendering operation according to the front-end instruction and synchronizes the rendering result to the front-end page.

[0009] Preferably, the model is pre-added with a custom unique label instead of a UE5 default label.

[0010] Preferably, Cesium requests a service according to the tile type and data through a Web server online HTTP, the requested tile data is formed into a three-dimensional earth surface through coordinate mapping and splicing, and the tile level resolution is automatically switched according to the distance.

[0011] Preferably, the space coordinate conversion comprises: converting WGS84 spherical coordinate system data into ECEF coordinates; converting the ECEF coordinates into root coordinates supported by the UE5; and adjusting the rigid body pose of the camera in the UE5 view angle.

[0012] Preferably, the cloud rendering specifically comprises: performing secondary processing on the model to add controllable labels, retaining model geometric information, controlling static mesh resource quality, and adjusting and encoding the rendering frame according to the rendering pipeline.

[0013] Preferably, the media stream environment is a two-way channel for constructing a pixel stream instruction, a long message transmission and analysis mechanism is used to adaptively encode, compress and transmit the cloud rendering picture to the front end, and the user input is transmitted to the cloud end in real time through a standardized encoding and check and compensation mechanism.

[0014] Preferably, the long message transmission and analysis mechanism comprises message blocking, compression and streaming transmission, and decoding and integrity checking of the blocked data are completed through adaptive compression encoding rules and check algorithms.

[0015] Preferably, the front-end page is a plugin-free front-end framework page built on WebGL technology, which receives pixel streams and displays 3D images through a browser, and handles standardized input and network quality feedback tasks.

[0016] Preferably, the pixel stream is a data packet in RDP format sent by the UE, which is transmitted via UDP to the WebRTC module of the front-end browser and reassembled. The WebRTC module uses a hardware decoder to decode the H.264 video stream, and the video stream is monitored by a videoContainer and dynamically rendered onto the canvas.

[0017] The 2D / 3D fusion modeling and interaction system based on the UE engine described in this invention includes:

[0018] Data processing layer: Based on the Cesium framework, it processes GIS data format conversion and data lightweighting to support cloud rendering, including full-size earth creation, high-resolution terrain surface rendering, and spatial coordinate transformation. The data processing layer converts spatial coordinates into adaptive local coordinates under UE, realizes adaptive rigid body posture adjustment and Euler angle viewport orientation, and supports the UE engine in interactive analysis of subsequent motion effects and presentation of realistic twin world content.

[0019] Cloud rendering layer: Using UE5 as the computing power resource hub, it performs secondary processing on the model, adds controllable identifiers, accurately preserves the geometric information of the model and controls the quality of static mesh resources, adjusts the encoding of rendering frames according to the rendering pipeline, loads and pushes the large model for rendering, and performs command animation analysis through long message transmission and parsing mechanism to simulate the state of real power scene in real time.

[0020] Streaming layer: The link layer between the front end and the back end. It builds a two-way channel for pixel stream instructions, transmits cloud-rendered images to the front end through adaptive encoding and compression, and transmits user input to the cloud in real time through standardized encoding, verification and compensation mechanisms.

[0021] Visual interaction: As a user interaction terminal, it processes captured user input into standardized input, optimizes the data structure of instructions through long message transmission mechanism, receives and reassembles laminar data based on WebRTC module, and dynamically renders it to the browser page.

[0022] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: 1. Based on cloud rendering, scene construction and visual interactive operation can be performed in real time with low requirements for device performance; 2. The system has strong data processing capabilities, high modeling accuracy, and can perform multi-platform device interaction simultaneously. Attached Figure Description

[0023] Figure 1This is a flowchart of the overall process of the present invention;

[0024] Figure 2 This is a schematic diagram of the data interaction designed for the front-end and back-end systems of this invention;

[0025] Figure 3 This is a schematic diagram of coordinate system transformation according to the present invention;

[0026] Figure 4 This is a 3D rendering of the invention from a macroscopic perspective;

[0027] Figure 5 This is a 3D rendering of the power plant modeling for the present invention;

[0028] Figure 6 This is a 3D rendering of the power transmission modeling of the present invention. Detailed Implementation

[0029] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0030] As shown in the attached diagram, the specific operation process of this invention is as follows: Collect relevant GIS data of the power grid, including scene and equipment latitude and longitude, elevation data, and grid structure; Based on the GIS data, perform high-precision modeling of the power grid model at the same scale and store it in a cloud model library. The model includes towers, lines, fittings, and substations; Import the model library into the UE5 engine for model matching and loading; Import the GIS data into the Cesium 3D world terrain. Cesium performs tile cutting on the high-precision terrain data and stores the tile data in the cloud server, loading the tiles required for rendering; Perform spatial coordinate transformation on the terrain data using a transformation algorithm, and render the transformed data in the cloud using the UE5 engine; Encode and package the UE5 rendering result into a media stream, generating a playable media stream environment provided by the browser point-to-point; The front-end page receives and displays the transmitted rendered screen, while simultaneously monitoring interactive commands in real time, focusing, simulating motion effects and distance measurement on the UE scene, performing interactive rendering operations according to the front-end commands, and linking the rendering results with the front-end in real time, synchronizing them to the front-end page.

[0031] The core logic of the system architecture of this invention is to migrate heavy rendering computation to the cloud, achieving a closed loop of "cloud rendering - streaming transmission - front-end interaction" through pixel streaming. Based on the design principle of "layered decoupling and collaborative linkage," it is divided into four main parts: data processing, cloud rendering, streaming transmission, and visual interaction. Figures 1-6 As shown, each layer independently undertakes a specific function, and achieves efficient flow of data and instructions through standardized interfaces.

[0032] Specifically, the front-end and back-end interaction architecture constructed in this invention adopts a layered decoupling approach, systematically stripping and standardizing data layer by layer to separate scene rendering and page loading, and loading and removing scene loads on demand. Long-sequence data is transmitted using a chunked transmission and reassembly mechanism, and WebSocket binary frame transmission ensures ordered data reassembly and integrity verification. The front-end and back-end synchronously maintain a device information status synchronization table to ensure consistent status of power equipment information.

[0033] Specifically, the message transmission mechanism designed in this invention automatically fragments the data instruction set, attaching 16 bytes of checksum information to each fragment. A retransmission mechanism utilizes WebSocket binary frames of pixel streams for fragment transmission. The cloud performs integrity verification after receiving each fragment. The UE maintains a fragment cache pool based on data indexes and records missing fragment numbers. When three consecutive fragments are received, a data feedback and missing fragment retransmission mechanism is activated. After confirming the completeness of all fragments, integrity is verified through MD5 checksum before data reassembly. When the front-end connects to the middleware, the ID of incomplete data in the cache is queried for continued transmission. The window size is dynamically adjusted via commands to avoid data congestion.

[0034] The specific execution flow of this invention is as follows:

[0035] 1. Data processing:

[0036] It should be noted that Cesium is the core layer for GIS data format conversion and lightweighting in the data processing layer, providing support for cloud rendering. GIS data refers to a collection of digital descriptions of the spatial location, attribute characteristics, and spatiotemporal relationships of various geographic entities and phenomena on the Earth's surface. Its core classifications are mainly vector data and raster data, with raster data primarily used for the wide-area overlay and rapid rendering of 3D terrain scene base maps. The specific execution flow is as follows:

[0037] Data Processing and Access—For massive geographic terrain data, Cesium employs a strategy of chunking, layering, and on-demand loading. It precisely locates and stitches high-precision terrain data by cutting it into tiles. Once the tiles to be loaded are determined, Cesium makes targeted online HTTP requests to a web server based on the tile type and data. The requested tile data is not directly generated; it undergoes coordinate mapping and stitching to form the final 3D Earth surface. This multi-layered detail approach improves tile loading quality and speed. The core of this approach is automatically switching tile layer resolution based on distance, avoiding performance waste while ensuring clarity of nearby terrain.

[0038] Spatial coordinate transformation—integrating the Cesium framework into the UE engine, its core includes full-size Earth creation, high-resolution terrain surface rendering, spatial coordinate transformation, and 3D content visualization and interaction. Spatial coordinate transformation consists of the following steps:

[0039] Spatial coordinate transformation first converts the WGS84 coordinate system (composed of latitude and longitude) to a geocentric-fixed coordinate system, thus digitizing the abstract data from the spherical coordinate system and achieving the transformation to ECEF coordinates. The specific coordinate transformation formula is as follows:

[0040]

[0041] The ellipsoid curvature is calculated using the formula above, converting geographic coordinates to Cartesian coordinates. Here, 'a' represents the length of the semi-major axis in WGS84, and 'f' is the flattening. Let be the radius of curvature. Then, the geographic coordinates are converted to Cartesian three-dimensional coordinates. The specific coordinate transformation formula is:

[0042]

[0043] in, This represents the xyz values ​​of Cartesian coordinates, where lon, lat, and h represent latitude, longitude, and altitude in the geographic coordinate system, respectively.

[0044] The core of the conversion from geographic coordinates to UE spatial coordinates is achieved through coordinate mapping and adaptation. ECEF coordinates and UE coordinates differ in units and direction. Cesium further processes these differences through affine transformations to form the root coordinates supported by the UE. Since the coordinates of objects in the UE are local coordinates, a matrix transformation is superimposed to form the final object coordinates. The specific coordinate transformation formula for converting Cartesian coordinates to UE coordinates is:

[0045]

[0046] Where xyz represents the final transformed coordinates, This represents the xyz values ​​of the UE's root coordinates.

[0047] Furthermore, to unify the axial coordinate system, the rigid body posture of the camera under a specified coordinate orientation is adjusted. The coordinate transformation method described above is used to adjust the camera to a suitable Euler angle viewport orientation, facilitating subsequent motion effect interaction analysis and reasonably presenting a precisely positioned and realistic twin-world content. The specific coordinate transformation formula is as follows:

[0048]

[0049] in, This refers to the relative offset of the camera coordinates, and R refers to the composite rotation matrix. These are the camera's final coordinates. This is the camera's initial position.

[0050] 2. Cloud rendering:

[0051] It should be noted that UE5 serves as the central computing resource hub for the cloud processing layer, ensuring that terrain and scene devices can achieve cinematic real-time rendering effects. It combines high-precision geographic data with UE's high-fidelity rendering capabilities, centrally handling three core tasks: 3D scene rendering, model management, and interactive commands. Specific execution flow:

[0052] Initialize the scene: Store tile data on a cloud server, set the URL in Cesium3DtileSet to the path of a JSON file, add the CesiumTileMapServiceRasterOverlay component, and add the grid structure to the URL. Load the PixelStreaming plugin and set the encoder to H.264. Configure the Streamert Port and Viewer Port for the pixel stream signaling server, and then stream the complete editor.

[0053] Viewport control: Add CesiumDanamicPawn, set Longitude, Latitude, and Height, set camera properties and switch viewport mode to Perspective, and set camera pitch, heading, row, and FOV based on the parsed front-end messages.

[0054] Device Identifier: It should be noted that the UE underlying layer only supports adding a unique identifier to an Actor using SetActorLabelUnique or obtaining a unique name using GetDisplayName during debugging. However, this identifier is uncontrollable and cannot be specified during the packaged runtime phase. To address the above issues and achieve unified management of all Actors, this invention designs an appending identifier mechanism. This mechanism performs secondary processing on the model and iteratively appends Labels as unique Tag identifiers to Actors and ActorMesh under the model's device tree structure. This makes the model's identifier attributes controllable and unique without affecting the UE's internal operating mechanism.

[0055] Model Loading: It should be noted that when loading and rendering large models, Cesium supports loading glTF format models. However, due to limitations in the number of times the model is modified and its size, the rendering and loading capabilities of the UE itself are somewhat reduced. In contrast, DataSmith format models can accurately preserve the geometric information of the model while adapting and optimizing it, controlling the quality of static mesh resources. It is a tool specifically designed by the UE to optimize the problem of slow and bloated model loading.

[0056] Message Interaction: The PixelStreamingInput component is introduced into the UE engine and bound to input events to respond to messages sent from the front end. The UE uses the message parsing mechanism described in this invention to identify, verify and merge the received messages, and then parse the corresponding attribute field values ​​to ensure that the instructions transmitted to the UE through the WebRTC channel are complete and accurate, reducing the forwarding delay and packet loss risk caused by small frequency band data. Finally, messages are sent to the front end Vue according to the event response.

[0057] Animation Generation: To simulate the state of a real-world power scenario in real time, the twin scene needs to generate animations based on the parsed front-end instructions. For example, when analyzing power equipment, it is necessary to disassemble the equipment with animation. Limited by the dynamic loading of glTF or FBX models and their animations during runtime in UE pixel stream mode, the glTFRunTime plugin can dynamically load models and animations, but its underlying mechanism does not accurately distinguish between animation loading under skeleton mesh and static mesh. The loading mechanism designed in this invention differentiates and optimizes the mesh type during the loading process, allowing it to load multiple corresponding animation frames under skeleton mesh and static mesh respectively, generating disassembly animation effects step by step. The specific disassembly formula is as follows:

[0058]

[0059] in, Indicates the final location of the disassembled module. This indicates the initial position of the disassembly module, d is the disassembly distance, and c is the model position.

[0060] Image rendering: After the program runs, the UE server encodes the rendered frames into H.264 video streams through its own Nanite rendering pipeline according to the current scene state. The signaling server maintains the connection with the browser and the UE program and pushes the stream to the signaling server, which then forwards it to the front end.

[0061] 3. Streaming:

[0062] It should be noted that the pixel stream transport layer is the "transmission hub" between the cloud and the front end, constructing a bidirectional channel for "downlink + uplink" pixel stream commands. On one hand, it transmits cloud-rendered images to the front end at low cost through adaptive encoding and compression; on the other hand, it transmits user input to the cloud in real time after standardized encoding and verification, using mechanisms such as packet loss compensation, thus solving the key problem of "efficient transmission of large-volume rendering". The specific execution process is as follows:

[0063] Initialize the connection: Open port 80 on the host. The client browser initiates an HTTP request to access the pixel stream service. The front-end page connects to the signaling server via WebSocket. The signaling server uses the default port 8888 to receive all incoming connection requests from the UE engine, assisting the UE server in exchanging SDP information with the client, including supported encoding formats. Configure the pixel stream transmission IP address under the LAN for subsequent pixel stream transmission. Run setup.bat in administrator mode under SignalallingWebServer to install necessary dependencies, then start_with_run.bat to connect to and run the UE engine program on the signaling server.

[0064] Screen transmission: When an Unreal Engine instance is running on a computer, it broadcasts media streams over the network to multiple connected devices. The pixel stream transmits the UE screen to the client browser via the WebRTC real-time audio and video transmission protocol, generating a stable screen with low latency, packet loss compensation, and jitter buffering.

[0065] Command transmission: It should be noted that the pixel stream underlying layer supports a limited message size. Through the message expansion mechanism of this invention, long front-end commands are transmitted to the UE through the WebRTC channel. Then, the processing logic is executed in the form of the UE's own system blueprint or code to generate new screen frames, which are transmitted to the front-end page through the pixel stream.

[0066] 4. Visual interaction:

[0067] It should be noted that the front-end visual interaction layer is the user's "interaction terminal." Built on WebGL technology, it's a plugin-free front-end framework that requires no UX installation. It receives pixel streams and displays 3D visuals solely through a browser, while simultaneously handling tasks such as standardized input and network quality feedback to ensure a smooth user experience. The specific execution flow is as follows:

[0068] Page initialization: The frontend downloads the lib-pixelstreamingfrontend module via npm, establishes a long connection with the signaling server via Config.ts on the default port 8888, and then makes requests and verifications in JSON format using {"type":"...","xxx":"..."}. It selects the link with the lowest latency to establish a P2P connection. After that, the browser loads the UI and style CSS components, configures the H.264 encoding format parameters for the video stream, and initializes the canvas.

[0069] Video stream reception: After establishing a WebRTC connection, the UE will push the rendered Cesium geographic scene to the front end in the form of a video stream. The video stream sent by the UE is transmitted via UDP in the form of RDP. The WebRTC module of the front-end browser is responsible for receiving and reassembling these streaming data packets. Then, the browser uses a hardware decoder to decode the H.264 video stream. The video stream is monitored by the videoContainer and dynamically rendered onto the canvas.

[0070] Message Interaction: The front-end page captures user actions through native JavaScript event listeners and converts the commands into JSON format for transmission to the back-end via webRtcController.emitUIInteraction. It should be noted that the pixel stream's underlying maxmassage is limited to 1024 bytes. To overcome this data length limitation, a long message transmission mechanism is used to optimize the data structure of the command message descriptor. Simultaneously, a bidirectional heartbeat data synchronization mechanism using windows.setInterVal ensures that the UE and the front-end maintain a unified "information status synchronization table."

Claims

1. A two-dimensional and three-dimensional fusion modeling and interaction method based on a UE engine, characterized in that, Specifically, the following steps are included: (1) Collect GIS data related to the power grid, including the latitude and longitude of the scene and equipment, elevation data, and grid structure; (2) Based on the GIS data, the power grid model is modeled at the same scale and with high precision, and stored in the cloud model library. The model includes towers, lines, fittings, and substations. (3) Import the model library into the UE5 engine for model matching and loading; (4) Import the GIS data into the Cesium 3D world terrain. Cesium will cut the high-precision terrain data into tiles and store the tile data in the cloud server and load the tiles required for rendering. (5) Transform the terrain data into spatial coordinates using a transformation algorithm, and then render the transformed data in the cloud using the UE5 engine; (6) Encode and package the UE5 rendering results into a media stream, and generate a playable media stream environment provided by the browser in a point-to-point manner; (7) The front-end page receives and displays the transmitted rendering screen, and at the same time monitors the interaction instructions in real time, performs focus, motion effect and distance measurement simulation on the UE scene, performs interactive rendering operation according to the front-end instructions, and links the rendering results with the front-end in real time and synchronizes them to the front-end page.

2. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 1, characterized in that, The model has a pre-added custom unique label, which replaces the UE5 default label.

3. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 1, characterized in that, Cesium provides online HTTP targeted request services to web servers based on tile type and data. The requested tile data is then mapped and stitched together to form a three-dimensional Earth surface, and the tile layer resolution is automatically switched according to the distance.

4. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 1, characterized in that, The spatial coordinate transformation includes: (51) Convert the WGS84 spherical coordinate system data to ECEF coordinates; (52) Convert the ECEF coordinates to the root coordinates supported by UE5; (53) Adjust the rigid body attitude of the camera in the UE5 view.

5. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 1, characterized in that, The cloud rendering specifically includes performing secondary processing on the model to add controllable identifiers, preserving the model's geometric information, controlling the quality of static mesh resources, and adjusting the encoding of rendering frames according to the rendering pipeline.

6. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 1, characterized in that, The media streaming environment includes constructing a bidirectional channel for pixel stream instructions, adaptively encoding and compressing the cloud-rendered screen based on a long message transmission and parsing mechanism and transmitting it to the front end, and transmitting user input to the cloud in real time after standardized encoding, verification and compensation mechanisms.

7. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 6, characterized in that, The long message transmission and parsing mechanism includes message segmentation, compression and streaming transmission, and decoding and integrity verification of segmented data through adaptive compression encoding rules and verification algorithms.

8. The two-dimensional and three-dimensional fusion modeling and interaction method based on a UE engine according to claim 1, characterized in that, The front-end page is a plugin-free front-end framework page built on WebGL technology. It receives pixel streams through the browser and displays 3D images, and handles standardized input and network quality feedback tasks.

9. The two-dimensional and three-dimensional fusion modeling and interaction method based on the UE engine according to claim 8, characterized in that, The pixel stream consists of data packets sent by the UE in RDP format. These packets are transmitted via UDP to the WebRTC module of the front-end browser, where they are reassembled. The WebRTC module uses a hardware decoder to decode the H.264 video stream, which is then monitored by a videoContainer and dynamically rendered onto the canvas.

10. A two-dimensional and three-dimensional fusion modeling and interactive system based on a UE engine, characterized in that, The system includes: Data processing layer: Based on the Cesium framework, it processes GIS data format conversion and data lightweighting to support cloud rendering, including full-size earth creation, high-resolution terrain surface rendering, and spatial coordinate transformation. The data processing layer converts spatial coordinates into adaptive local coordinates under UE, realizes adaptive rigid body posture adjustment and Euler angle viewport orientation, and supports the UE engine in interactive analysis of subsequent motion effects and presentation of realistic twin world content. Cloud rendering layer: Using UE5 as the computing power resource hub, it performs secondary processing on the model, adds controllable identifiers, accurately preserves the geometric information of the model and controls the quality of static mesh resources, adjusts the encoding of rendering frames according to the rendering pipeline, loads and pushes the large model for rendering, and performs command animation analysis through long message transmission and parsing mechanism to simulate the state of real power scene in real time. Streaming layer: The link layer between the front end and the back end. It builds a two-way channel for pixel stream instructions, transmits cloud-rendered images to the front end through adaptive encoding and compression, and transmits user input to the cloud in real time through standardized encoding, verification and compensation mechanisms. Visual interaction: As a user interaction terminal, it processes captured user input into standardized input, optimizes the data structure of instructions through long message transmission mechanism, receives and reassembles laminar data based on WebRTC module, and dynamically renders it to the browser page.

Images