Digital twinning application development method and device and electronic equipment

Abstract

The invention discloses a digital twin application development method and device and electronic equipment. The method comprises the steps that in response to a creation instruction, a digital twin project task is constructed, and the digital twin project task comprises a three-dimensional scene which is rendered in a virtual environment and reflects the real world; a pixel stream transmission strategy corresponding to the digital twin project task is determined, and the pixel stream transmission strategy is at least used for converting the three-dimensional scene into pixel streams meeting the requirements of different terminal devices; and transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy. According to the method and the device, the technical problem that the development cost is increased due to the fact that a developer needs to develop and maintain applications for different platforms because digital twinning applications developed by related technologies are limited to a single platform is solved.

Description

Technical Field

[0001] The present application relates to the field of digital twin technology, and more specifically, to a method, device, and electronic device for developing digital twin applications. Background Art

[0002] Digital twins are a technology that creates a fully digitalized representation of an object or system in the virtual world. This model reflects the physical object's state in real time and can simulate and predict its behavior and performance using both historical and real-time data. Digital twins combine advanced technologies, including the Internet of Things (IoT), artificial intelligence (AI), machine learning, software analytics, and sensor technology, to seamlessly connect the physical and digital worlds.

[0003] With the rapid development of CIM (City Information Modeling) digital twins in recent years, more and more industries have begun to use CIM digital twins to reduce costs and enhance user experience. However, CIM digital twin applications developed with related technologies are mostly limited to a single platform (such as PC or mobile) and lack cross-platform compatibility and consistency. As a result, developers need to develop and maintain applications separately for different platforms, increasing development costs and workload.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, and electronic device for developing a digital twin application, so as to at least solve the technical problem that the digital twin applications developed by related technologies are limited to a single platform, which requires developers to develop and maintain applications separately for different platforms, thereby increasing development costs.

[0006] According to one aspect of an embodiment of the present application, a method for developing a digital twin application is provided, comprising: constructing a digital twin project task in response to a creation instruction, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment; determining a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least used to convert the three-dimensional scene into a pixel stream that adapts to the requirements of different terminal devices; and transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

[0007] In some embodiments of the present application, determining a pixel stream transmission strategy corresponding to a digital twin project task includes: obtaining device information of at least one terminal device, wherein the device information includes decoding capabilities and network transmission capabilities of at least one terminal device; determining a codec parameter configuration based on the device information and rendering information of the three-dimensional scene, wherein the codec parameter configuration is used to adapt to the network transmission capabilities of different terminal devices, and the rendering information is used to reflect the real-time rendering requirements of the three-dimensional scene; determining a communication protocol configuration based on the device information, wherein the communication protocol configuration includes the address of a signaling server, and the signaling server is used to establish a communication channel with the terminal device; determining a pixel stream transmission strategy corresponding to the codec parameter configuration and the communication protocol configuration.

[0008] In some embodiments of the present application, constructing a digital twin project task includes: obtaining target data, wherein the target data includes three-dimensional model files in different formats, and the three-dimensional model files are used to map physical entities in the real world; determining a parser corresponding to the format of the three-dimensional model file, and using the parser to extract target information of the three-dimensional model file, wherein the target information includes one of the following: vertex coordinates, normals, texture coordinates and material information; converting the target information into model data in a preset format, and constructing the digital twin project task based on the model data.

[0009] In some embodiments of the present application, it also includes: receiving editing instructions, wherein the editing instructions are used to instruct editing operations on objects in a three-dimensional scene, and the editing operations include one of the following: changing position, adjusting size, and modifying material properties; determining an identifier corresponding to the object from the editing instructions; editing the target object corresponding to the identifier according to the editing instructions, and synchronizing the editing results to at least one terminal device.

[0010] In some embodiments of the present application, a digital twin project task is transmitted to at least one terminal device based on a pixel stream transmission strategy, including: determining status data corresponding to multiple objects in a three-dimensional scene in the digital twin project task; rendering multiple objects based on the status data to obtain a video frame sequence; encoding the video frame sequence into a video stream, and transmitting the video stream to at least one terminal device.

[0011] In some embodiments of the present application, transmitting a video stream to at least one terminal device includes: responding to a connection request from a target terminal device, establishing a connection with the target terminal device and sending encoding parameters to the target terminal device, wherein the encoding parameters are determined according to the state of the connection, the encoding parameters include the resolution and frame rate of the video stream, and the target terminal device is any one of the at least one terminal device; sending the video stream to the target terminal device through the connection.

[0012] In some embodiments of the present application, after transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy, the method also includes: receiving an operation instruction from at least one terminal device, wherein the operation instruction includes an editing operation performed by the target object on the scene copy on the terminal device, the scene copy is a copy of the three-dimensional scene, and each terminal device corresponds to a scene copy; updating the status data of the three-dimensional scene according to the operation instruction, and determining an updated scene copy corresponding to the updated three-dimensional scene; and transmitting the updated scene copy to at least one terminal device.

[0013] According to another aspect of the embodiment of the present application, a development device for a digital twin application is also provided, including: a creation module for constructing a digital twin project task in response to a creation instruction, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment; a determination module for determining a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least used to convert the three-dimensional scene into a pixel stream that adapts to the requirements of different terminal devices; a transmission module for transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

[0014] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory and a processor, the memory being used to store program instructions; the processor being connected to the memory and being used to execute the development method for implementing the above-mentioned digital twin application.

[0015] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned digital twin application development method by running the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-mentioned method for developing digital twin applications.

[0017] In an embodiment of the present application, a method of dynamically adjusting the pixel stream transmission strategy is adopted. By converting the three-dimensional scene in the digital twin project task into a pixel stream that adapts to the needs of different terminal devices for transmission, the purpose of more widely applying digital twin applications to various devices and scenarios is achieved, thereby achieving the technical effect of reducing the development complexity and cost of digital twin applications and realizing the unification and efficient collaboration of multiple platforms, and thus solving the technical problem that the digital twin applications developed by related technologies are limited to a single platform, which requires developers to develop and maintain applications for different platforms separately, increasing development costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer terminal according to a method for developing a digital twin application in accordance with an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for developing a digital twin application according to an embodiment of the present application;

[0021] Figure 3 This is an overall flow chart of a method for developing a digital twin application according to an embodiment of the present application;

[0022] Figure 4 This is a system architecture diagram of a method for developing a digital twin application according to an embodiment of the present application;

[0023] Figure 5 It is a structural diagram of a digital twin application development device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0027] CIM (City Information Modeling): A comprehensive urban modeling and management technology that integrates Geographic Information Systems (GIS), Building Information Modeling (BIM), and other related technologies to create detailed three-dimensional city models and their information management systems. In this application, digital twin project tasks can be built based on CIM to obtain a digital platform that comprehensively reflects the physical and non-physical elements of the city to support various applications such as urban planning, construction, operation, and management.

[0028] Digital Twin: A technology that simulates a digital version of a physical entity or system. By collecting and integrating real-time data from physical objects, it creates a highly accurate virtual model of them. In this application, digital twin technology can be applied to CIM, for example, to enable real-time simulation and data analysis of cities, improving the level of intelligent urban management.

[0029] 3D rendering (Three-dimensional Rendering): The process of converting a three-dimensional model into a two-dimensional image, involving the simulation of effects such as lighting, shadows, and reflections. In this application, 3D rendering is used to achieve visual realism in the digital twin, ensuring that users can clearly view the details of the city model from any angle.

[0030] Digital twin technology uses real-time interaction between the physical and virtual worlds to digitally model physical entities and manage them throughout their lifecycle. Digital twin technology has a wide range of applications in the industrial sector, such as factory automation system management, industrial production line optimization, and remote equipment diagnostics. Digital twin applications enable full lifecycle management of physical entities through virtualization, eliminating the need for physical investment. The development process for digital twin applications includes building a 3D scene model of the physical entity, defining the physical entity's operating rules, and then generating a digital twin application based on the physical entity's operating rules and 3D scene model. However, the development frameworks used in these technologies are limited to a single platform (such as PC or mobile), lack cross-platform compatibility and consistency, and are limited by hardware performance. This inability to deliver the desired digital twin results hinders widespread deployment and use. Furthermore, these technologies suffer from numerous shortcomings in 3D rendering, data synchronization, resource management, and security, failing to meet the growing demand for cross-platform applications.

[0031] In order to solve the above technical problems, the embodiments of the present application provide corresponding solutions, which are described in detail below.

[0032] The digital twin application development method embodiment provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal for implementing a development method for digital twin applications is shown. Figure 1 As shown, the computer terminal 10 may include one or more (illustrated by 102a, 102b, ..., 102n in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0033] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the development method of the digital twin application in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the development method of the digital twin application described above. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0035] The transmission module 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0037] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0038] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a method for developing a digital twin application. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 2 is a flow chart of a method for developing a digital twin application according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0040] Step S202 , in response to the creation instruction, construct a digital twin project task, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment.

[0041] In the above step S202, in the digital twin development framework, the creation instruction can be regarded as a signal initiated by a user or system, which is used to instruct the framework to start creating a digital twin project for a specific city or place. This signal can include basic configuration information of the project, such as name, location, required data set, etc.

[0042] A digital twin project task is a collection of specific operations and goals generated based on creation instructions, aiming to build a digital twin that reflects a real-world 3D scene. Project tasks include, but are not limited to, data collection, model building, scene rendering, and function implementation. For example, a digital twin project task might be a city information model application built using CIM and digital twin technology, including all 3D models, real-time data, and user interaction features.

[0043] A virtual environment refers to a software platform provided by the digital twin application development framework for rendering three-dimensional scenes. For example, it can be built based on a game engine (such as Unreal Engine) to simulate the physical, environmental, and behavioral characteristics of the real world.

[0044] A 3D scene is a three-dimensional model of a city or place constructed in a virtual environment by integrating GIS, BIM, IoT (Internet of Things) data, etc. For example, a 3D model can include elements such as buildings, roads, vegetation, terrain, as well as real-time dynamic data (such as traffic flow, climate conditions, energy consumption, etc.) to reflect the multi-dimensional information of the real world.

[0045] In some embodiments of the present application, raw data from GIS, BIM, and IoT can be collected and converted into a format and structure suitable for 3D rendering; the collected data is constructed into a three-dimensional model, and the model can be optimized through technologies such as Lumen (a fully dynamic global illumination system that can simulate the transmission and reflection of light in a real-time environment) and Nanite (a virtualized geometry technology that can directly import and render hundreds of millions of polygons); the optimized 3D model is rendered into a high-quality three-dimensional scene, and the performance differences of different terminal devices are taken into consideration to adaptively transmit the pixel stream.

[0046] When building a digital twin project task, the system needs to process 3D model files in multiple formats, such as FBX, OBJ, GLTF, etc. These files come from different design software or scanning devices. In order to improve the efficiency of subsequent rendering and transmission, the following steps can also be performed: obtain target data, where the target data includes 3D model files in different formats, and the 3D model files are used to map physical entities in the real world; determine the parser corresponding to the format of the 3D model file, and use the parser to extract the target information of the 3D model file, where the target information includes one of the following: vertex coordinates, normals, texture coordinates and material information; convert the target information into model data in a preset format, and build a digital twin project task based on the model data.

[0047] A three-dimensional model file is an electronic file used to describe the shape, size, and position of an object in three-dimensional space. It can contain the object's geometric information (such as vertex coordinates, normals) and visual information (such as texture coordinates, material information), etc. It is the basic 3D model resource for building digital twin projects. The formats supported by three-dimensional model files include but are not limited to FBX, OBJ, GLTF, etc., which are used to map physical entities in the real world.

[0048] A parser is a software component that can read, understand, and extract specific data from a 3D model file. In some embodiments of this application, the parser can determine and load the corresponding parsing module based on different 3D model file formats to obtain detailed information about the model's geometry, textures, materials, and so on. Within a 3D model file, the target information contains key data required to build and render the model, such as vertex coordinates used to determine the model's shape, normal information used to calculate lighting effects, texture coordinates used for mapping, and material information that defines the physical properties of the model's surface, such as reflectivity and transparency.

[0049] The method also includes: receiving an editing instruction, wherein the editing instruction is used to instruct an editing operation on an object in a three-dimensional scene, and the editing operation includes one of the following: changing the position, adjusting the size, and modifying the material properties; determining an identifier corresponding to the object from the editing instruction; editing the target object corresponding to the identifier according to the editing instruction, and synchronizing the editing result to at least one terminal device.

[0050] An editing instruction is a command issued by a user through a terminal device to instruct an editing operation on a specific object in a digital twin three-dimensional scene. The operation includes but is not limited to changing the object position, adjusting the size or modifying the material properties. The editing instruction is the main way for users to interact with the digital twin environment, enabling users to adjust the entities in the scene as needed, achieve personalized settings or conduct professional analysis. In some embodiments of the present application, the user's editing request can be received through the front-end user interface. Specifically: intuitive editing tools are designed on the user interface, such as drag bars, rotation handles and material selectors, to ensure that users can easily issue editing instructions; an event monitoring mechanism is used on the front end, such as mouse clicks or touch screen operations. After these events are triggered, the editing instructions will be transmitted to the back-end server in the form of data packets.

[0051] The identifier is a unique code assigned to each object in the digital twin scene to facilitate system identification and management. In editing operations, the identifier is used to clearly indicate which specific object is the target of the operation. On the server side, after receiving the editing instruction, it can parse it and extract the object identifier and other editing parameters.

[0052] Step S204: determine the pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least used to convert the three-dimensional scene into a pixel stream that adapts to the requirements of different terminal devices.

[0053] In the above step S204, the pixel stream transmission strategy refers to a strategy for determining how to convert a 3D scene into a pixel stream and transmit it based on objectives such as the hardware capabilities of the terminal device, network conditions, and user needs.

[0054] In some embodiments of the present application, a client-server architecture is adopted to achieve widespread distribution and access to high-performance 3D graphics content. In this process, the rendering calculation of the 3D scene is completed by the server side equipped with a powerful GPU. For example, modern graphics APIs such as DirectX or Vulkan are used for efficient rendering. The rendered picture frames are then compressed through video codec standards such as H.264 or HEVC to reduce bandwidth requirements and transmitted to the client through WebRTC, RTMP or other real-time communication protocols. For client devices, such as personal computers, mobile devices or web browsers, only basic decoding capabilities and network connections are required to present high-quality 3D content through lightweight components such as HTML5 Video tags.

[0055] Specifically, the pixel stream transmission strategy corresponding to the digital twin project task can be determined in the following manner: obtain the device information of at least one terminal device, wherein the device information includes the decoding capability and network transmission capability of at least one terminal device; determine the codec parameter configuration based on the device information and the rendering information of the three-dimensional scene, wherein the codec parameter configuration is used to adapt to the network transmission capability of different terminal devices, and the rendering information is used to reflect the real-time rendering requirements of the three-dimensional scene; determine the communication protocol configuration based on the device information, wherein the communication protocol configuration includes the address of the signaling server, and the signaling server is used to establish a communication channel with the terminal device; determine the pixel stream transmission strategy corresponding to the codec parameter configuration and the communication protocol configuration.

[0056] Device information includes hardware specifications (such as GPU and CPU model), operating system information, screen resolution, network connection type (Wi-Fi, 4G / 5G), and performance metrics (such as downlink / uplink speed and latency). This information is used to assess the device's decoding and network transmission capabilities. Decoding capabilities determine whether the device can successfully decode the video stream transmitted from the server. Different devices may support different codec formats, such as H.264, HEVC, and VP9.

[0057] In order to solve the problem of high-definition video streaming transmission being difficult to maintain under different network conditions, and the problem of terminal devices freezing due to excessive processing pressure, the bit rate, resolution and other parameters of video encoding can be dynamically adjusted according to device information and real-time 3D scene rendering requirements to balance image quality and network bandwidth. In addition, based on the decoding capabilities of the terminal device, the most suitable encoding format is selected to ensure fast decoding speed and high quality of the video stream.

[0058] When configuring the communication protocol, you can pre-configure the signaling server address on the server side to eliminate the need to search for it each time a connection is established, speeding up communication establishment. During transmission, you can choose a communication protocol such as WebRTC, RTMP, or HLS based on network transmission capabilities and real-time requirements. This configuration ensures stable and efficient communication between the server and terminal devices, especially in scenarios with multiple users accessing the network simultaneously, ensuring real-time and synchronous data transmission.

[0059] Step S206: Transmit the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

[0060] In the above step S206, streaming media data can be generated according to the pixel stream transmission strategy and then transmitted to the terminal device. Specifically, on the server side, a graphics API such as DirectX or Vulkan can be used to render the three-dimensional scene in real time, and then encoded using a high-efficiency video coding format such as H.264 or HEVC to generate a video stream suitable for network transmission. Based on WebRTC or a similar real-time communication protocol, adaptive bit rate video streaming transmission is achieved, and the video quality is dynamically adjusted according to the network status of the terminal device to ensure that a good video experience can be maintained under various network conditions. A real-time communication connection with the terminal device is established and maintained through a configured signaling server to ensure stable transmission of streaming media data. In addition, the multiplexing function of WebRTC can be used to simultaneously transmit video streams, audio streams and control signals to achieve high-quality synchronous communication and low-latency user interaction.

[0061] With the increase in terminal devices and the complexity of the network environment, how to maintain the real-time and consistency of the digital twin project on each device has become a challenge. In order to solve this problem, WebRTC signaling and data transmission can be used. The WebRTC protocol supports point-to-point (P2P) data transmission, but in this application, a signaling server can be used to establish a centrally managed network to ensure that all terminal devices can keep in sync with the server through the real-time communication protocol. In this way, even when the network fluctuates, the intelligent routing and retransmission mechanism of the signaling server can maintain stable data transmission and synchronization. In addition, a multi-client synchronization mechanism can also be used. For example, when the server transmits the pixel stream, it records the rendering timestamp of each frame. After the terminal device receives the pixel stream, it sorts and displays the video frames according to the timestamp to ensure that the scenes seen by all users are synchronized. It should be noted that if a user has a large network delay, the server can provide him with additional cached frames to compensate for the impact of the delay, thereby maintaining a real-time experience for all users.

[0062] The digital twin project task is transmitted to at least one terminal device according to the pixel stream transmission strategy, including: determining the status data corresponding to multiple objects in the three-dimensional scene in the digital twin project task; rendering the multiple objects according to the status data to obtain a video frame sequence; encoding the video frame sequence into a video stream, and transmitting the video stream to at least one terminal device.

[0063] In digital twin projects, state data refers to a collection of real-time information about each object in a 3D scene, including but not limited to its position, rotation, scale, material properties, animation status, etc. In some embodiments of this application, by monitoring the changes of objects in a 3D scene in real time, each object's state information, such as real-time position updates, lighting changes, and actions generated by user interactions, can be collected and recorded.

[0064] A video frame sequence is a sequence of continuous image frames, each of which represents the rendering result of a three-dimensional scene at a certain point in time. Video stream encoding is the process of converting a video frame sequence into a digital signal that can be transmitted over a network. In some embodiments of the present application, Unreal Engine's Lumen global illumination system and Nanite virtualized micropolygon geometry technology can be used to perform real-time, high-quality rendering of a three-dimensional scene to generate a video image for each frame. During the rendering process, in order to achieve low-latency, high-fidelity 3D scene rendering while taking transmission efficiency into consideration, the rendering order and accuracy can be adjusted based on the importance of the object status data and user attention, and key objects can be prioritized to improve rendering efficiency and the smoothness of network transmission.

[0065] In some embodiments of the present application, a video stream can be transmitted to at least one terminal device through the following steps: in response to a connection request from a target terminal device, establishing a connection with the target terminal device and sending encoding parameters to the target terminal device, wherein the encoding parameters are determined according to the state of the connection, the encoding parameters include the resolution and frame rate of the video stream, and the target terminal device is any one of the at least one terminal device; and sending the video stream to the target terminal device through the connection.

[0066] Specifically, when the target terminal device initiates a connection request, the server immediately monitors its network conditions. For example, through the Ice Candidate exchange mechanism of WebRTC, the server can obtain the network quality of the target terminal in real time; use the WebSocket protocol or WebRTC to establish a two-way communication channel with the target terminal device to ensure that video stream transmission and control instructions can interact in a timely manner; the server dynamically determines the most suitable encoding parameters based on the hardware decoding capabilities and network transmission capabilities of the target terminal device. Encoding parameters refer to the configuration information used for video stream encoding, mainly including the resolution, frame rate, encoding format, etc. of the video stream; if the network conditions of the target terminal device are poor, the server can automatically reduce the resolution and frame rate of the video stream to adapt to the poor network environment and ensure that the video stream can be transmitted smoothly.

[0067] After transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy, the following steps can also be performed: receiving an operation instruction from at least one terminal device, wherein the operation instruction includes an editing operation performed by the target object on the scene copy on the terminal device, the scene copy is a copy of the three-dimensional scene, and each terminal device corresponds to a scene copy; updating the status data of the three-dimensional scene according to the operation instruction, and determining an updated scene copy corresponding to the updated three-dimensional scene; and transmitting the updated scene copy to at least one terminal device.

[0068] In digital twin projects, operation instructions are request messages sent to the server when users on terminal devices (such as computers, mobile phones, and tablets) interact with 3D scenes. These operations may include but are not limited to rotation, scaling, moving objects, property queries, video playback control, etc.

[0069] A scene copy is an independent instance of a 3D scene running on each terminal device. It contains the same data and status information as the main 3D scene, but can perform editing operations independently on its own device, ensuring that multiple users do not interfere with each other during collaboration.

[0070] In some embodiments of the present application, after receiving an operation instruction, the server can parse the editing operation in the instruction, update the status data of the three-dimensional scene in real time, and synchronize these updates to all scene copies through, for example, a CIM function module. It should be noted that if multiple users edit the same object simultaneously, the server adopts a first-come, first-served principle or sets conflict resolution rules, such as a priority mechanism, to ensure the consistency and accuracy of the status data. For example, when the server receives an operation instruction, it adds a timestamp to each instruction for subsequent conflict resolution and order determination of status data updates.

[0071] After updating the 3D scene status data, the server can generate a snapshot to record all changes, facilitate rapid reconstruction or synchronization of scene copies, and send status data update information to all associated scene copies to ensure that each copy reflects the latest editing results and maintain the consistency of multi-user collaboration.

[0072] Through the above steps S202 to S206, the pixel stream transmission strategy is dynamically adjusted. By converting the three-dimensional scene in the digital twin project task into a pixel stream that adapts to the needs of different terminal devices for transmission, the purpose of more widely applying digital twin applications to various devices and scenarios is achieved, thereby achieving the technical effect of reducing the development complexity and cost of digital twin applications and realizing the unification and efficient collaboration of multiple platforms, and thus solving the technical problem that the digital twin applications developed by related technologies are limited to a single platform, which requires developers to develop and maintain applications for different platforms separately, increasing development costs.

[0073] Figure 3 This is an overall flow chart of a method for developing a digital twin application according to an embodiment of the present application, such as Figure 3 As shown, in some embodiments of the present application, the development process may include the following steps:

[0074] Step S302: The development process begins, marking the starting point of digital twin application development.

[0075] Step S304: Create an Unreal Engine (UE) project and import necessary plugins and resources (such as fmap for UE). Through Unreal Engine, developers can use its advanced graphics rendering engine and rich development tools to build high-quality digital twin applications.

[0076] Step S306: Import art assets and 3D models. This step is the foundation of the application's visual presentation, ensuring that the digital twin scene is lifelike and detailed. For example, in the project's Content Browser, select the "Import" button, choose a supported file type (such as FBX, OBJ, or GLB), and set import options, such as merging meshes and creating skeletal meshes, to ensure that the assets can be correctly displayed and used in UE4.

[0077] Step S308: Apply Lumen and Nanite technologies. Lumen is the global illumination system in Unreal Engine, which can provide extremely realistic lighting effects. Nanite is a virtual geometry technology that can achieve efficient real-time rendering of complex models. These two technologies together improve the visual quality and performance of digital twin applications.

[0078] Step S310: In the Unreal Engine Content editor, add the CIM functional module and other required functional blueprints to the level. This is a key step in customizing application functionality. The blueprint system allows for intuitive building of the application's logical framework. For example, in the level blueprint editor, by dragging the CIM functional module and other functional blueprints into the level, you can use node programming to connect and configure the blueprint components to implement the scene's interactive logic and data processing.

[0079] Step S312: Create and configure the UI widgets and text information used by the project. Widgets are the building blocks of the user interface, and text information is an important medium for users to interact with the application. This step ensures the interactivity and usability of the application.

[0080] Step S314: Performance optimization, including but not limited to code optimization, resource management, and rendering optimization, to ensure that the application can run efficiently in different devices and network environments.

[0081] Step S316: Build and package the project to generate an application or executable file that can run on different platforms, including all the resources, codes, and configuration files, and provide standardized output for deployment on different platforms.

[0082] Step S318: Configure the pixel streaming environment, including server-side settings and client access methods, to lay the foundation for cross-platform access and real-time data transmission. For example, on the server side, deploy the execution environment of the digital twin application and services of WebRTC or similar real-time communication protocols. Configure the signaling server address, codec format, and network transmission protocol to ensure stable transmission of video streams and control signals.

[0083] Step S320: The user enters the pixel stream address in the browser and directly runs the digital twin application. This allows users to access complex applications on different devices without installing additional software or plug-ins. For example, when a user enters the pixel stream URL in the browser, the browser automatically establishes a connection with the server, begins receiving pixel stream data, displays the 3D scene of the digital twin application, and interacts with the server through communication methods such as WebSocket, enabling remote control and collaboration.

[0084] Step S322: The development and deployment process is completed, and the digital twin application is ready for user use.

[0085] Figure 4 This is a system architecture diagram of a method for developing a digital twin application according to an embodiment of the present application. Figure 4 As shown, in some embodiments of the present application, the system architecture may include the following parts:

[0086] (1) User interface layer 402, including terminal devices (PC and mobile web pages). The user interface layer is the front end of the digital twin application and the user interaction, including PC, mobile phone, tablet and other devices and web browsers. It is responsible for receiving and processing user input, displaying the 3D scene transmitted through the pixel stream, and providing an interface for interaction with the CIM functional module.

[0087] (2) Pixel stream 404, including pixel stream rendering and communication module 404a, and cross-platform operating environment module 404b, wherein:

[0088] The pixel stream rendering and communication module 404a includes a core engine (HTML5 / CSS3 / JS) and an adaptation layer (API interface), forming the underlying framework of the front-end interface. It is used to build and control the user interface, decode and render data transmitted via the pixel stream, and provide a bridge to technologies such as WebGL and WebRTC. It is responsible for converting the 3D scene rendered in real time by the server into a pixel stream and efficiently transmitting it via protocols such as WebRTC. It also provides real-time, two-way communication between the front-end and the server, ensuring that user operations are immediately fed back to the server and that server updates are synchronized to the front-end in real time.

[0089] The cross-platform operating environment module 404b includes a WebGL rendering engine, which is used to implement 3D graphics rendering in a Web environment, so that terminal devices can smoothly display 3D scenes transmitted by pixel streams; pixel stream transmission, that is, using video encoding and compression technologies such as H.264, HEVC, etc., to compress and transmit 3D scenes in real time, ensuring efficient data transmission under different network conditions; real-time communication protocol (WebSocket), which is used to provide a low-latency full-duplex communication channel, allowing the front-end and back-end to exchange data in real time, including but not limited to control signals, status information, etc.; compression algorithms, including video compression and data compression technologies, such as VP9, ​​H.265 and other video compression formats, and data compression algorithms such as gzip, bzip2, etc., which are used to reduce the amount of transmitted data and improve transmission efficiency.

[0090] (3) Functional module 406, including CIM functional module 406a (explained using a CIM-based digital twin project as an example), and 3D resource loading module 406b, wherein:

[0091] The CIM function module 406a focuses on the urban information management functions of digital twins, including object editing, attribute query, data visualization, measurement tools, video playback, and layer management. Through this module, users can edit 3D models, query object attributes, perform visual analysis on data, measure spatial parameters, play videos, and manage different layers to meet diverse needs and scenarios.

[0092] The 3D resource loading module 406b is responsible for loading, previewing, editing, and optimizing 3D models and resources. It supports multiple 3D formats, such as FBX, OBJ, GLTF, etc., provides model preview and editing functions, as well as performance optimization tools such as model simplification and texture compression, to ensure that applications can efficiently use and manage 3D resources.

[0093] Based on this system architecture, the CIM functional module, three-dimensional (3D) asset loading module, cross-platform pixel flow rendering and communication module are described in detail below.

[0094] The CIM functional module is an important component of the digital twin application development framework, focusing on CIM information and management. This module provides a wealth of tools and functions to help users manage and analyze building data in a digital twin environment (i.e., building digital twin project tasks). Specifically, it includes: 1) Object editing: provides 3D object editing functions, supporting operations such as rotation, scaling, and translation; 2) Property query: users can quickly find the property information of specific objects through the tag search function; 3) Data visualization: combines various types of data with real-time sensor data to generate reports and charts for easy analysis and decision-making; 4) Measurement tools: supports real-time measurement of projected area, continuous distance, and height; 5) Video playback: MP4 and RTSP video playback within the scene; 6) Layer management: display and hide layers, add or delete POIs, road markings, satellite images, and elevation image services, etc.

[0095] The CIM functional modules demonstrate the advanced and comprehensive nature of this technology in urban management and planning, enabling the construction of precise digital twin project tasks. First, the "Object Editing" function leverages BIM technology, enabling users to precisely create, edit, and optimize urban infrastructure models in three dimensions, ensuring detailed geometric and semantic information for each element. Second, through "Attribute Query," the system leverages database management and SQL technologies to provide users with an efficient interface for querying and analyzing the properties of urban objects, covering multi-dimensional information from physical characteristics to environmental impacts. Furthermore, the "Data Visualization" function combines GIS and 3D graphics rendering technology to transform abstract data into intuitive visual representations, supporting multi-scale and multi-level spatial data analysis, helping decision-makers quickly understand urban current conditions and development trends. Furthermore, the "Measurement Tool" integrates advanced spatial analysis algorithms, such as Euclidean distance calculation, area, and volume measurement, providing accurate spatial references for urban planning, architectural design, and engineering management. Finally, the "Video Playback" function seamlessly integrates video data with CIM scenarios by integrating streaming technologies such as RTSP and HLS, making it suitable for real-time data analysis in areas such as public safety and traffic monitoring. Finally, the “layer management” function is based on hierarchical storage and display technology, allowing users to flexibly control the visibility and hierarchical relationships of different data layers, enhancing the customizability and ease of use of the CIM system.

[0096] The 3D asset loading module is responsible for managing and loading 3D models and resources, ensuring that applications can efficiently use high-quality 3D content. The module provides a variety of loading and management tools, and supports a variety of 3D formats and resource types in the process of building digital twin project tasks. Specifically, it includes: 1) Resource loading: users can upload 3D models and resources by themselves, supporting multiple FBX, OBJ, and GLTF formats; 2) Resource preview: Provides a preview function for 3D models and resources, making it convenient for users to retrieve and view relevant information, such as memory usage and the number of mesh triangles; 3) Resource editing: Supports basic editing operations for resources, such as rotation, scaling, material adjustment, setting labels, etc.; 4) Performance optimization: Provides resource optimization tools to improve loading efficiency without reducing the quality of model rendering.

[0097] The 3D asset loading module is a key component of the digital twin application development framework that supports cross-platform access. Its primary task is to efficiently load, manage, and optimize 3D models and related resources. By supporting multiple 3D model formats (such as FBX, OBJ, and glTF), the module utilizes a built-in parser to extract vertex coordinates, normals, texture coordinates, and material information, ensuring that models of different formats can be processed uniformly. Furthermore, in terms of resource management, the module establishes a resource index that records the model's path, size, and dependencies. It also optimizes memory usage through caching mechanisms (such as the LRU algorithm) to reduce the time overhead of repeated loading. Furthermore, the dynamic loading strategy utilizes on-demand and asynchronous loading techniques to ensure that necessary resources are loaded only when needed, avoiding long initialization times and excessive memory usage. The module also provides model simplification tools (such as LOD technology) and texture compression algorithms (such as ETC2 and ASTC) to reduce computational and transmission overhead and improve rendering performance. The resource packaging tool merges multiple small files into a single large file, reducing the number of file I / O operations and further improving loading efficiency.

[0098] The cross-platform pixel stream rendering and communication module is responsible for handling the rendering and real-time communication of 3D scenes, ensuring high-quality graphics display and low-latency data transmission. This module is a key part of achieving cross-platform high-performance rendering. Specifically, it includes: 1) High-quality rendering: supports complex 3D models and special effects, providing realistic visual effects; 2) Multi-client synchronization: can achieve real-time synchronous display of the same 3D scene and support multi-person collaboration, and can also be deployed independently to achieve a single independent experience; 3) Interactive operations: supports interactive operations on the client (using mouse on PC and touch on mobile devices), such as rotation, zooming, clicking, etc.; 4) Two-way communication: a complete communication module that can implement WebSocket communication with the front-end, receive and send messages, implement various functions, and realize secondary development and custom interfaces; 5) Performance monitoring: provides performance monitoring tools to help developers optimize rendering and communication performance.

[0099] It should be noted that in order to optimize the user experience, the PixelStreaming technology in the cross-platform pixel stream rendering and communication module integrates an adaptive bitrate mechanism, which can automatically adjust the resolution and frame rate of the video stream according to the real-time network conditions to ensure a smooth viewing experience. In addition, it also supports multi-user synchronous views, allowing multiple clients to view different perspectives of the same 3D scene at the same time, which is suitable for multiple fields.

[0100] In order to facilitate the understanding of the development process of the above-mentioned digital twin application, it is explained below in combination with some existing embodiments based on the system architecture. The development process includes the editor stage and the runtime stage (the state or environment in which the program runs). In the editor stage, a new project is created by importing the integrated plug-in package. Communication is mainly achieved through BP_Link (blueprint component) to achieve two-way communication with the front end, and functions are achieved by referencing different functional blueprint modules BlueComponment (encapsulating specific functions or behaviors), such as measurement, pipeline generation, setting image service functions, etc. You only need to drag and drop the functions to be implemented into the level. The runtime stage uses the incoming interfaces and data to implement different business functions, providing a very convenient editing method. Even if the project has been deployed, you can modify any object in the project during the runtime state.

[0101] (1) CIM functional modules:

[0102] 1) Get the Pawn class (an object in Unreal Engine that represents a controllable character or agent) in the current level. When the left mouse button is clicked, set the Boolean value of MouseHold1 (a Boolean variable used to detect whether the left mouse button is pressed) to true. At this time, the axis mappings LookUpRate (mouse Y) and TurnRate (mouse X) in the project will take effect. At this time, get the current PlayerController (controller), get the control rotation node, and add the current rotation value to the increasing or decreasing X and Y axes to set a new rotation value and assign it to the Pawn, achieving the rotation effect. When the right mouse button is pressed, set MouseHold02 (a Boolean variable used to detect whether the right mouse button is pressed) to True, get the current Pawn's rotation body, get its forward vector and right vector respectively, and set them to the Pawn according to the offset value after pressing, achieving the displacement effect.

[0103] 2) Based on the incoming Json data (JavaScript Object Notation), after serialization, obtain the name of the Point (a specific entity or feature in the CIM environment) to be edited, as well as its Scale, Location, Rotation, etc., obtain the object Class with the Tag (a label used to identify an object) as Point in the level, foreach (traverse) all Point classes, find the object with the same name, and reset its Transform (a combination of an object's position, rotation, and scale, representing the state of a 3D object in space).

[0104] 3) Get the unique ID of the object, pass it to the API, get the corresponding information through callback, create a window, initialize the data, and add it to the viewport for display.

[0105] 4) Combine the various types of data obtained with charts to create bar charts, pie charts, line charts, etc., or set Prefab chart thresholds. When the value of the parameter changes, respond and call back different decision strings based on the value of the parameter.

[0106] 5) When a measurement tool is launched, the event dispatcher bound to EventPlay (which provides an event-driven mechanism for responding to various in-game events in real time) receives the corresponding message and spawns the corresponding measurement tool, such as area, distance, or height. For example, the distance measurement process: BP_MeasureDistance (a Blueprint specifically designed for distance measurement, containing all the logic and functionality required to perform distance measurements) is instantiated and generated. Simultaneously, UMG_TrackDistance (a toolkit for creating game interfaces) is automatically loaded to provide operation instructions. When the left button is clicked, the mouse position is converted to world space and a line trace is performed, starting from the mouse itself and targeting the WorldLocation (the object's position in world coordinates) plus the WorldDirection (the object's direction or orientation) as the target node (assuming WorldDirection is multiplied by a very large number, similar to infinity). The line trace stops when it encounters a Collision (collision detection mechanism) or Trigger (collision detection mechanism), and the object intercepted by the ray is transmitted back. At this point, create a CableActor (a game object used to create dynamic cables or ropes) and set an event timer for the second point to be generated. Once the second point is generated, text will be displayed in the middle of the line segment, indicating the distance between the two points. If you no longer want to measure, you can cancel the measurement manually by sending a command through the event dispatcher or by clicking the right mouse button, and then destroy the measuring tool.

[0107] 6) Playing the video uses the event dispatcher to transmit the PlayVideo command (a custom control signal or instruction) and obtain the URL (Uniform Resource Locator) parameter. CreateWidget (a method for dynamically creating user interface widgets) launches different interface windows based on the URL (such as RTSP or HTTPS). This uses the third-party library OpenCV (Open Source Computer Vision Library).

[0108] 7) Use open source Cesium for Unreal (an open source plug-in that combines the Cesium geographic information platform with the Unreal Engine game engine) to load map image services and terrain services, convert between three-dimensional space and real longitude and latitude, and input longitude and latitude in the interface to automatically convert them into three-dimensional space coordinates.

[0109] (2) 3D asset loading module:

[0110] 1) Models can be imported into the editor for editing, or loaded in real time at runtime through the interface. You can view the model's memory usage, face count, referenced materials, and textures. Lumen global illumination and Ninate can be dynamically set.

[0111] 2) In the Editor, select Import / Game. Supported formats include FBX, OBJ, gltf, and DataSmtih. You can choose whether to merge meshes and create matching skeletons.

[0112] 3) Load the model in real time through the URL in the interface at runtime, and set the Location, Scale, and Rotation based on the data in the interface.

[0113] (3) Cross-platform pixel flow rendering and communication module:

[0114] 1) In the designated module management blueprint class, add PixelStreamingInput (a component used to receive input events from the frontend in pixel streaming mode) to the blueprint component, create a CustomEvent (a developer-defined event that allows for the creation of a series of triggerable logical operations), initiate the SendPixelStreamingResponse event (an event specific to pixel streaming communication, used to send data or responses from the Unreal Engine backend to the frontend), and place the message to be transmitted to the frontend Vue in the Descriptor parameter. To receive the message, use the OnIputEvent event (an event handler used to process input events from the frontend). Parse the string received from the frontend Vue using ExecuteJson (a function or method that performs JSON parsing).

[0115] 2) The front-end Vue can reference app.js (the entry point file of the application) and use the SendJsonMessage method (used to send JSON format messages to the Unreal Engine server) to send data to the UE. The data received from the UE is received by listening to the HandleResponse (listener or processor), achieving two-way communication.

[0116] 3) Pixel Streaming uses WebRTC, using a signaling server to exchange connection information, configure IP addresses and port numbers, and media codecs, thereby establishing a direct communication channel between the client and server. WebRTC can be used to transmit media streams, including audio and video. In the case of pixel streaming, this primarily involves transmitting video streams generated by Unreal Engine.

[0117] Figure 5 This is a structural diagram of a development device for a digital twin application according to an embodiment of the present application, such as Figure 5 As shown, the device includes:

[0118] A creation module 502 is configured to construct a digital twin project task in response to a creation instruction, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment;

[0119] A determination module 504 is configured to determine a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least configured to convert the three-dimensional scene into a pixel stream adapted to the requirements of different terminal devices;

[0120] The transmission module 506 is used to transmit the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

[0121] It should be noted that Figure 5 The development device of the digital twin application shown is used to execute Figure 2 The development method of digital twin applications shown in Figure 2 The explanations in the development method of digital twin applications in the Figure 5 The development device of the digital twin application shown will not be described in detail here.

[0122] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the steps of the development method of the digital twin application in each embodiment of the present application.

[0123] For example, the processor performs the following functions by executing program instructions stored in the memory: in response to a creation instruction, constructing a digital twin project task, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment; determining a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least used to convert the three-dimensional scene into a pixel stream that adapts to the requirements of different terminal devices; and transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

[0124] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the steps of the digital twin application development method in each embodiment of the present application by running the computer program.

[0125] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the digital twin application development method in each embodiment of the present application.

[0126] An embodiment of the present application also provides a computer program, which, when executed by a processor, implements the steps of the method for developing a digital twin application in each embodiment of the present application.

[0127] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0128] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0130] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0131] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0132] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0133] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for developing a digital twin application, characterized in that: include: In response to the creation instruction, construct a digital twin project task, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment; Determine a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least used to convert the three-dimensional scene into a pixel stream that adapts to the requirements of different terminal devices; The digital twin project task is transmitted to at least one terminal device according to the pixel stream transmission strategy.

2. The method according to claim 1, characterized in that Determine the pixel streaming strategy corresponding to the digital twin project tasks, including: Acquiring device information of the at least one terminal device, wherein the device information includes decoding capability and network transmission capability of the at least one terminal device; Determining a codec parameter configuration based on the device information and rendering information of the three-dimensional scene, wherein the codec parameter configuration is used to adapt to the network transmission capabilities of different terminal devices, and the rendering information is used to reflect the real-time rendering requirements of the three-dimensional scene; Determining a communication protocol configuration based on the device information, wherein the communication protocol configuration includes an address of a signaling server, and the signaling server is used to establish a communication channel with the terminal device; Determine a pixel stream transmission strategy corresponding to the codec parameter configuration and the communication protocol configuration.

3. The method according to claim 1, characterized in that Building digital twin project tasks, including: Acquiring target data, wherein the target data includes three-dimensional model files in different formats, and the three-dimensional model files are used to map physical entities in the real world; Determining a parser corresponding to the format of the 3D model file, and extracting target information of the 3D model file using the parser, wherein the target information includes one of the following: vertex coordinates, normals, texture coordinates, and material information; The target information is converted into model data in a preset format, and the digital twin project task is constructed based on the model data.

4. The method according to claim 1, wherein The method further comprises: receiving an editing instruction, wherein the editing instruction is used to instruct an editing operation to be performed on an object in the three-dimensional scene, the editing operation comprising one of the following: changing a position, adjusting a size, and modifying a material property; determining an identifier corresponding to the object from the editing instruction; The target object corresponding to the identifier is edited according to the editing instruction, and the editing result is synchronized to the at least one terminal device.

5. The method according to claim 1, wherein Transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy includes: Determining state data corresponding to a plurality of objects in the three-dimensional scene in the digital twin project task; Rendering the plurality of objects according to the state data to obtain a video frame sequence; The video frame sequence is encoded into a video stream, and the video stream is transmitted to the at least one terminal device.

6. The method according to claim 5, characterized in that Transmitting the video stream to the at least one terminal device includes: In response to a connection request from a target terminal device, establishing a connection with the target terminal device and sending encoding parameters to the target terminal device, wherein the encoding parameters are determined according to a state of the connection and include a resolution and a frame rate of a video stream, and the target terminal device is any one of the at least one terminal device; The video stream is sent to the target terminal device through the connection.

7. The method according to claim 1, characterized in that After transmitting the digital twin project task to at least one terminal device according to the pixel stream transmission strategy, the method further includes: receiving an operation instruction from the at least one terminal device, wherein the operation instruction includes an editing operation performed by a target object on a scene copy on the terminal device, the scene copy being a copy of the three-dimensional scene, and each terminal device corresponds to one scene copy; updating the state data of the three-dimensional scene according to the operation instruction, and determining an updated scene copy corresponding to the updated three-dimensional scene; The updated scene copy is transmitted to the at least one terminal device.

8. A device for developing digital twin applications, characterized in that: include: A creation module, configured to construct a digital twin project task in response to a creation instruction, wherein the digital twin project task includes a three-dimensional scene reflecting the real world rendered in a virtual environment; A determination module, configured to determine a pixel stream transmission strategy corresponding to the digital twin project task, wherein the pixel stream transmission strategy is at least configured to convert the three-dimensional scene into a pixel stream adapted to the requirements of different terminal devices; A transmission module is used to transmit the digital twin project task to at least one terminal device according to the pixel stream transmission strategy.

9. An electronic device, characterized in that: include: A memory and a processor, the memory being used to store program instructions; the processor being connected to the memory and being used to execute the development method for a digital twin application according to any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the method for developing a digital twin application according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method for developing a digital twin application described in any one of claims 1 to 7 is implemented.

Images