A cross-rendering engine intermediate device, data processing method and storage medium

Abstract

The embodiment of the present application provides an intermediate device, a data processing method and a storage medium across rendering engines, and relates to the field of computer technology. The device includes at least one application layer, which is used to receive the data to be rendered sent by the business layer, and convert the data to be rendered into a logical data structure adapted to the project; an intermediate layer, which is used to convert the data of the application layer from a logical data structure into a linked list; at least one implementation layer, which is used to process the data in the linked list according to the data type of the rendering engine, and send it to the rendering engine for rendering, and establish an intermediate interface between the business module and the rendering engine, which can ensure the fast and convenient switching of the engine, and solve the problem of difficult engine switching in the existing method.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an intermediate device across rendering engines, a data processing method, and a storage medium. Background Art

[0002] With the development of rendering technology, more and more rendering engines have appeared in the market, such as Unity, UE, Cocos, etc. Each engine can render excellent images, and each engine has its own emphasis. On the other hand, cloud engines are also developing actively, and major engine manufacturers are also actively deploying their own cloud platforms.

[0003] Generally, the engine to be used is determined at the beginning of the project and has an independent interface. When the business is quickly switched from engine A to engine B, a large amount of business code is strongly coupled with the engine without an obvious boundary, which is not suitable for expansion and engine switching verification. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an intermediate device, a data processing method and a storage medium across rendering engines, establish an intermediate interface between the business module and the rendering engine, ensure fast and convenient switching of the engine, and solve the problem of difficult engine switching in existing methods.

[0005] An embodiment of the present application provides an intermediate device across rendering engines, the device comprising:

[0006] At least one application layer, used for receiving the data to be rendered sent by the business layer, and converting the data to be rendered into a logical data structure adapted to the project;

[0007] The middle layer is used to transform the data of the application layer from a logical data structure into a linked list;

[0008] At least one implementation layer is used to process the data in the linked list according to the data type of the rendering engine, and send the data to the rendering engine for rendering.

[0009] In the above implementation process, by setting up an intermediate device, the business layer and various rendering engines can be connected, and various engines can be switched and quickly connected, which ensures the stability of the business side and solves the problem of difficult engine switching in the existing method.

[0010] Furthermore, the intermediate layer comprises:

[0011] A structure for managing a scene, including a camera and rendering objects;

[0012] Resources include models, materials, and textures. The rendering object generates a drawable rendering object by referencing the corresponding resources.

[0013] In the above implementation process, the structure in the middle layer is the object responsible for managing the scene and is globally unique. The rendering object forms a drawable rendering object by referencing resources.

[0014] Furthermore, the logical data structure includes a tree structure, and the application layer is used for:

[0015] Identify the location information in three-dimensional space on the node;

[0016] A model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

[0017] In the above implementation process, as an implementation method, the rendering scene can be organized through a tree structure, and the vertex data, index data, texture data and data types corresponding to the mesh nodes of the data can be transmitted and distributed through the data types of each data, so that multi-engine and multi-platform distribution can be achieved.

[0018] Furthermore, the implementation layer is used to:

[0019] The data type and data order of the data in the linked list are converted and processed to adapt to the data requirements of webgl, and the processed data is sent to webgl, wherein the data types include array type, Float type and Int type.

[0020] In the above implementation process, the implementation layer is used to reorganize the data obtained from the middle layer and convert it into a form that conforms to these engines or platforms according to the characteristics of different engines.

[0021] Furthermore, the implementation layer is used to:

[0022] The data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

[0023] In the above implementation process, the communication protocol stipulates how the mesh data is transmitted to the server, and the implementation layer can convert the data into the form required by the communication protocol.

[0024] The embodiment of the present application also provides a data processing method across rendering engines, the method comprising:

[0025] Receive the data to be rendered sent by the business layer, and convert the data to be rendered into a logical data structure adapted to the project;

[0026] Convert the data to be rendered from a logical data structure into a linked list;

[0027] According to the data type of the rendering engine, the data in the linked list is processed accordingly and sent to the rendering engine for rendering.

[0028] In the above implementation process, by setting up an intermediate device, the business layer and various rendering engines can be connected, and various engines can be switched and quickly connected, which ensures the stability of the business side and solves the problem of difficult engine switching in the existing method.

[0029] Furthermore, the logical data structure includes a tree structure, the tree structure includes a root node, a node node and a mesh node, and the converting the to-be-rendered data into the tree structure includes:

[0030] Identify the location information in three-dimensional space on the node;

[0031] A model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

[0032] In the above implementation process, as an implementation method, the rendering scene can be organized through a tree structure, and the vertex data, index data, texture data and data types corresponding to the mesh nodes of the data can be transmitted and distributed through the data types of each data, so that multi-engine and multi-platform distribution can be achieved.

[0033] Furthermore, the data in the linked list is processed accordingly according to the data type of the rendering engine, and sent to the rendering engine for rendering, including:

[0034] The data type and data order of the data in the linked list are converted and processed to adapt to the data requirements of webgl, and the processed data is sent to webgl, wherein the data types include array type, Float type and Int type.

[0035] In the above implementation process, the implementation layer is used to reorganize the data obtained from the middle layer and convert it into a form that conforms to these engines or platforms according to the characteristics of different engines.

[0036] Furthermore, the data in the linked list is processed accordingly according to the data type of the rendering engine, and sent to the rendering engine for rendering, including:

[0037] The data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

[0038] In the above implementation process, the communication protocol stipulates how the mesh data is transmitted to the server, and the implementation layer can convert the data into the form required by the communication protocol.

[0039] An embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores computer program instructions. When the computer program instructions are read and executed by a processor, any of the above-mentioned cross-rendering engine data processing methods is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A structural block diagram of an intermediate device across rendering engines provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of the data required for rendering by the rendering engine provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of the relationship between the existing business modules and the engine provided in the embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of the intermediate layer provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of a tree structure provided for an embodiment of the present application;

[0046] Figure 6 A schematic diagram of the model structure provided in the embodiment of the present application;

[0047] Figure 7 A flowchart of a cross-rendering engine data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0049] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0050] Please see Figure 1 , Figure 1 This is a structural block diagram of a cross-rendering engine intermediate device provided in an embodiment of the present application. The device is an intermediate interface applied between the business layer and the rendering engine, which can smoothly transfer the rendering data of the business layer to the intermediate interface, and then pass it to each rendering engine through the intermediate interface.

[0051] like Figure 2 As shown in the figure, it is a schematic diagram of the data required by the rendering engine for rendering. Regardless of any engine, there are two main elements required. One major category is resource data, including geometric data, material, and texture data. When rendering an object, this data is used for rendering; the other major category is organized rendering information, such as scene management, rendering objects, etc. In other words, the rendering object references and organizes the resource data, and then passes it to the rendering engine for rendering. Figure 3 As shown, it is a schematic diagram of the relationship between the existing business modules and the engine. In actual applications, the business layer is often strongly coupled with the engine interface, and different products based on different engines are very different. Therefore, each module in the product will have a strong impact on the engine. As a result, when the business layer switches from engine A to engine B, a large amount of business code is strongly coupled with the engine without an obvious boundary, which is not conducive to expansion and engine switching verification. The intermediate interface is added in this application, which fundamentally solves this problem.

[0052] The middle interface is connected to the business layer, and can be repackaged according to the user habits of different business layers; it is connected to the rendering engine, for example, the data can be processed and passed to the local rendering engine, the data can be passed to the cloud engine after processing, and the data can be passed to other exe through inter-process communication after processing.

[0053] The intermediate interface may specifically include:

[0054] At least one application layer 100, used to receive the data to be rendered sent by the business layer, and convert the data to be rendered into a tree structure;

[0055] The middle layer 200 is used to transform the data of the application layer 100 from a tree structure into a linked list;

[0056] At least one implementation layer 300 is used to process the data in the linked list according to the data type of the rendering engine, and send it to the rendering engine for rendering.

[0057] The intermediate interface includes an application layer 100, an intermediate layer 200 and an implementation layer 300, wherein the intermediate layer 200, as the core, determines the complete expression result of a rendering engine. Both the application layer 100 and the implementation layer 300 are developed around the intermediate layer 200, and according to the characteristics of different platforms and different projects, the application layer 100 and the implementation layer 300 will have different forms. As one of the implementation methods, they can be encapsulated based on a tree structure.

[0058] The middle layer 200 includes:

[0059] A structure for managing a scene, including a camera and rendering objects;

[0060] Resources include models, materials, and textures. The rendering object generates a drawable rendering object by referencing the corresponding resources.

[0061] In addition, it should be noted that the application layer, like the implementation layer, can have multiple application layers, but in a specific project, there is often only one application layer, one middle layer and multiple implementation layers; the application layer is not only a tree structure, but can also be other forms of logical data structures, such as ECS structure, chain structure, etc., which can be flexibly selected according to the needs of the specific project. No limitation is made here, except that the tree structure is most in line with the characteristics of the scene and is often used; and the application layer is also encapsulated according to the data structure corresponding to the application layer 100, such as encapsulating the application layer 100 according to the tree structure.

[0062] like Figure 4 As shown in the figure, it is a schematic diagram of the structure of the middle layer 200. Among them, the structure (Root) is the object responsible for managing the scene, which is globally unique. The scene (Scene) is a scene created as needed, and the scene manages the camera (Camera) and the rendering object (Object). Resources include models, materials and textures, and each category includes multiple small categories. Object will reference the corresponding resources to generate a drawable rendering object.

[0063] However, due to their own unique docking methods, the corresponding structures and resource data of each rendering engine will be different. Therefore, the implementation layer 300 is concerned with how to transfer data, while the middle layer 200 is concerned with how to obtain and save data from the application layer 100, and how to transfer it to the implementation layer 300. Therefore, the middle layer 200 is the carrier of data, and each implementation layer 300 will obtain the corresponding data from the middle layer 200 when transferring data.

[0064] As one of the implementation modes, for a rendering scene with a tree structure, for the application layer 100, the location information in the three-dimensional space is identified on the node; and a model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

[0065] like Figure 5 As shown in the figure, it is a tree structure diagram. The scene is an attribute structure. From the root node, many nodes are mounted. Each node identifies its position information (transform) in 3D space. Models can be mounted on the node for display. The Mesh node is mounted on the leaf node of the scene attribute structure. The Mesh node is the model itself. Its data includes vertex data, index data and texture data. Figure 6 The following is a schematic diagram of the model structure.

[0066] Among them, vertex data and index data are used to express the shape of the Mesh node.

[0067] Vertex data is a collection of buffers and their corresponding uses, for example:

[0068] {

[0069] Buffer1: Position: Type Float32Array, ElementSize: 3,

[0070] Buffer2: Normal: type Float32Array, ElementSize: 3,

[0071] Buffer3: Texture coordinates UV: type Float32Array, ElementSize: 2,

[0072] }

[0073] The index data is a buffer of data type Uint16Array, which is used to describe the order of vertex data to form a triangle.

[0074] Material data is used to express the display of Mesh nodes.

[0075] The material data also includes textures and parameters for material calculations. Textures refer to common images in png, jpg and other formats, while material parameters participate in the actual calculations together with the texture data, calculating pixel by pixel and generating the final color of the screen. The meaning of material parameters is variable, but their types are often fixed, such as float, vector3 and matrix4. Since this application focuses on the forwarding of data to multiple different rendering engines through intermediate interfaces at the data level, it pays more attention to the type of data rather than its meaning in the material.

[0076] In summary, the data that this application actually focuses on is Mesh (vertices + indices) and the material data associated with Mesh (textures + parameters). Therefore, as long as the above data types are transmitted and distributed, multi-engine and multi-platform distribution can be achieved.

[0077] The application layer 100 is an interface encapsulation method for actual business applications. It is used to encapsulate the interface form of the middle layer 200 into a more user-friendly form. It is generally organized in a tree structure, and the rendering data (models, materials, textures, etc.) are attached to each child node of the tree structure. The renderable model data is attached as a node under the child nodes of the entire tree; specifically, each model consists of vertices, indexes, and materials, and the materials are composed of maps and material parameters. Its purpose is to fully express a scene through data.

[0078] The data types of model data, material data, and map data mainly include two forms: Buffer and basic type.

[0079] Among them, for Buffer types, such as geometric data and texture data, they are all large arrays. For example, for a 256*256 image, it holds 256*256*4 bytes of Buffer data; for geometric bodies, the position information of each vertex is an array of type float.

[0080] Basic types, such as material information and rendering objects. Material information contains descriptions of the object's color (Color type) and glossiness (float type); while rendering objects describe which material (material id, int type) and geometry (geometry id, int type) to use, where to start and end drawing the geometry data (offset, count, both uint type), etc.

[0081] The middle layer 200 is used to organize and manage the data of the application layer 100. Specifically, it is used to change the data of the application layer 100 from a tree structure to a linked list; cut out hidden nodes and nodes outside the camera; determine the drawing order of each model entity, and dynamically merge the models in batches.

[0082] The implementation layer 300 (Impl) is used to convert and process the data type and data order of the data in the linked list to adapt to the data requirements of webgl, and send the processed data to webgl. The data types include array type, Float type and Int type.

[0083] The implementation layer 300 obtains the required data from the middle layer 200 and reorganizes it so that different impls convert the data into a form that conforms to the engine or platform according to the characteristics of each engine or platform.

[0084] As an implementation method, for local rendering, the implementation layer 300 directly passes data to WebGL, and the implementation can be named WebglImpl. The WebGL interface has specified what form the data must be in, or what form it must be in. The main task of the impl is to make the data object it obtains consistent with the data format agreed upon by the WebGL standard.

[0085] I will not elaborate on the interface form of WebGL and what kind of data is needed to drive the browser and hardware to draw images. However, from a macro level, the array type data, Float type data, and Int type data obtained from the middle layer 200 are converted into the form required by WebGL. In addition, WebGL also has certain requirements for the order in which data is passed in. impl needs to be created, passed in, modified, and deleted in the order required by it.

[0086] The implementation layer 300 is used to:

[0087] The data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

[0088] As another implementation, for cloud rendering, data is sent to the cloud through the network for cloud rendering. The implementation can be named SocketImpl. A set of communication protocols are agreed with the server in advance. The communication protocol stipulates how the mesh data is transmitted to the server. What the impl needs to do is to convert the array type data, Float type data, and Int type data obtained from the middle layer 200 into the form required by the communication protocol.

[0089] Both forms mentioned data conversion, specifically the conversion process from data structure to Buffer, for example:

[0090]

[0091]

[0092] Convert to

[0093] Uchar*matParamBuffer; / / 4 float buffer

[0094] The buffer will be passed to the cloud rendering engine through the socket, or to the local webgl communication object.

[0095] The implementation layer 300 actually delivers the data of the middle layer 200, and performs corresponding operations according to the types of different rendering engines: for example, for webgl, the work of the implementation layer 300 is to convert the data of the middle layer 200 into a GPU-friendly form, and call the webglAPI interface to drive the hardware for drawing; the work of the cloud rendering implementation layer 300 is to convert the data of the middle layer 200 into a websocket byte stream, and push it to the server for rendering. Its own work focuses on the compression of data streams and the reasonable design of transmission protocols.

[0096] The application layer 100, the middle layer 200 and the implementation layer 300 are in a serial relationship and are also a complete whole with functional emphasis and low coupling, thus achieving the purpose of being cross-engine and cross-platform.

[0097] The intermediate interface can completely isolate the business layer and each rendering engine. When you want to replace the rendering engine, you only need to generate the corresponding impl structure in the intermediate interface, which is convenient for maintenance and scalability.

[0098] In addition, due to the large differences between rendering engines, the corresponding functions in the intermediate interface can be independent of each other, such as maintaining them in their own impls, adding some auxiliary classes, or encapsulating more detailed structures, so as not to affect each other.

[0099] The present application also provides a cross-rendering engine data processing method, which is applied to an intermediate interface, such as Figure 7 FIG. 1 is a flowchart of a method for processing data across rendering engines, which method may specifically include the following steps:

[0100] Step S100: receiving the data to be rendered sent by the business layer, and converting the data to be rendered into a tree structure;

[0101] Specifically:

[0102] The position information in the three-dimensional space is identified on the node; and the model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

[0103] Step S200: converting the data to be rendered from a tree structure into a linked list;

[0104] Step S300: According to the data type of the rendering engine, the data in the linked list is processed accordingly, and sent to the rendering engine for rendering.

[0105] As one implementation method, the data type and data order of the data in the linked list are converted and processed to adapt to the data requirements of webgl, and the processed data is sent to webgl, and the data type includes array type, Float type and Int type.

[0106] As another implementation, the data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

[0107] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned cross-rendering engine data processing method.

[0108] An embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores computer program instructions. When the computer program instructions are read and executed by a processor, the above-mentioned cross-rendering engine data processing method is executed.

[0109] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0111] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0112] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0114] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. An intermediate device across rendering engines, characterized in that: The device comprises: At least one application layer, used for receiving the data to be rendered sent by the business layer, and converting the data to be rendered into a logical data structure adapted to the project; The middle layer is used to convert the data of the application layer from a logical data structure into a linked list, and the middle layer includes: A structure for managing a scene, including a camera and rendering objects; Resources, including models, materials and textures, the rendering object generates a drawable rendering object by referencing the corresponding resources; At least one implementation layer, used to process the data in the linked list according to the data type of the rendering engine, and send it to the rendering engine for rendering; The logical data structure includes a tree structure, and the application layer is used for: Identify the location information in three-dimensional space on the node; A model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

2. The cross-rendering engine intermediate device according to claim 1, characterized in that: The implementation layer is used to: The data type and data order of the data in the linked list are converted and processed to adapt to the data requirements of webgl, and the processed data is sent to webgl, wherein the data types include array type, Float type and Int type.

3. The cross-rendering engine intermediate device according to claim 1, characterized in that: The implementation layer is used to: The data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

4. A data processing method across rendering engines, characterized in that: The method applied to the cross-rendering engine intermediate device according to any one of claims 1 to 3 comprises: Receive the data to be rendered sent by the business layer, and convert the data to be rendered into a logical data structure adapted to the project; Convert the data to be rendered from a logical data structure into a linked list; According to the data type of the rendering engine, the data in the linked list is processed accordingly, and sent to the rendering engine for rendering; The logical data structure includes a tree structure, the tree structure includes a root node, a node node and a mesh node, and the converting the to-be-rendered data into the tree structure includes: Identify the location information in three-dimensional space on the node; A model is mounted on the mesh node, wherein the model includes vertex data, index data, texture data and corresponding data types.

5. The cross-rendering engine data processing method according to claim 4, characterized in that: The processing of the data in the linked list according to the data type of the rendering engine and sending the data to the rendering engine for rendering includes: The data type and data order of the data in the linked list are converted and processed to adapt to the data requirements of webgl, and the processed data is sent to webgl, wherein the data types include array type, Float type and Int type.

6. The cross-rendering engine data processing method according to claim 4, characterized in that: The processing of the data in the linked list according to the data type of the rendering engine and sending the data to the rendering engine for rendering includes: The data type of the data in the linked list is converted into a form required by the communication protocol, and the converted data is sent to the cloud through the communication protocol for cloud rendering.

7. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the cross-rendering engine data processing method according to any one of claims 4 to 6 is executed.

Images