Model rendering method and apparatus, electronic device, and computer-readable storage medium

Abstract

Embodiments of the present application provide a model rendering method and device, electronic equipment and computer readable storage medium, and relate to the field of three-dimensional visualization. The method comprises: distributing a model rendering task of a client to at least one rendering server, and returning result data to a rendering engine after the rendering server processes the model rendering task; meanwhile, a remote rendering communication protocol is formulated based on a binary format, and a corresponding rendering engine is encapsulated based on the protocol, so that the rendering engine can directly identify binary format data and write binary result data directly into a graphics card, thereby improving rendering efficiency and graphics operation performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional visualization, in particular, the present application relates to a model rendering method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] A three-dimensional model is a polygonal representation of an object, which is usually displayed by a computer or other video equipment. Today, three-dimensional models have been used in various fields, such as smart medical treatment, film entertainment, and smart city applications.

[0003] Currently, when a three-dimensional model is displayed in a browser, backend rendering can be selected. When a three-dimensional model is rendered in the backend, a rendering engine is generally called by a Web (World Wide Web) application running in the browser to communicate with a backend server to complete real-time rendering of a three-dimensional scene. However, when the display requirement is for a large or super-large three-dimensional scene model, problems such as display scene lag and low graphics operation performance occur, resulting in the application lacking real-time rendering capability. SUMMARY

[0004] The present application aims to at least solve one of the above technical defects, in particular, the technical defect of low graphics operation performance.

[0005] In a first aspect, a model rendering method is provided, applied to a rendering engine, the method comprising:

[0006] Receiving an access instruction for an application, and establishing a connection with at least one rendering server respectively; wherein the rendering engine is obtained based on a remote rendering communication protocol encapsulation, and the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a binary format;

[0007] Sending a model rendering task to a rendering router; the rendering router is used to send the model rendering task to at least one rendering server respectively; any rendering server is used to obtain corresponding result data based on the model rendering task, and send the result data to the rendering router;

[0008] Receiving a protocol response message sent by the rendering router; the protocol response message is generated based on the result data of the model rendering task;

[0009] Parsing the protocol response message to obtain a rendering result of the model rendering task; the rendering result is used to generate a display page of the application.

[0010] In an optional embodiment of the first aspect, the rendering router is configured to split the model rendering task into at least one subtask, and send the subtasks to at least one rendering server respectively; and any of the rendering servers is configured to process the subtasks to obtain corresponding result data of the subtasks.

[0011] In an optional embodiment of the first aspect, the protocol response message is obtained by the rendering router by merging the result data of the subtasks.

[0012] In an optional embodiment of the first aspect, the protocol response message is parsed to obtain the rendering result of the model rendering task, including:

[0013] The protocol response message is parsed to obtain the at least one rendering block data.

[0014] The rendering block data is written to the graphics card, and the graphics card is configured to process the rendering block data to obtain the rendering result of the model rendering task.

[0015] In a second aspect, a model rendering method is provided, and the method is applied to a rendering router, including:

[0016] The model rendering task sent by a rendering engine is received; the rendering engine is connected to at least one rendering server; the rendering engine is obtained based on a remote rendering communication protocol encapsulation; the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a binary format;

[0017] The model rendering task is sent to the at least one rendering server respectively; any of the rendering servers is configured to obtain corresponding result data based on the model rendering task and send the result data;

[0018] The result data sent by the rendering server is received, and a protocol response message is generated based on the result data of the model rendering task;

[0019] The protocol response message is sent to the rendering engine; the rendering engine is configured to parse the protocol response message to obtain a rendering result corresponding to the model rendering task; and the rendering result is used to generate a display page of an application.

[0020] In an optional embodiment of the second aspect, the model rendering task is sent to the at least one rendering server respectively, including:

[0021] The model rendering task is split into at least one subtask, and the subtasks are sent to the at least one rendering server respectively;

[0022] Any of the rendering servers is configured to process the subtasks to obtain corresponding result data of the subtasks.

[0023] In an optional embodiment of the second aspect, the protocol response message is generated based on the result data of the model rendering task, including:

[0024] Merging the result data corresponding to the subtasks to obtain a protocol response message.

[0025] In an optional embodiment of the second aspect, the rendering engine is configured to parse the protocol response message to obtain at least one rendering block data and write the rendering block data to the graphics card; and the graphics card is configured to process the rendering block data to obtain the rendering result of the model rendering task.

[0026] In a third aspect, a device for model rendering is provided, and the device comprises:

[0027] The connection module is configured to receive an access instruction for an application and establish a connection with at least one rendering server respectively; the rendering engine is obtained based on encapsulation of a remote rendering communication protocol; and the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a binary format.

[0028] The rendering module is configured to send a model rendering task to a rendering router; the rendering router is configured to send the model rendering task to at least one rendering server respectively; and any rendering server is configured to obtain corresponding result data based on the model rendering task and send the result data to the rendering router.

[0029] The first receiving module is configured to receive a protocol response message sent by the rendering router; and the protocol response message is generated based on the result data of the model rendering task.

[0030] The first analysis module is configured to parse the protocol response message to obtain a rendering result of the model rendering task; and the rendering result is used to generate a display page of the application.

[0031] In an optional embodiment of the third aspect, the rendering router is configured to split the model rendering task into at least one subtask and send the subtasks to at least one rendering server respectively; and any rendering server is configured to process the subtasks to obtain result data corresponding to the subtasks.

[0032] In an optional embodiment of the third aspect, the protocol response message is obtained by merging the result data corresponding to the subtasks by the rendering router.

[0033] In an optional embodiment of the third aspect, when the first analysis module parses the protocol response message to obtain the rendering result of the model rendering task, the first analysis module is specifically configured to:

[0034] parse the protocol response message to obtain at least one rendering block data;

[0035] write the rendering block data to a graphics card; and the graphics card is configured to process the rendering block data to obtain the rendering result of the model rendering task.

[0036] In a fourth aspect, a device for model rendering is provided, and the device comprises:

[0037] a receiving module configured to receive a model rendering task sent by a rendering engine, wherein the rendering engine is connected to at least one rendering server, and the rendering engine is obtained based on a remote rendering communication protocol encapsulation, and the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a binary format;

[0038] a sending module configured to send the model rendering task to the at least one rendering server, and any rendering server is configured to obtain corresponding result data based on the model rendering task and send the result data;

[0039] a second receiving module configured to receive a protocol response message sent by a rendering router, wherein the protocol response message is generated based on the result data of the model rendering task;

[0040] a second analysis module configured to analyze the protocol response message to obtain a rendering result of the model rendering task, and the rendering result is used to generate a display page of an application.

[0041] In an optional embodiment of the fourth aspect, when the sending module sends the model rendering task to the at least one rendering server, the sending module is specifically configured to:

[0042] split the model rendering task into at least one subtask, and then send the subtasks to the at least one rendering server respectively;

[0043] wherein any rendering server is configured to process the subtasks to obtain result data corresponding to the subtasks.

[0044] In an optional embodiment of the fourth aspect, when the second receiving module generates the protocol response message based on the result data of the model rendering task, the second receiving module is specifically configured to:

[0045] merge the result data corresponding to the subtasks to obtain the protocol response message.

[0046] In an optional embodiment of the fourth aspect, the rendering engine is configured to analyze the protocol response message, obtain at least one rendering block data, and write the rendering block data to a graphics card, and the graphics card is configured to process the rendering block data to obtain a rendering result of the model rendering task.

[0047] In a fifth aspect, an electronic device is provided, and the electronic device comprises:

[0048] a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the model rendering method of any of the above embodiments is implemented.

[0049] In a sixth aspect, the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the model rendering method of any one of the above embodiments.

[0050] The model rendering method reduces resource consumption of the client by distributing rendering tasks of the client to at least one server, and formulates a remote rendering communication protocol based on a binary format, encapsulates a corresponding rendering engine based on the protocol, and directly writes rendering result data to a graphics card, thereby improving rendering efficiency and graphics operation performance, and ensuring real-time rendering capability even for large or super-large three-dimensional scene models. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced.

[0052] Figure 1 A flowchart of a model rendering method provided by the embodiments of the present application;

[0053] Figure 2 A flowchart of a model rendering method provided by the embodiments of the present application;

[0054] Figure 3 A flowchart of a model rendering method provided by the embodiments of the present application;

[0055] Figure 4 A flowchart of a model rendering method provided by the embodiments of the present application;

[0056] Figure 5 A structural diagram of a model rendering device provided by the embodiments of the present application;

[0057] Figure 6 A structural diagram of a model rendering device provided by the embodiments of the present application;

[0058] Figure 7 A structural diagram of a model rendering electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be interpreted as a limitation of the present application.

[0060] Those skilled in the art can understand that the singular forms "a," "an," and "the" used herein include plural forms unless otherwise stated. It should be further understood that the use of the term "include" in the specification of the application means that a feature, integer, step, operation, element, and / or component exists, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0061] For the purpose, technical solutions and advantages of the present application to be clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0062] Three-dimensional model, namely 3D(three-dimensional) model, refers to a three-dimensional model, and in the field of computers, it refers to a three-dimensional model built by three-dimensional software, including various buildings, figures, vegetation, machinery and various model types of scene collection and the like.

[0063] For current three-dimensional application programs, pursuing picture realism is an endless goal, so three-dimensional model scenes become more complex and diversified, which brings great load to graphics hardware, and problems such as insufficient graphics processing computing resources occur, resulting in failure to achieve real-time frame rate.

[0064] The model rendering method, device, electronic equipment and computer readable storage medium provided by the present application aim to solve the above technical problems of the prior art.

[0065] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0066] The model loading method provided in the embodiments of the present application can be applied to a server or a terminal.

[0067] Those skilled in the art can understand that the "terminal" used herein can be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a MID (Mobile Internet Device), etc.; and the "server" can be implemented by a stand-alone server or a server cluster composed of multiple servers.

[0068] The IP (Internet Protocol) referred to in the embodiments of the present application is a network layer protocol in the TCP / IP system, is the core of the entire TCP / IP protocol family, and is also the basis of the Internet. The IP protocol provides communication between hosts, and completes communication between different hosts using an identifier that uniquely determines a host, i.e., an IP address. Through the IP address, the IP protocol can send a data packet to another host. The IP protocol provides communication between hosts, and the TCP (Transmission Control Protocol) protocol completes communication between processes on two hosts on the basis of the communication function between hosts provided by the IP protocol.

[0069] The embodiments of the present application provide a model rendering method, as shown in Figure 1 The method can be applied to a rendering engine and includes the following steps:

[0070] In step S101, an access instruction for an application is received, and a connection with at least one rendering server is established respectively; wherein the rendering engine is obtained based on remote rendering communication protocol encapsulation, and the remote rendering communication protocol is a set of remote rendering communication protocols formulated on the basis of a binary format.

[0071] In the embodiments of the present application, the rendering mode of the three-dimensional model can be selected as a back-end rendering, i.e., an SSR (Server Side Render) mode. For example, the browser can directly receive a final HTML (Hyper Text Markup Language) string presented to the user after server calculation, at this time, the browser can only perform a parsing operation on the rendering result data, and the rendering result data is presented to the user by the system call of the operating system for manipulating the display to display the content.

[0072] Wherein, the rendering engine can be set in the browser, when the rendering task needs to be handed over to the rendering server for calculation and the rendering result is returned, the client and the server need to communicate. Therefore, when the browser receives an access instruction to the application, for example, the user opens the three-dimensional model display interface, the browser can call the rendering engine to connect with the rendering server, and the connection can be a socket connection.

[0073] The socket can be understood as a TCP layer encapsulation, which is an intermediate software abstraction layer for communication between the application layer and the TCP / IP protocol family, and it is a set of interfaces. In the design pattern, the socket is actually a facade pattern, which hides the complex TCP / IP protocol family behind the socket interface, and lets the socket organize data to meet the specified protocol to achieve the purpose of TCP communication.

[0074] Wherein, the socket connection can be a socket long connection, in the whole communication process, the client and the server are only used for a socket object, and the socket connection is kept for a long time. To keep the socket connection, data can be continuously written to the other end, and then the data of the other end is read, which can be simply understood as "heartbeat". As long as the "heartbeat" is in, the socket connection is kept, and the interval of writing data can be determined according to the actual application requirement. Wherein, the "heartbeat packet" is not the actual business data, which can be processed differently according to different communication protocols. For example, JSON (JavaScript Object Notation, JS object notation) communication can add a "type" field to indicate whether the JSON is a heartbeat or business data. Binary protocol can also be used, and the processing method is similar, which can meet the requirement of distinguishing whether a data packet is "heartbeat" or real data.

[0075] In the embodiment of the present application, the remote rendering communication protocol is based on a binary format. Specifically, the binary format can be WebAssembly. WebAssembly is a binary format that can solve some performance defects of JavaScript (abbreviated as "JS"). JS was originally positioned as an interpreted language for non-professional programmers and designers, but eventually became one of the most important languages in the Internet field, and was widely used in web applications. With the increasing complexity of web pages, the performance of JS can no longer meet people's needs, and WebAssembly provides a way to make code written in various languages run in the Web at a speed close to native code, greatly improving the performance of JS and the portability of web pages.

[0076] In the embodiment of the present application, the rendering engine can be obtained based on the remote rendering communication protocol encapsulation.

[0077] Wherein, the rendering engine, also known as the layout engine, is responsible for obtaining the content of the webpage, organizing the information, and calculating the display method of the webpage to output to the display or printer. The rendering engine can be applied to web browsers, email clients, and other applications that need to edit and display web content. Encapsulation refers to hiding the attributes and implementation details of an object, only exposing the interface to the outside, and controlling the access level of reading and modifying attributes in the program.

[0078] In one embodiment, the rendering engine is obtained by encapsulating the remote rendering communication protocol. The remote rendering communication protocol can be an application-level protocol developed based on network communication principles, which can provide the following functions:

[0079] (1) Provide a standard transmission format that is easier to use or more language-friendly;

[0080] (2) Convert the transmission format to a stream for the user, and transmit it to a remote computer (rendering server) through any transmission protocol. The remote computer converts the stream to a transmission format after receiving it, and stores or notifies the remote computer in other ways.

[0081] Specifically, the remote rendering communication protocol can also be a set of socket-based protocols implemented using the WebAssembly format. The rendering engine can also be obtained by encapsulating the protocol. The engine can only receive binary data formats that can be recognized by the graphics card, and directly communicate with the graphics card, thereby further improving rendering efficiency and program running speed.

[0082] Step S102, send the model rendering task to the rendering router; the rendering router is used to send the model rendering task to at least one rendering server; any rendering server is used to obtain corresponding result data based on the model rendering task, and send the result data to the rendering router.

[0083] In the embodiments of the present application, the rendering engine can send the model rendering task to the rendering router. The rendering router can be a hardware device that connects two or more networks, and can complete network layer relay and layer 3 relay tasks, and store and packet forward data packets between different networks.

[0084] In the embodiments of the present application, the rendering router can send the rendering model task to at least one rendering server according to a preset rule. The at least one rendering server can be a rendering server cluster, which is a group of computers connected together through a communication protocol, capable of migrating workloads from an overloaded computer to other computers in the cluster, and has the load balancing feature.

[0085] The preset rule can be a routing strategy formulated according to different sizes and categories of model rendering tasks. After any rendering server receives the model rendering task sent by the rendering router respectively, the model rendering task is processed to obtain processed result data. The processing content can include converting a three-dimensional model from a three-dimensional model grid into an image presenting a realistic feeling through a calculation manner, and the calculation process can include comprehensive variables such as light and auxiliary light, material texture and texture, camera-related settings, and the like. After obtaining the result data corresponding to the model rendering task, the rendering server sends the result data to the rendering router.

[0086] In step S103, a protocol response message sent by the rendering router is received; the protocol response message is generated based on result data of the model rendering task.

[0087] In the embodiment of the present application, the rendering engine can receive the protocol response message sent by the rendering router, and the protocol response message can be generated by the rendering router based on result data obtained by processing the model rendering task according to a preset algorithm.

[0088] In one embodiment, if the remote rendering protocol is formulated based on a WebAssembly binary format, and the corresponding rendering engine is encapsulated by the remote rendering communication protocol, then the result data containing the model rendering result is also in a binary format. The protocol response message is generated from the result data, which is also in a binary format at this time.

[0089] In step S104, the protocol response message is parsed to obtain a rendering result of the model rendering task; the rendering result is used to generate a display page of an application.

[0090] In the embodiment of the present application, after the rendering engine receives the protocol response message, the protocol response message is parsed to obtain data containing the rendering result of the model rendering task from the protocol response message.

[0091] The data not containing the rendering result can be discarded, for example, the part used for communication in the response header of the protocol response message, and the data containing the rendering result can be encoded or compressed, and further analysis is needed to obtain the rendering result data that can be used to generate an application display interface.

[0092] In one embodiment, when the model rendering task is for a large model or a super large model, the rendering router can split the model rendering task into at least one subtask, and then send the subtasks to at least one rendering server respectively; any rendering server can process the subtasks to obtain result data corresponding to the subtasks.

[0093] The router can divide, distribute and coordinate the model rendering task when splitting the model rendering task into multiple sub-tasks. The sub-task can be a simple rendering object or a part of a large rendering task, and is divided based on tiled rendering (Tiled rendering). Each sub-task can correspond to a grid or a tile. The rendering router can be used to send the sub-tasks to at least one rendering server according to a preset distribution rule. Any rendering server processes the sub-task assigned to it to obtain the result data corresponding to the sub-task. The rendering server can process the sub-task in parallel based on its high-performance hardware and software. The processing content can include converting a three-dimensional model from a three-dimensional model grid to an image that presents a realistic feeling through calculation. The calculation process can include light and auxiliary light, material texture and texture, camera-related settings, and other comprehensive variables.

[0094] In one embodiment, the protocol response message is obtained by the rendering router merging the result data corresponding to the sub-tasks.

[0095] Specifically, when the model rendering task is split into at least one sub-task, any rendering server processes the sub-task assigned to it to obtain the result data corresponding to the sub-task. When there are multiple sub-tasks, multiple result data are obtained. The rendering router can be used to merge the multiple result data into a protocol response message according to a preset algorithm.

[0096] In one embodiment, step S104, parsing the protocol response message to obtain the rendering result of the model rendering task further includes:

[0097] (1) parsing the protocol response message to obtain at least one rendering block data;

[0098] (2) writing the rendering block data to the graphics card; the graphics card is used to process the rendering block data to obtain the rendering result of the model rendering task.

[0099] The protocol response message can be in binary format. The rendering engine can be used to parse the protocol response message to obtain at least one rendering block data. The rendering block data can be a data set representing the rendering result of each grid or tile of an image. The block data is associated with each other and has a sequence.

[0100] In one embodiment, if the remote rendering protocol is based on the WebAssembly binary format, and the corresponding rendering engine is encapsulated by the remote rendering communication protocol, then the result data containing the model rendering result is also in binary format. The protocol response message is generated from the result data, which can also be in binary format. Further, the rendering engine can parse the protocol response message to obtain at least one binary rendering block data that can be directly recognized by the video memory, and then write the binary rendering block data directly to the graphics card according to a specific algorithm. The graphics card displays the rendering block data to achieve the effect of quickly displaying the three-dimensional model.

[0101] In the embodiments of the present application, a model rendering method is provided, as shown in Figure 2 The method can be applied to a rendering router and includes the following steps:

[0102] In step S201, a model rendering task sent by a rendering engine is received. The rendering engine and at least one rendering server are respectively connected. The rendering engine is obtained based on encapsulation of a remote rendering communication protocol. The remote rendering communication protocol is a set of remote rendering communication protocols based on a binary format.

[0103] In the embodiments of the present application, the rendering router can accept the model rendering task sent by the rendering engine. The rendering router can be a hardware device connecting two or more networks, which can complete network layer relay and layer 3 relay tasks, and store and forward packets between different networks. The rendering router can also be a relay server with routing function, which further classifies or preprocesses the received model rendering task. The rendering router can also be a cloud computing rendering system containing a router or a relay server, which also has the functions of sending and receiving tasks and integration.

[0104] In the embodiments of the present application, the rendering engine can be respectively connected with at least one rendering server, and the connection can be a socket long connection. The rendering engine can be obtained based on encapsulation of a remote rendering communication protocol, and the remote rendering communication protocol can be based on WebAssembly.

[0105] WebAssembly is a binary format that provides a way to run code written in various languages in the Web at a speed close to the original, greatly improving the performance of JS and the portability of web pages.

[0106] In step S202, the model rendering task is sent to at least one rendering server. Any rendering server is used to obtain corresponding result data based on the model rendering task and send the result data.

[0107] In the embodiments of the present application, the rendering router can send the rendering model tasks to at least one rendering server according to preset rules. The preset rules can be a routing strategy, which is a strategy for routing publishing and receiving. To determine the routing strategy, it can be determined first which routing protocol to use. Different routing protocols can produce different routing tables due to different implementation mechanisms, different overhead calculation rules, and different priority definitions, even for the same network structure. The routing strategy can be based on normal routing protocols and set rules according to requirements. The results of generating, publishing, and selecting routes can be changed by changing parameters or control methods.

[0108] The rendering server can be a rendering server cluster, which is a group of computers connected together through a communication protocol, capable of migrating workloads from an overloaded computer to other computers in the cluster, and has the feature of load balancing. Any rendering server receiving a model rendering task can process it, and the processing content can include generating scene graphics with realistic effects based on a three-dimensional model. The rendering server can send the result data corresponding to the processed model rendering task to the rendering router.

[0109] In step S203, the result data sent by the rendering server is received, and a protocol response message is generated based on the result data of the model rendering task.

[0110] In the embodiments of the present application, the rendering router receives the result data sent by the rendering server, and generates a protocol response message based on the result data of the model rendering task. The protocol response message can be generated according to a preset algorithm, and the protocol response message can include a response line, a response header, and a response body. The response line can include protocol version, status code, and status description information; the response header can include the name and version of the server application software, the media type of the entity text sent to the recipient, and the length of the entity text; and the response body can store the result data.

[0111] In one embodiment, if the remote rendering protocol is based on the WebAssembly binary format and the corresponding rendering engine is encapsulated by the remote rendering communication protocol, then the result data containing the model rendering result is also in binary format. The protocol response message is generated from the result data, which is also in binary format at this time.

[0112] In step S204, the protocol response message is sent to the rendering engine; the rendering engine is used to parse the protocol response message to obtain the rendering result corresponding to the model rendering task; and the rendering result is used to generate a display page of an application.

[0113] In the embodiments of the present application, the rendering router sends the protocol response message to the rendering engine, and the rendering engine parses the protocol response message to obtain data containing the rendering result of the model rendering task. The data without the rendering result can be discarded, for example, the part for communication in the response header of the protocol response message, and the data containing the rendering result may be encoded or compressed, and further parsing is required to obtain the rendering result data that can be used to generate the application display interface.

[0114] In one embodiment, when the model rendering task is for a large model or a super large model, the rendering router can split the model rendering task into at least one subtask, and then send the subtasks to at least one rendering server respectively; any of the rendering servers is used to process the subtask to obtain the result data corresponding to the subtask.

[0115] In one embodiment, before the rendering router splits the model rendering task, it can also include a parsing process. When the rendering router splits the model rendering task into multiple subtasks, it can divide, allocate and coordinate the model rendering task.

[0116] Among them, the task division can be a structured task definition method, which follows the principle of top-down refinement, splits the model rendering task into relatively independent and mutually related subtasks, and the decomposition result can be stored in the database in the form of a table.

[0117] Among them, the subtask can be a simple rendering object or a part of a large model rendering task, and it is also divided based on tiled rendering (Tiled rendering, tile rendering). Each subtask can correspond to a grid or a tile. The division of the model rendering task can also be based on task constraints, which refer to various constraint relationships existing within the rendering object, such as size, position and topological relationship, etc. After the model rendering task is split, these constraint relationships become constraints between subtasks. The division of the model rendering task can also be the division of geometric elements in a three-dimensional model scene, that is, the rendering object is divided into several independent and disjoint subareas, which can be sent by the rendering router to at least one rendering server as subtasks, and then the result data of all subtasks after rendering is combined.

[0118] In one embodiment, the protocol response message is obtained by the rendering router combining the result data corresponding to the subtasks.

[0119] Specifically, the rendering router can split the model rendering task into at least one subtask, and send the subtasks to at least one rendering server respectively, any rendering server processes the subtask allocated to itself, and obtains the result data corresponding to the subtask after processing. When there are multiple subtasks, multiple result data are obtained.

[0120] The rendering router can combine the multiple result data into a protocol response message according to a preset algorithm. The protocol response message can include a response line, a response header and a response body, and the result data can be stored in the response body. The preset algorithm can correspond to the division and allocation method of the model rendering task. For example, when the rendering model task is divided according to the image grid, after receiving the result data of each subtask fed back by the rendering server, the result data is synthesized according to a certain grid arrangement order and stored in the response body of the protocol response message in order, so as to facilitate subsequent analysis of the protocol response message.

[0121] In an embodiment, the rendering engine can be used to analyze the protocol response message, obtain at least one rendering block data and write the rendering block data to the graphics card; the graphics card is used to process the rendering block data to obtain the rendering result of the model rendering task. If the remote rendering protocol is based on the WebAssembly binary format, and the corresponding rendering engine is encapsulated by the remote rendering communication protocol, then the result data containing the model rendering result is also in binary format. The protocol response message is generated from the result data, which can also be in binary format at this time. Further, the rendering engine can analyze the protocol response message to obtain at least one binary rendering block data, and according to a specific algorithm, the binary rendering block data can be directly written to the graphics card. The graphics card displays the rendering block data to achieve the effect of quickly displaying the three-dimensional model.

[0122] In order to more clearly explain the model loading method of the present application, the model loading method will be further described below in combination with specific examples.

[0123] In an embodiment, the present application provides a model rendering method, such as Figure 3 , comprising the following steps:

[0124] Step S300, the browser calls the rendering engine;

[0125] Step S301, the rendering engine establishes a connection with the rendering server;

[0126] Step S302, the rendering engine sends a model rendering task to the rendering router;

[0127] Step S303, the rendering router splits the model rendering task into at least one subtask;

[0128] Step S304, the rendering router parses the model rendering task and sends at least one subtask to at least one rendering server in the rendering server cluster respectively;

[0129] Step S305, any rendering server processes the subtask assigned to itself;

[0130] Step S306, any rendering server sends the result data corresponding to the subtask to the rendering router;

[0131] Step S307, the rendering router combines all the result data corresponding to the subtasks into a protocol response message;

[0132] Step S308, the rendering router sends the protocol response message to the rendering engine;

[0133] Step S309, the rendering engine parses the protocol response message to obtain multiple rendering block data in the message;

[0134] Step S310, write the rendering block data into the video memory;

[0135] Step S311, the browser displays the corresponding 3D model.

[0136] Taking the rendering of the model in the client browser as an example, further description is made in combination with specific examples.

[0137] In one example, the present application provides a model rendering method, such as Figure 4 , comprising the following steps:

[0138] (1) the user accesses a Web three-dimensional visualization application through a browser A, and the browser A establishes a socket long connection with a rendering server through a rendering engine; wherein the rendering engine is obtained by encapsulating a remote rendering protocol, and the remote rendering communication protocol is based on WebAssembly;

[0139] (2) the rendering engine sends a rendering task request to a rendering router;

[0140] (3) the rendering router parses the rendering task request and splits it into multiple subtasks, and sends at least one subtask to at least one rendering server according to a routing rule;

[0141] (4) the rendering server processes the subtask based on its high-performance hardware and software in parallel, obtains corresponding result data, and sends the result data to the rendering router;

[0142] (5) After all the subtasks of a character have been rendered, the rendering router merges the result data of all the subtasks into a protocol response message according to the algorithm. This message contains binary rendering block data that can be recognized by the video memory. The rendering router sends the protocol response message to the rendering engine.

[0143] (6) The rendering engine parses the protocol response message, identifies multiple binary rendering block data in the message, and writes the binary rendering block data directly into the video memory in sequence according to a specific algorithm. The browser will then quickly display the corresponding 3D scene.

[0144] In the above example, by distributing the client's rendering tasks to a backend rendering server cluster with better hardware configuration and stronger computing power, the client's resource consumption is reduced. At the same time, a remote rendering communication protocol is developed based on the WebAssembly binary format, and a corresponding rendering engine is encapsulated based on this protocol, which can directly write the binary format rendering result data to the graphics card and improve the program's running speed in the Web. This achieves the effect of improving rendering efficiency and graphics computing performance, and can ensure real-time rendering capabilities even for displaying large or ultra-large 3D scene models.

[0145] This application provides a model rendering device, such as... Figure 5 As shown, the model rendering device 500 may include: a connection module 501, a rendering module 502, a first receiving module 503, and a first parsing module 504, wherein,

[0146] The connection module 501 is used to receive access instructions for the application and establish connections with at least one rendering server; wherein, the rendering engine is obtained by encapsulating a remote rendering communication protocol, which is a set of remote rendering communication protocols based on binary format.

[0147] The rendering module 502 is used to send model rendering tasks to the rendering router; the rendering router is used to send the model rendering tasks to at least one rendering server respectively; any rendering server is used to obtain the corresponding result data based on the model rendering task and send the result data to the rendering router.

[0148] The first receiving module 503 is used to receive the protocol response message sent by the rendering router; the protocol response message is generated based on the result data of the model rendering task.

[0149] The first parsing module 504 is used to parse the protocol response message to obtain the rendering result of the model rendering task; the rendering result is used to generate the application's display page.

[0150] In the embodiments of the present application, the rendering router is configured to split the model rendering task into at least one subtask, and send the subtasks to at least one rendering server respectively; and any rendering server is configured to process the subtasks to obtain corresponding result data of the subtasks.

[0151] In one of the embodiments, the protocol response message is obtained by the rendering router by merging the result data corresponding to the subtasks.

[0152] In one of the embodiments, when the first analysis module 504 analyzes the protocol response message to obtain the rendering result of the model rendering task, the first analysis module 504 is specifically configured to:

[0153] analyze the protocol response message to obtain at least one rendering block data;

[0154] write the rendering block data into a graphics card; and the graphics card is configured to process the rendering block data to obtain the rendering result of the model rendering task.

[0155] The embodiments of the present application provide a model rendering device, as shown in Figure 6 The model rendering device 600 can include a receiving module 601, a sending module 602, a second receiving module 603, and a second analysis module 604, wherein,

[0156] The receiving module 601 receives a model rendering task sent by a rendering engine; the rendering engine is connected to at least one rendering server; the rendering engine is obtained based on a remote rendering communication protocol encapsulation; the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a binary format;

[0157] The sending module 602 sends the model rendering task to at least one rendering server respectively; any rendering server is configured to obtain corresponding result data based on the model rendering task and send the result data;

[0158] The second receiving module 603 is configured to receive a protocol response message sent by a rendering router; the protocol response message is generated based on the result data of the model rendering task;

[0159] The second analysis module 604 is configured to analyze the protocol response message to obtain a rendering result of the model rendering task; the rendering result is used to generate a display page of an application.

[0160] In the embodiments of the present application, when the sending module 602 sends the model rendering task to at least one rendering server respectively, the sending module 602 is specifically configured to:

[0161] split the model rendering task into at least one subtask, and send the subtasks to at least one rendering server respectively;

[0162] Each rendering server is used to process the subtask and obtain the result data corresponding to the subtask.

[0163] In one embodiment, when the second receiving module 603 generates a protocol response message based on the result data of the model rendering task, it is specifically used for:

[0164] The result data corresponding to the subtasks are merged to obtain the protocol response message.

[0165] In one embodiment, the rendering engine is used to parse the protocol response message, obtain at least one rendering block data, and write the rendering block data to the graphics card; the graphics card is used to process the rendering block data to obtain the rendering result of the model rendering task.

[0166] The aforementioned model rendering device reduces client resource consumption by distributing client rendering tasks to a backend rendering cluster with better hardware configuration and stronger computing power. At the same time, it develops a remote rendering communication protocol based on binary format and encapsulates a corresponding rendering engine based on this protocol, enabling it to directly write rendering result data to the graphics card, thereby improving rendering efficiency and graphics computing performance. Even for large or ultra-large 3D scene models with display requirements, it can guarantee real-time rendering capabilities.

[0167] This application provides an electronic device comprising: a memory and a processor; at least one program stored in the memory, which, when executed by the processor, can achieve the following compared to existing technologies: by distributing the client's rendering tasks to a backend rendering cluster with better hardware configuration and stronger computing power, the client's resource consumption is reduced; at the same time, a remote rendering communication protocol is formulated based on binary format, and a corresponding rendering engine is encapsulated based on this protocol. This engine can receive binary format data that the graphics card can recognize and can communicate directly with the graphics card, thereby improving rendering efficiency and graphics computing performance. Even for large or ultra-large 3D scene models requiring display, real-time rendering capability can be guaranteed.

[0168] In one alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of this electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0169] The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in connection with the disclosure. The processor 4001 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0170] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 7 In the figure, only one thick line is used, but it does not mean that there is only one bus or only one type of bus.

[0171] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, an optical disk storage (including a compact disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0172] The memory 4003 is configured to store application program codes for implementing the solutions of the present application, and the processor 4001 is configured to control the execution of the application program codes stored in the memory 4003. The processor 4001 is configured to execute the application program codes stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0173] The electronic device includes, but is not limited to, mobile terminals such as mobile phones, notebook computers, PADs, and the like, and fixed terminals such as digital TVs, desktop computers, and the like.

[0174] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run on a computer, the computer can execute the corresponding content in the foregoing method embodiments.

[0175] It should be understood that, although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or other steps, sub-steps or stages.

[0176] The above is only some embodiments of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method of model rendering, characterized by, The application is applied to a rendering engine, and comprises: receiving an access instruction for an application, and establishing a connection with at least one rendering server respectively; wherein the rendering engine is obtained based on a remote rendering communication protocol encapsulation, the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a WebAssembly binary format, and is used for receiving a binary data format adapted to GPU processing; sending a model rendering task to a rendering router; the rendering router is used for sending the model rendering task to at least one rendering server based on tile rendering; any rendering server is used for obtaining corresponding result data based on the model rendering task, and sending the result data to the rendering router; receiving a protocol response message sent by the rendering router; the protocol response message is generated based on the result data of the model rendering task; parsing the protocol response message to obtain at least one binary rendering block data, and the data format of the binary rendering block data is a binary data format adapted to GPU processing; writing the binary rendering block data into a GPU; the GPU is used for processing the rendering block data to obtain a rendering result of the model rendering task; and the rendering result is used for generating a display page of the application.

2. The model rendering method of claim 1, wherein, The rendering router is used for splitting the model rendering task into at least one subtask, and sending the subtask to at least one rendering server respectively; any rendering server is used for processing the subtask to obtain result data corresponding to the subtask.

3. The model rendering method of claim 2, wherein, The protocol response message is obtained by the rendering router by merging the result data corresponding to the subtask.

4. A method of model rendering, characterized by, The application is applied to a rendering router, and comprises: receiving a model rendering task sent by a rendering engine; the rendering engine has a connection with at least one rendering server respectively; the rendering engine is obtained based on a remote rendering communication protocol encapsulation, the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a WebAssembly binary format, and is used for receiving a binary data format adapted to GPU processing; sending the model rendering task to at least one rendering server based on tile rendering; any rendering server is used for obtaining corresponding result data based on the model rendering task and sending the result data; receiving the result data sent by the rendering server, and generating a protocol response message based on the result data of the model rendering task; sending the protocol response message to the rendering engine; the rendering engine is used for parsing the protocol response message to obtain at least one binary rendering block data, and the data format of the binary rendering block data is a binary data format adapted to GPU processing; the binary rendering block data is written into a GPU; the GPU is used for processing the rendering block data to obtain a rendering result of the model rendering task; and the rendering result is used for generating a display page of the application.

5. The model rendering method of claim 4, wherein, sending the model rendering task to at least one rendering server comprises: splitting the model rendering task into at least one subtask, and sending the subtasks to at least one of the rendering servers respectively; wherein any of the rendering servers is configured to process the subtasks to obtain corresponding result data of the subtasks.

6. The model rendering method of claim 5, wherein, generating a protocol response message based on the result data of the model rendering task, including: merging the result data corresponding to the subtasks to obtain the protocol response message.

7. A model rendering apparatus, characterized by comprising: including: a connection module configured to receive an access instruction for an application, and establish a connection between the connection module and at least one rendering server respectively; wherein the rendering engine is obtained based on a remote rendering communication protocol encapsulation, the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a WebAssembly binary format, and the rendering engine is configured to receive a binary data format adapted to GPU processing; a rendering module configured to send a model rendering task to a rendering router; the rendering router is configured to send the model rendering task to at least one of the rendering servers based on tile rendering; and any of the rendering servers is configured to obtain corresponding result data based on the model rendering task, and send the result data to the rendering router; a first receiving module configured to receive a protocol response message sent by the rendering router; the protocol response message is generated based on the result data of the model rendering task; a first analysis module configured to analyze the protocol response message to obtain at least one binary rendering block data, the binary rendering block data is in a binary data format adapted to GPU processing; and the binary rendering block data is written into a GPU; the GPU is configured to process the rendering block data to obtain a rendering result of the model rendering task; and the rendering result is used to generate a display page of the application.

8. A model rendering apparatus, characterized by comprising: including: a receiving module configured to receive a model rendering task sent by a rendering engine; the rendering engine has a connection with at least one rendering server; the rendering engine is obtained based on a remote rendering communication protocol encapsulation, the remote rendering communication protocol is a set of remote rendering communication protocols formulated based on a WebAssembly binary format, and the rendering engine is configured to receive a binary data format adapted to GPU processing; a sending module configured to send the model rendering task to at least one of the rendering servers based on tile rendering; any of the rendering servers is configured to obtain corresponding result data based on the model rendering task and send the result data; a second receiving module configured to receive the result data sent by the rendering server, and generate a protocol response message based on the result data of the model rendering task; a second analysis module configured to send the protocol response message to the rendering engine; the rendering engine is configured to analyze the protocol response message to obtain at least one binary rendering block data, the binary rendering block data is in a binary data format adapted to GPU processing; the binary rendering block data is written into a GPU; the GPU is configured to process the rendering block data to obtain a rendering result of the model rendering task; the rendering result is used to generate a display page of the application.

9. An electronic device, comprising: including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the model rendering method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer storage medium is used to store computer instructions, when it runs on the computer, makes the computer can execute the model rendering method of any one of the above claims 1 to 6.

Images