Rendering method, device and system

Abstract

The invention discloses a rendering method, device and system, which can ensure the normal operation of an application as much as possible. The method comprises the following steps: executing rendering on a first region of a first picture of the first application to obtain a first rendering result of the first region; and the terminal sends a first rendering request to a server, and obtains the second rendering result from the server. And then, the terminal fuses the first rendering result and the second rendering result to obtain the first picture, and displays the first picture.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a rendering method, device and system. Background Art

[0002] Currently, users can use terminals to experience high-definition 3D games. In order to improve the realism and image quality of 3D games, developers can design and add a variety of special effects to 3D games, such as light and shadow effects. This results in a large amount of GPU computing resources being consumed when rendering the game screen in 3D games, which in turn causes freezes and frame drops when running 3D games, making it impossible to ensure a good gaming experience.

[0003] In order to solve the problem of lag in 3D game operation, some game manufacturers have provided a cloud gaming solution, which is to use cloud servers to complete the logic of game operation. However, this solution has high requirements on the network quality between the terminal and the cloud server. Once there is a problem with the network, the 3D game cannot run normally. Summary of the invention

[0004] The embodiments of the present application provide a rendering method, device and system to achieve normal operation of an application.

[0005] In order to achieve the above-mentioned purpose, the embodiment of the present application adopts the following technical solution.

[0006] In the first aspect, a rendering method provided by an embodiment of the present application is applied to a terminal. The execution subject of the method can be a terminal or a component (for example, a chip, a chip system or a processor of the terminal). The following description is taken as an example that the execution subject is a terminal. The terminal is installed with a first application. The method includes: performing rendering on a first area of ​​a first screen of the first application to obtain a first rendering result of the first area. The terminal sends a first rendering request to a server and obtains the second rendering result from the server. Afterwards, the terminal merges the first rendering result and the second rendering result to obtain the first screen, and displays the first screen.

[0007] The first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area.

[0008] In the present application, during the operation of the first application, the terminal and the server can collaboratively process the rendering task of the first application, thereby avoiding the performance bottleneck caused by the server processing the rendering task alone, and ensuring that the first application can operate normally as much as possible.

[0009] In a possible design of the first aspect, the method may further include:

[0010] Performing rendering on a third area of ​​the second screen of the first application to obtain a third rendering result of the third area;

[0011] Sending a second rendering request to the server, where the second rendering request is used to request the server to render a fourth area of ​​the second picture to obtain a fourth rendering result; the fourth area is an area of ​​the second picture excluding the third area;

[0012] When the fourth rendering result is not obtained from the server, the second picture is presented according to the third rendering result.

[0013] In this method, the terminal renders the third area in the game screen. In this way, even if the network quality between the terminal and the server is poor and the terminal cannot receive or cannot receive the rendering result of the server in time, it will not affect the terminal's generation of the game screen containing the third area (such as the area where the character is located), so as to meet the demand for real-time generation and presentation of the game screen containing the third area and improve the smoothness of the game screen display.

[0014] In a possible design of the first aspect, the present invention further includes:

[0015] When the network quality between the terminal and the server is restored to meet the preset conditions, a rendering request is sent to the server. In this way, if the network quality does not meet the conditions, the terminal does not send a rendering request to the server, and the rendering is performed locally by the terminal. Subsequently, when the network quality meets the conditions, the terminal can hand over the rendering task of subsequent frames to the server.

[0016] In a possible design of the first aspect, the first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture;

[0017] And / or, the third area is a dynamic picture area in the second picture, and the fourth area is a static picture area in the second picture; or, the third area is a low-order high-frequency area in the second picture, and the fourth area is a high-order low-frequency area in the second picture.

[0018] In some examples, the terminal renders the game screen in areas with high real-time requirements (such as the area where the characters are located), or it is understood that the terminal renders the game screen in areas that have a high impact on screen smoothness, user operation, or user vision. In this way, even if the network quality between the terminal and the server is not good, the terminal cannot receive or cannot receive the rendering results of the server in a timely manner, it will not affect the terminal's generation of the game screen containing this part of the area (such as the area where the characters are located), so as to meet the real-time generation and presentation of the game screen containing this part of the area, and improve the smoothness of the game screen display.

[0019] In a possible design of the first aspect, sending a first rendering request to a server includes:

[0020] Determining the network quality between the terminal and the server;

[0021] If the network quality between the terminal and the server meets a preset condition, the first rendering request is sent to the server.

[0022] If the network quality meets the conditions, it means that the network quality between the terminal and the server is good, and the server can return the rendering result of the first application's screen to the terminal as soon as possible after rendering, thereby improving the smoothness of the operation of the first application.

[0023] On the contrary, if the network quality does not meet the conditions, it means that the network quality between the mobile phone and the server is poor. In this case, in order to avoid the server performing rendering and failing to return the rendering results to the terminal in time, the terminal can not send a rendering request to the server, and perform rendering locally. Later, when the network quality meets the conditions, the terminal can hand over the rendering tasks of subsequent frames to the server.

[0024] In a possible design of the first aspect, the network quality between the terminal and the server meeting the preset condition includes satisfying at least one of the following conditions:

[0025] The packet loss rate between the terminal and the server is less than or equal to a first threshold, the bandwidth between the terminal and the server is greater than or equal to a second threshold, the transmission delay between the terminal and the server is less than or equal to a third threshold, and the transmission rate between the terminal and the server is greater than or equal to a fourth threshold.

[0026] In a possible design of the first aspect, the present invention further includes:

[0027] During the running of the first application, receiving a first operation;

[0028] In response to the first operation, launching a second application;

[0029] The first task is performed by the second application.

[0030] Since some tasks of the terminal (such as the first task) mostly rely on local resources of the mobile phone, compared with the related cloud gaming technology in which all game logic runs on the server, the server cannot directly use the corresponding resources of the terminal, and the processing efficiency of executing the first task is low. In this application, during the operation of the first application, after the terminal receives the first operation, it can directly launch the local second application to process the first task.

[0031] In a possible design of the first aspect, the present invention further includes:

[0032] Acquire data of the first screen of the first application;

[0033] According to a rendering strategy, the first area and the second area in the first picture are determined.

[0034] In a possible design of the first aspect, the rendering strategy includes: whether to enable the end-cloud collaboration capability when the first application is running, the preset conditions for enabling the end-cloud collaboration capability, and the screen area that needs to be rendered by the server when the first application is running.

[0035] Through the rendering strategy, the terminal can determine the screen area that needs to be rendered by the terminal and the screen area that needs to be rendered by the server. In this way, through end-cloud collaborative rendering, the efficiency of the first application's screen rendering can be improved, and the smooth display of the first application's screen can be supported as much as possible to ensure the normal operation of the first application.

[0036] In a possible design of the first aspect, the rendering strategy is a strategy preset by a developer in the first application.

[0037] In this way, after the developer sets the rendering strategy in the first application, during the operation of the first application, end-cloud collaborative rendering can be achieved according to the rendering strategy to improve rendering efficiency.

[0038] In a possible design of the first aspect, the present invention further includes:

[0039] Determine information of a first rendering resource corresponding to the second area; the first rendering request includes information of the first rendering resource; the information is used by the server to search whether the first rendering resource is stored.

[0040] In this way, the server can determine whether the rendering resource has been uploaded based on the rendering resource information (such as hash value). For rendering resources that have been uploaded to the server in the past, the terminal does not need to upload the rendering resource again, thereby reducing transmission overhead.

[0041] In a possible design of the first aspect, the first task includes a payment task, and the second application is an application with a payment function.

[0042] In this method, during the operation of the first application, tasks such as payment that rely on the support of terminal functions are processed by the terminal. Therefore, the terminal can quickly process payment tasks through the terminal functions, thereby improving the processing efficiency of payment tasks.

[0043] In a possible design of the first aspect, the first application includes a 3D application, and the 3D application includes at least one of the following applications: a 3D game application, a 3D social application, and a 3D live broadcast application.

[0044] In this way, for 3D applications that focus on interactive experience, the end and cloud can collaborate to render the 3D application screen during operation, which can improve rendering efficiency and enhance the interactive experience between users and 3D applications.

[0045] In a possible design of the first aspect, performing rendering on a first area of ​​a first screen of the first application includes:

[0046] Rendering is performed on a first area of ​​a first screen of the first application by a graphics processor GPU.

[0047] In a second aspect, a rendering method is provided, which is applied to a server or a component (for example, a chip, a chip system or a processor of a server), and the method includes:

[0048] receiving a first rendering request from a terminal, wherein the first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area;

[0049] The second rendering result is sent to the terminal; the second rendering result is used to be merged with the first rendering result obtained by the terminal rendering the first area to obtain the first picture.

[0050] In a possible design of the second aspect, the first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture.

[0051] Through the reasonable division of computing power between the terminal and the cloud, tasks that need to be calculated on the cloud and tasks that need to be calculated on the terminal side can be allocated on demand on the terminal side. For example, tasks that are not sensitive to latency (such as rendering tasks in the second area) can be assigned to the cloud for running. Even if the network is interrupted or the network quality is poor, the basic local operating experience of the terminal can be guaranteed.

[0052] Optionally, after the rendering resources of the static picture area are uploaded to the cloud, they can be stored by the cloud. There is no need to upload the rendering resources repeatedly in the future. Only the information of the rendering resources (such as hash value) needs to be uploaded to realize the retrieval of the rendering resources on the cloud.

[0053] In a third aspect, a rendering method is provided, which is applied to a terminal or a corresponding component, wherein the terminal is installed with a first application, and the method includes:

[0054] When the network quality between the terminal and the server can meet a preset condition, sending a rendering request to the server, wherein the rendering request is used to request the server to perform a first rendering task on a first picture;

[0055] receiving, from the server, a first rendering result obtained by the server executing the first rendering task;

[0056] According to the first rendering result, the first picture is presented.

[0057] In a possible design of the third aspect, the preset conditions include:

[0058] The packet loss rate between the terminal and the server is less than or equal to a first threshold, the bandwidth between the terminal and the server is greater than or equal to a second threshold, the transmission delay between the terminal and the server is less than or equal to a third threshold, and the transmission rate between the terminal and the server is greater than or equal to a fourth threshold.

[0059] In a possible design of the third aspect, the first rendering task is a rendering task for a first area of ​​the first picture; and the method further includes:

[0060] executing a second rendering task for a second area of ​​the first picture to obtain a second rendering result for the second area; the second area is an area of ​​the first picture other than the first area;

[0061] Presenting the first frame according to the first rendering result includes:

[0062] The first picture is presented according to a first rendering result of the first area and a second rendering result of the second area.

[0063] Among them, the first area and the second area can refer to any possible design of the first aspect or the second aspect mentioned above.

[0064] In this way, by handing over the rendering tasks of the second area (such as high computing power, insensitivity to latency) to the server, other tasks (such as rendering tasks and payment tasks in the first area) are still executed by the terminal, which can improve the screen rendering efficiency of the first application and ensure the user experience of using the first application.

[0065] In a possible design of the third aspect, the present invention further includes:

[0066] Presenting the first picture according to the first rendering result of the first area and the second rendering result of the second area includes:

[0067] If the second rendering result is not received within the preset period of time, the first picture is presented according to the first rendering result.

[0068] In this way, when the end-side rendering request is uploaded to the cloud, the rendering tasks that need to be processed by the cloud and the rendering tasks that need to be processed by the end-side can be dynamically allocated according to business needs, network conditions and other conditions to improve rendering efficiency.

[0069] In a possible design of the third aspect, the present invention further includes:

[0070] Receiving a first operation from a user;

[0071] In response to the operation, launching a second application;

[0072] The corresponding task is executed by the second application.

[0073] In a fourth aspect, a rendering method is provided, which is applied to a terminal or a corresponding component, wherein the terminal is installed with a first application, and the method includes:

[0074] During the process of running the first application, receiving a first operation performed by a user;

[0075] In response to the operation, launching a second application;

[0076] The corresponding task is executed by the second application.

[0077] As a possible implementation manner, the second application runs on the CPU.

[0078] In a fifth aspect, a rendering device is applied to a terminal or a corresponding component, wherein the terminal is installed with a first application, and the device includes:

[0079] A processing module, configured to perform rendering on a first area of ​​a first screen of the first application to obtain a first rendering result of the first area;

[0080] A communication module, configured to send a first rendering request to a server, wherein the first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area;

[0081] The communication module is further used to obtain the second rendering result from the server;

[0082] The processing module is further used to merge the first rendering result and the second rendering result to obtain the first picture;

[0083] The display module is used to display the first picture.

[0084] In a possible design of the fifth aspect, the processing module is further configured to perform rendering on a third area of ​​the second screen of the first application to obtain a third rendering result of the third area;

[0085] The communication module is further used to send a second rendering request to the server, wherein the second rendering request is used to request the server to render a fourth area of ​​the second picture to obtain a fourth rendering result; the fourth area is an area of ​​the second picture excluding the third area;

[0086] The display module is further configured to present the second picture according to the third rendering result when the fourth rendering result is not obtained from the server.

[0087] In a possible design of the fifth aspect, the first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture;

[0088] And / or, the third area is a dynamic picture area in the second picture, and the fourth area is a static picture area in the second picture; or, the third area is a low-order high-frequency area in the second picture, and the fourth area is a high-order low-frequency area in the second picture.

[0089] In a possible design of the fifth aspect, sending a first rendering request to a server includes:

[0090] Determining the network quality between the terminal and the server;

[0091] If the network quality between the terminal and the server meets a preset condition, the first rendering request is sent to the server.

[0092] In a possible design of the fifth aspect, the network quality between the terminal and the server meeting the preset condition includes satisfying at least one of the following conditions:

[0093] The packet loss rate between the terminal and the server is less than or equal to a first threshold, the bandwidth between the terminal and the server is greater than or equal to a second threshold, the transmission delay between the terminal and the server is less than or equal to a third threshold, and the transmission rate between the terminal and the server is greater than or equal to a fourth threshold.

[0094] In a possible design of the fifth aspect, the device further includes an input module, configured to receive a first operation during the execution of the first application;

[0095] The processing module is further configured to launch a second application in response to the first operation; and execute the first task through the second application.

[0096] In a possible design of the fifth aspect, the processing module is further used to:

[0097] Acquire data of the first screen of the first application;

[0098] According to a rendering strategy, the first area and the second area in the first picture are determined.

[0099] In a possible design of the fifth aspect, the rendering strategy includes: whether to start the end-cloud collaboration capability when the first application is running, the preset conditions for starting the end-cloud collaboration capability, and the screen area that needs to be rendered by the server when the first application is running.

[0100] In a possible design of the fifth aspect, the rendering strategy is a strategy preset by a developer in the first application.

[0101] In a possible design of the fifth aspect, the processing module is further used to:

[0102] Determine information of a first rendering resource corresponding to the second area; the first rendering request includes information of the first rendering resource; the information is used by the server to search whether the first rendering resource is stored.

[0103] In a possible design of the fifth aspect, the first task includes a payment task, and the second application is an application with a payment function.

[0104] In a possible design of the fifth aspect, the first application includes a 3D application, and the 3D application includes at least one of the following applications: a 3D game application, a 3D social application, and a 3D live broadcast application.

[0105] In a possible design of the fifth aspect, performing rendering on a first area of ​​a first screen of the first application includes:

[0106] Rendering is performed on a first area of ​​a first screen of the first application by a graphics processor GPU.

[0107] In a sixth aspect, a rendering device is provided, comprising:

[0108] A communication module, configured to receive a first rendering request from a terminal, wherein the first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area;

[0109] The communication module is further used to send the second rendering result to the terminal; the second rendering result is used to be merged with the first rendering result obtained by the terminal rendering the first area to obtain the first picture.

[0110] In a possible design of the sixth aspect, the first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture.

[0111] In a seventh aspect, a rendering device is provided, comprising:

[0112] a communication module, configured to send a rendering request to the server when the network quality between the terminal and the server meets a preset condition, wherein the rendering request is used to request the server to perform a first rendering task on a first picture;

[0113] The communication module is further configured to receive, from the server, a first rendering result obtained by the server executing the first rendering task;

[0114] The display module is used to present the first picture according to the first rendering result.

[0115] In a possible design of the seventh aspect, the preset conditions include:

[0116] The packet loss rate between the terminal and the server is less than or equal to a first threshold, the bandwidth between the terminal and the server is greater than or equal to a second threshold, the transmission delay between the terminal and the server is less than or equal to a third threshold, and the transmission rate between the terminal and the server is greater than or equal to a fourth threshold.

[0117] In a possible design of the seventh aspect, the first rendering task is a rendering task for a first area of ​​the first picture; and the method further includes:

[0118] executing a second rendering task for a second area of ​​the first picture to obtain a second rendering result for the second area; the second area is an area of ​​the first picture other than the first area;

[0119] Presenting the first frame according to the first rendering result includes:

[0120] The first picture is presented according to a first rendering result of the first area and a second rendering result of the second area.

[0121] Among them, the first area and the second area can refer to any possible design of the first aspect or the second aspect mentioned above.

[0122] In a possible design of the seventh aspect, presenting the first picture according to the first rendering result of the first area and the second rendering result of the second area includes:

[0123] If the second rendering result is not received within the preset period of time, the first picture is presented according to the first rendering result.

[0124] In a possible design of the seventh aspect, an input module is further included, which is used to receive a first operation from a user;

[0125] The processing module is further used to launch a second application in response to the operation; and execute the corresponding task through the second application.

[0126] According to an eighth aspect, a rendering device is provided, comprising:

[0127] An input module, used for receiving a first operation from a user during the running of the first application;

[0128] The processing module is used to launch a second application in response to the operation; and execute a corresponding task through the second application.

[0129] As a possible implementation manner, the second application runs on the CPU.

[0130] In the ninth aspect, an embodiment of the present application provides a terminal, which includes a touch screen, a memory, a transceiver, one or more processors, multiple applications, and one or more programs; wherein the one or more programs are stored in the memory, and the transceiver is used to transmit or receive wireless signals; when the one or more processors execute the one or more programs, the terminal implements the method described in the first aspect.

[0131] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in any of the above aspects.

[0132] In the eleventh aspect, an embodiment of the present application provides a chip system, which includes one or more processors. When the one or more processors execute instructions, the one or more processors execute the method as described in the first aspect, or the method as described in the second aspect.

[0133] In the twelfth aspect, the present application provides a device, which terminal includes a processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, it executes the method described in any possible design of any of the above aspects of the present application.

[0134] In a thirteenth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in any aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0135] Figure 1 This is a schematic diagram of the scenario of cloud gaming technology;

[0136] Figure 2 A flowchart for application development used in the embodiments of the present application;

[0137] Figure 3 A schematic diagram of an application screen of an embodiment of the present application;

[0138] Figure 4 A schematic diagram of the structure of a rendering system provided in an embodiment of the present application;

[0139] Figure 5 A schematic diagram of the structure of a rendering system provided in an embodiment of the present application;

[0140] Figure 6 A schematic diagram of rendering resources provided in an embodiment of the present application;

[0141] Fig. 7A A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0142] Figure 7B A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0143] Figure 8 A flowchart of a rendering method provided in an embodiment of the present application;

[0144] Fig. 9 A schematic diagram of a scene of a rendering method provided in an embodiment of the present application;

[0145] Fig. 10AA schematic diagram of a scene of a rendering method provided in an embodiment of the present application;

[0146] Fig. 10B A flowchart of a rendering method provided in an embodiment of the present application;

[0147] Fig.11 A schematic diagram of a scene of a rendering method provided in an embodiment of the present application;

[0148] Fig.12 A flowchart of a rendering method provided in an embodiment of the present application;

[0149] Fig.13 A flowchart of a rendering method provided in an embodiment of the present application;

[0150] Fig.14 A schematic diagram of the structure of a rendering device provided in an embodiment of the present application;

[0151] Fig.15 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0152] Figure 1 The scheme of cloud gaming in the related art is shown. In this scheme, a client of a game application is installed and run in a terminal, and a server of a game application is installed and run in a cloud server, which is used in conjunction with the client. The terminal receives the control instructions input by the game player and sends the control instructions to the cloud server, which processes the tasks during the operation of the game application. After the cloud server processes the tasks, it forms the corresponding processing results and sends the processing results to the terminal. For example, the player moves the characters in the game through the client in the terminal. After the terminal receives the mobile operation input by the player, it sends the mobile operation to the cloud server. The cloud server renders the game screen according to the mobile operation, and returns the rendered game screen to the terminal, and the terminal presents the game screen through the client. For another example, if the player wants to buy game equipment while playing the game, he can click the purchase button in the client interface. After the terminal receives the user's purchase operation, it sends the purchase operation to the cloud server. The cloud server triggers the terminal to execute the payment process according to the user's purchase operation. For example, the cloud server sends a QR code to the terminal, and after the terminal receives the QR code, it scans the QR code to jump to the corresponding payment interface to execute the payment process.

[0153] It can be seen that in the above cloud gaming solution, the terminal is only responsible for receiving user operations related to the game application, and the tasks of the game application are all handled by the cloud server. On the one hand, it will consume a lot of computing resources of the cloud server, such as CPU and GPU resources, and the computing cost of the cloud server is high. On the other hand, since the tasks of the entire game application are handled by the cloud server, and the payment application for payment is not installed in the cloud server, the cloud server cannot directly execute the payment process, but needs to trigger the terminal to execute the payment process through methods such as QR code, and the payment process is cumbersome.

[0154] In addition, cloud gaming methods are highly dependent on network quality. Once the network between the terminal and the cloud server is interrupted or unstable, it will lead to high packet loss rate and large latency, causing the cloud server to delay the return of the game screen to the terminal, and then the terminal will also experience delays and freezes in displaying the game screen, and the game application will not be able to run smoothly.

[0155] The present application embodiment provides a rendering method, which can be applied to 3D application scenarios, such as 3D games, 3D digital human social networking, 3D digital human live broadcast, etc. Alternatively, the method can also be applied to other scenarios, such as other applications with high requirements for fluency.

[0156] like Figure 2 An example of a development and use process of a 3D application is shown, which may include:

[0157] 1. Developers use the 3D engine integrated development environment (IDE) to develop 3D applications. 3D applications can be game applications or non-game 3D applications.

[0158] 2. Application development. The application development process includes the following steps:

[0159] 1) Scene development: Through the scene development process, developers can place objects in the 3D world in corresponding positions and debug the effects, such as placing characters and static objects in 3D applications, adding lights, adding special effects, etc.

[0160] 2) Script development: Through script development, a logic script (also called logic code) can be obtained to describe the specific business logic of the 3D application. Script development includes but is not limited to the development of backpack systems and fighting systems.

[0161] 3) Debugging and development: During this process, developers can use the 3D engine IDE to debug the functions of the 3D application.

[0162] 4) Compilation and packaging: When the functions of the 3D application are developed and debugged, the developer can use the compilation and packaging function provided by the 3D engine IDE to package the entire 3D application project into an executable file, such as an APK installation package.

[0163] The installation package obtained by compilation and packaging includes the logic code obtained by script development, 3D engine runtime (Runtime), and scenes obtained by scene development.

[0164] The 3D engine runtime is the specific implementation of the application programming interface (API) that the engine opens to developers. The engine runtime is decoupled from the logic of the application. For example, the 3D engine runtime has the capabilities of the engine framework, but is not limited to basic capabilities such as task scheduling, basic algorithm library, and inter-module communication. For another example, the 3D engine runtime also has capabilities such as physical simulation, animation simulation, particle simulation, and rendering engine. During the running stage of the 3D application, the 3D application inputs data to the engine runtime, such as input materials, game logic and other data. The engine runtime returns results based on the input data, such as returning the particle simulation results in the game screen.

[0165] Scene: The scene contains objects, and the objects have corresponding resources (also called assets). Resources include but are not limited to model resources, material resources, and texture resources.

[0166] For example, Figure 3 The scene shown includes objects such as a person 202 and a television 201 . Figure 3 In the example, the outline of the character 202 can be represented by a 3D model resource, and the materials of different parts of the character 202 can be represented by material resources. For example, the material of the skin and the material of the clothes (such as cloth material) can be represented by corresponding material resources. The pattern and texture of the clothes of the character 202 can be represented by corresponding texture resources.

[0167] In the embodiment of the present application, when developing an application, the developer adds the logical function development of the end-cloud scheduling strategy. The end-cloud scheduling strategy may include at least one of the following: whether to enable the end-cloud collaboration capability, the conditions for enabling the end-cloud collaboration capability, and the tasks that need to be processed by the cloud when the 3D application is running (such as the screen area rendered by the cloud).

[0168] End-cloud collaboration refers to the collaboration between the terminal and the cloud server (referred to as the server) to process tasks in order to support the operation of 3D applications. In some examples, all tasks to support the operation of 3D applications are processed by the terminal. Alternatively, some tasks are processed by the terminal and some tasks are processed by the server. In this way, the terminal at least processes part of the tasks, which can avoid the problems caused by the server processing all the tasks of the 3D application and ensure the normal operation of the 3D application as much as possible.

[0169] The conditions for starting the end-cloud collaboration capability include, but are not limited to, the network quality between the end and the cloud meeting the preset conditions. The network quality between the end and the cloud meeting the preset conditions includes meeting at least one of the following conditions:

[0170] The packet loss rate between the end and the cloud is less than or equal to the first threshold, the bandwidth between the end and the cloud is greater than or equal to the second threshold, the transmission delay between the end and the cloud is less than or equal to the third threshold, and the transmission rate between the end and the cloud is greater than or equal to the fourth threshold.

[0171] Of course, the preset condition may also be other conditions, such as throughput being greater than a threshold. The embodiment of the present application does not limit the preset condition, as long as the network quality between the end and the cloud is good enough to render the screen of the 3D application without delay, or with a small degree of delay, and the screen display of the 3D application is smooth.

[0172] Taking 3D games as an example, when the terminal is running a 3D game, for area A (such as the background) in a frame of 3D game screen that does not require high latency, the server renders area A, and for area B (such as the area where the characters are located) in the frame of 3D game screen that has high latency requirements, the terminal renders area B. Afterwards, the terminal merges area A and area B into a complete frame of 3D game screen, and displays the frame of 3D game screen during the running of the 3D game. It can be seen that this method no longer completely relies on the server for rendering when rendering the 3D game screen, but the terminal can render the screen area with high latency requirements. Therefore, the smoothness of the 3D application operation is no longer completely dependent on the network quality between the terminal and the server, which can improve the smoothness of the 3D application operation.

[0173] As a possible implementation method, developers can configure the end-cloud scheduling strategy through the development interface of the 3D engine IDE. In some examples, buttons or controls can be set on the development interface, and developers can set the end-cloud scheduling strategy through the corresponding controls. For example, Figure 3 In the example, interface 20 is used as a development interface. A switch 203 is provided on interface 20. When switch 203 is turned on, the end-cloud collaboration capability is enabled, that is, the end-cloud collaboration capability is started. In some examples, after starting the end-cloud collaboration capability, the terminal and the server can collaborate to realize the screen rendering in the 3D application, such as rendering the screen of the scene shown in the game interface.

[0174] As another possible implementation method, the developer adds corresponding code to the logic code, and calls this part of the code when the 3D application is running to enable the end-cloud collaboration capability.

[0175] Similarly, developers can set the conditions for enabling end-cloud collaboration capabilities and tasks that need to be processed by the cloud through the development interface or logic code.

[0176] 3. After the developer completes the development of the 3D application, the 3D application will be put on the application market through operational activities for users to download and use.

[0177] 4. Users download the APK of the 3D application through the application market.

[0178] 5. The user opens the 3D application on a terminal (such as a mobile phone), and the terminal runs the 3D application.

[0179] After the developer configures the end-cloud scheduling strategy for the 3D application, during the operation of the 3D application, the 3D application will collaborate with the server according to the configured end-cloud scheduling strategy to render the 3D application screen.

[0180] Figure 4 It is a rendering system composed of terminals and cloud servers. Figure 4 As shown, the rendering system 100 may include a cloud server 101 and at least one terminal 102. The cloud server 101 and the terminal 102 may communicate with each other via a network.

[0181] In some embodiments, Figure 5 ,Terminal 102 may include an input device, a display device, a game running logic module, a graphics rendering system, and a local GPU.

[0182] The input device can be used to receive input operations from the user. In some examples, the input device is a device or module built into the terminal. For example, the input device is a touch screen, and the user can input operations through the touch screen of the terminal. In other examples, the input device can be a device independent of the terminal, such as a handle, keyboard, mouse, etc. Take the 3D application as a game application as an example. For example, the user inputs control instructions through the handle to control the movement of the character in the game.

[0183] The display device can be used to display the screen of a 3D application (such as a game screen). Exemplarily, the display device is a display screen.

[0184] The application running logic module can be used to calculate the tasks of 3D applications. The application running logic module can be located in the game application or the SDK in the game application or the game service module in the mobile phone system, which is used by the upper-level application to call and execute related functions. When the application running logic module is a game service module in the mobile phone system, it can provide related service functions for multiple applications. For example, in a 3D game, the application running logic module can calculate the rendering resources of the game screen and send a rendering request to the graphics rendering system.

[0185] Optionally, the application running logic module may be a functional module in the CPU. Optionally, the application running logic module may also be a software module running on the CPU. Of course, the application running logic module may also be in other forms, and the embodiments of the present application do not limit the specific implementation form of the application running logic module.

[0186] Optionally, the application execution logic module may be included in the engine runtime of the 3D application.

[0187] The graphics rendering system, included in the engine runtime of the 3D application, can be used to process the rendering tasks of the 3D application. Optionally, the graphics rendering system includes a core rendering system and a render hardware interface (RHI).

[0188] The application running logic module calls the graphics rendering system for each frame of the game screen. The graphics rendering system prepares the rendering data of the corresponding frame of the game screen through the core rendering system and loads the pipeline data corresponding to the frame of the game screen. For example, there are 30 or 60 frames of game screen per second. The specific frame rate may be different depending on the performance of the 3D application or device.

[0189] The rendering data and the corresponding pipeline data may be collectively referred to as rendering resources. The pipeline data may be used to indicate that the rendering data is used for rendering through a rendering pipeline.

[0190] Rendering data can be called Draw Packet. Rendering data contains the information required for underlying rendering and can be understood as the data required in the rendering task graph.

[0191] For example, Figure 6 As shown in (a), the rendering data includes: mesh data (mesh) of the 3D model, vertex buffer data (vertex buffer) data, index buffer data (index buffer) data, material data, map data, texture data (texture) and related shader (shader) data.

[0192] The mesh data of a 3D model can be used to represent the shape of the 3D model. The 3D model can be composed of multiple triangular faces. The mesh data includes one or more information such as vertex coordinates, normals, texture coordinates (UV), etc. of the 3D model.

[0193] Shaders can be used to render images. Through shaders, you can customize the graphics card rendering algorithm to achieve the desired effect. Shaders can include vertex shaders and pixel shaders. Vertex shaders can be used to calculate the vertices of each triangle of the 3D model in the scene, and the calculation results can be used for the final pixel rendering. Pixel shaders can be used to calculate the lighting, color and other information in the scene in units of pixels. Shader data includes data information used by shaders in the process of rendering images.

[0194] Pipeline data can be called Frame Graph, which can be used to represent a rendering task graph or pipeline. Pipeline data can be understood as a kind of configuration data. By reading the configuration data, the device can know the channels (passes) included in the rendering process, the input and output of each channel, and the order in which the rendering data is used. The device can use the rendering data in order through each channel for rendering.

[0195] A channel can be understood as a processing flow, or a processing step. The processing step may contain multiple sub-steps. Optionally, each channel can be implemented by one or more rendering algorithms.

[0196] Alternatively, pipeline data can be represented by a directed acyclic graph. For example, taking a game application as an example, Figure 6 (b), the device processes the depth information of objects in the game screen through GbufferPass, and stores the processed information in DepthBuffer. Depth information includes but is not limited to the distance between the object and the camera in the scene, and the occlusion between objects. The device can also obtain one or more of the following information through GbufferPass: the reaction of the object to light, the roughness of the object. The device can also obtain rendering data such as the normal of the object through GbufferPass, and the one or more information can be stored in Gbuffer (such as Gbuffer0-Gbuffe2). Afterwards, the device can process and calculate the lighting conditions of the object through LightingPass to obtain LightingBuffer. Afterwards, the device can perform post-processing on the information in LightingBuffer and DepthBuffer through PostProcessPass. For example, if the jagged edges in the picture are relatively large, anti-aliasing processing is performed. Afterwards, backup processing can be performed, and the relevant backup data can be stored in BackBuffer.

[0197] Figure 6 (b) only shows the general workflow of the channel during the rendering process, not the specific rendering process.

[0198] Usually, each GPU has its own rendering logic. There are differences in the rendering logics of different GPUs when rendering. RHI can be used to shield the differences caused by the rendering logics of different GPUs and provide a common interface layer. In this way, the core rendering system in the graphics rendering system can call the GPU of the terminal 102 for rendering through the unified interface provided by RHI.

[0199] Optionally, the terminal 102 may be a mobile phone, tablet computer, desktop, laptop, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, smart TV, netbook, virtual reality device or other device installed with 3D applications. The specific form of the terminal is not particularly limited in the embodiments of the present application.

[0200] In some embodiments, Figure 5 The cloud server 101 may include a graphics rendering system, and the graphics rendering system may include a data storage retrieval service and an RHI. The data storage retrieval service may call the GPU of the cloud server 101 through the RHI to perform rendering.

[0201] In the embodiment of the present application, some functions of the engine runtime on the terminal side can be deployed to the cloud side to form a graphics rendering system on the cloud side, which is equivalent to the engine runtime on the terminal side and the graphics rendering system on the cloud side forming a distributed engine. Subsequently, the graphics rendering system on the cloud side can provide unified services for the terminal side, so that the cloud side can cooperate with the terminal side to support the normal operation of 3D applications. In addition, compared with the related art that the cloud side needs to deploy a server that matches the client on the terminal side (such as the client of application 1 is installed on the terminal side, then the cloud side needs to deploy the server of application 1; the client of application 2 is installed on the terminal side, then the cloud side needs to deploy the server of application 2), in the method of the embodiment of the present application, the graphics rendering system on the cloud side uniformly handles the related tasks of the 3D application, the graphics rendering system is decoupled from the client, and a set of standard APIs can be uniformly provided to the outside, without the need to deploy a server on the cloud side that matches the client on the terminal side, which can reduce the deployment complexity of the cloud side.

[0202] For example, Fig. 7A A schematic diagram of the structure of a device (such as terminal 102) is shown.

[0203] The device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0204] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0205] The controller can be the nerve center and command center of the device. The controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of instruction fetching and execution.

[0206] The processor 110 may also be integrated with an internal memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or is cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0207] The charging management module 140 is used to receive charging input from a charger.

[0208] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the display screen 194, the camera 193 and the like.

[0209] The wireless communication function of the device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0210] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied on the device. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc., applied on the device.

[0211] The device realizes the display function through GPU, display screen 194, and application processor, etc. GPU is a microprocessor for image processing, connecting display screen 194 and application processor.

[0212] The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0213] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0214] The device can realize the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0215] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, etc. of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0216] Camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.

[0217] In some embodiments, the device may include 1 or N cameras 193 , where N is a positive integer greater than 1. The specific position and form of the camera 193 are not limited in the embodiment of the present application.

[0218] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0219] Video codecs are used to compress or decompress digital video. A device can support one or more video codecs. In this way, the device can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0220] NPU is a neural network (NN) computing processor. It can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0221] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0222] The internal memory 121 may be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the device by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor. The internal memory 121 may be any type of memory inside the device. The internal memory 121 may be an independent module or may be integrated with other modules.

[0223] The device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0224] The audio module 170 is used to convert digital audio data into analog audio electrical signals for output, and is also used to convert analog audio electrical signal input into digital audio data. The audio module 170 may include an analog / digital converter and a digital / analog converter. For example, the audio module 170 is used to convert the analog audio electrical signals output by the microphone 170C into digital audio data. The audio module 170 may also be used to encode and decode audio data. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

[0225] The speaker 170A, also called a "speaker", is used to convert analog audio electrical signals into sound signals. The device can listen to music or listen to hands-free calls through the speaker 170A.

[0226] The receiver 170B, also called a "handset", is used to convert analog audio electrical signals into sound signals. When the device receives a call or voice message, the voice can be heard by placing the receiver 170B close to the human ear.

[0227] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into analog audio electrical signals. When making a call or sending a voice message, the user can speak by approaching the microphone 170C with his mouth to input the sound signal into the microphone 170C. The microphone 170C can be a built-in component of the device or an external accessory of the device.

[0228] In some embodiments, the device may include one or more microphones 170C, wherein each microphone or multiple microphones working together can collect sound signals from all directions and convert the collected sound signals into analog audio electrical signals, and can also achieve noise reduction, sound source identification, or directional recording functions, etc.

[0229] The embodiment of the present application does not impose any specific limitation on the number of microphones 170C.

[0230] The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0231] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the device. In other embodiments of the present application, the device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0232] For example, Figure 7B Another exemplary structure of the device is shown. Figure 5 As shown, the device includes: a processor 501, a memory 502, and a transceiver 503. The implementation of the processor 501 and the memory 502 can be found in Figure 4 The processor and memory of the device are implemented as shown. The transceiver 503 is used for the device to interact with other devices. The transceiver 503 can be a device based on Wi-Fi, Bluetooth or other communication protocols.

[0233] also, Figure 4 , Figure 5 The structures of other devices in the system shown can be found in Fig. 7A , Figure 7B The structure of the device shown will not be described in detail here.

[0234] Above Fig. 7A , Figure 7B These are only possible examples of the structures of the devices (such as terminals) and other devices (such as servers) in the above system, and do not constitute limitations on the structures of the devices or other devices in the advertising system.

[0235] The following takes a mobile phone as the above-mentioned device as an example and describes in detail the technical solution provided in the embodiment of the present application in combination with the accompanying drawings.

[0236] The rendering method provided in the embodiment of the present application is described below in conjunction with the drawings in the embodiment of the present application.

[0237] Below Figure 5 or Fig. 7A Taking the architecture shown in the figure as an example, the rendering method provided by the embodiment of the present application is described. Each unit in the following embodiment may have Figure 5 or Fig. 7AThe components shown are not described in detail. It should be noted that the message name or number of messages or parameter name in the message exchanged between the various devices in the embodiments of the present application is only an example, and other names can also be used in the specific implementation. In addition, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation.

[0238] A rendering method provided in an embodiment of the present application is described in detail below. The following mainly takes a mobile phone as the terminal and a 3D game application as an example, which is described uniformly here, but does not constitute a limitation on the terminal type and application (APP) type.

[0239] The present application embodiment provides a rendering method, during the running process of the application, the terminal and the server can jointly process the application tasks, avoiding the performance bottleneck caused by the server processing the application tasks alone, and ensuring the normal operation of the application as much as possible. Take the processing flow of a user playing a 3D game on a mobile phone as an example, Figure 8 As shown, the method includes:

[0240] S101. A display device displays the Nth frame of a game screen.

[0241] Exemplarily, the display device is a display screen of a mobile phone.

[0242] S102. The input device receives a control instruction A for the Nth frame of the game screen (which may be referred to as the Nth frame).

[0243] Optionally, the input device may be a handle, a keyboard, or a mobile phone screen. For example, a mobile phone display may be Figure 3 In the Nth frame game screen 20 of the game application shown, the user (player) moves a character 202 in the game screen 20 , and the mobile phone receives a control instruction A for moving the character 202 .

[0244] As a possible implementation manner, after the input device receives the control instruction A input by the player, the control instruction A can be converted into a corresponding input event.

[0245] S103: The game operation logic module receives control instruction A.

[0246] As a possible implementation, the input device may transmit the input event corresponding to the control instruction A to the game running logic module. The game running logic module may be, for example, a module in an engine of a game application.

[0247] As a possible implementation method, the game running logic module can be implemented as a CPU, or a functional module in the CPU.

[0248] S104: The game running logic module determines the rendering resources of the N+1th frame.

[0249] The game operation logic module can perform corresponding game logic calculations according to the player's different control instructions A to determine the rendering resources. Figure 3 As shown in FIG. 1 , if the character 202 in the Nth frame game screen 20 moves, the game operation logic module can calculate according to the duration and direction of the moving operation, and determine the position of the character 202 in the subsequent game screen (such as the N+1th frame) according to the calculation result. For another example, if the player inputs a skill release operation, the game operation logic module can perform corresponding particle simulation or animation simulation calculation according to the skill release operation and the configuration logic of the game application, and determine the rendering effect of the character 202 in the subsequent game screen according to the simulation calculation result.

[0250] Through this step, the game running logic module can determine the rendering resources required for rendering the game screen, such as determining the grid information, position information, and shader (used to calculate the lighting conditions related to the character) information of the character in the game screen.

[0251] S105. The game running logic module sends a rendering request A for the N+1th frame, where the rendering request A is used to indicate rendering resources for the N+1th frame.

[0252] After the game running logic module determines the rendering resources required to render the game screen, it needs to send a rendering request to the graphics rendering system in the mobile phone's game application so that the graphics rendering system calls the GPU to perform rendering.

[0253] Exemplarily, before rendering the N+1th frame of the game screen, the game running logic module sends a rendering request A for the N+1th frame, and the rendering request A is used to indicate the rendering resources of the N+1th frame.

[0254] Optionally, the rendering resource includes rendering data and pipeline data. Optionally, the pipeline data may vary depending on factors such as the terminal model, computing power, and the rendering effect requirements of the game application.

[0255] S106 . The graphics rendering system of the mobile phone determines region A and region B of the N+1th frame.

[0256] In an embodiment of the present application, if the developer configures an end-cloud scheduling strategy (rendering strategy) for the game application during the development stage of the game application, the graphics rendering system of the mobile phone will calculate the area rendered by the server (area A) and the area rendered by the mobile phone (area B) according to the end-cloud scheduling strategy, where area B is the area excluding area A in the N+1th frame.

[0257] In an embodiment of the present application, if the game application is configured and the end-cloud collaboration capability is enabled, the graphics rendering system of the mobile phone can determine the area A in the N+1 frame that can be rendered by the server based on the business needs of the game application, so that the server can share part of the rendering task of the mobile phone, reduce the rendering amount of the mobile phone, and improve rendering efficiency.

[0258] In some embodiments, the game screen includes a static area and a dynamic area. The static area may refer to an area in the game screen where the content remains unchanged, or an area where the degree of change of the picture content between frames is very low (such as below a certain threshold), or an area where the degree of change of the picture content between frames is very slow (such as below a certain threshold). The dynamic area may refer to an area in the game screen where the content changes, or an area where the degree of change of the picture content between frames is very high, or an area where the degree of change of the picture content between frames is very fast. For example, the background (houses, other static objects, etc.) area of ​​the game screen can be used as a static area, and the characters can change with the change of the control instruction A, and the area where the characters are located belongs to the dynamic area.

[0259] As a possible implementation method, the graphics rendering system of the mobile phone can determine that the game screen of the dynamic area in the N+1th frame is rendered locally by the mobile phone, and the game screen of the static area in the N+1th frame is rendered by the server, that is, area A is the static area in the N+1th frame, and area B is the dynamic area in the N+1th frame. In this method, the screen content of the static area in the game screen changes little or does not change. Therefore, the screen of this area is handed over to the server for rendering. Even if the network quality between the mobile phone and the server is poor and the server does not feed back the rendering results to the mobile phone in time, the player will not have a clear perception. The screen content of the dynamic area has changed or changed greatly. Therefore, the screen of this area is handed over to the mobile phone for rendering, which enables the mobile phone to immediately respond to the player's control instruction A to perform rendering, and present the screen of the dynamic area to the player, thereby improving the smoothness of the game operation.

[0260] In other embodiments, the game screen includes high-order low-frequency areas and low-order high-frequency areas. The rendering computing power consumption of the high-order low-frequency area is relatively high, and the player does not have high requirements for the real-time performance of the screen in this area. Even if the screen in this area is not updated in time, the player's perception is not obvious. The high-order low-frequency area can be, for example, but not limited to, a dynamic global illumination area, a light reflection area, a shadow area, etc. The rendering computing power consumption of the low-order high-frequency area is relatively low, and the player has high requirements for the real-time performance of the screen in this area. The mobile phone needs to respond and update the screen in this area immediately after the player issues control instruction A.

[0261] As a possible implementation method, the graphics rendering system of the mobile phone can determine that the low-order high-frequency area in the N+1th frame is rendered by the mobile phone, and the high-order low-frequency area in the N+1th frame is rendered by the server, that is, area A is the high-order low-frequency area in the N+1th frame, and area B is the low-order high-frequency area in the N+1th frame.

[0262] S107 . The graphics rendering system of the mobile phone calls the GPU to render area B.

[0263] As a possible implementation manner, after the graphics rendering system of the mobile phone determines the area B that needs to be rendered by the mobile phone in the N+1th frame, it prepares the rendering data and rendering pipeline of the area B according to the rendering request A.

[0264] As a possible implementation method, after the core rendering system prepares the rendering data of area B and the rendering pipeline of area B, it calls the GPU of the mobile phone to perform rendering.

[0265] For example, Fig. 9 After the mobile phone determines that the area to be rendered by the mobile phone is the area where the character is located in the N+1th frame (an example of area B), the rendering of the area where the character is located is performed.

[0266] S108. The GPU returns the rendering result of area B to the graphics rendering system of the mobile phone.

[0267] S109: The graphics rendering system of the mobile phone sends a rendering request B for area A to the server.

[0268] In some embodiments, the rendering request includes rendering data of region A in the N+1th frame.

[0269] For example, Fig. 9 After the mobile phone determines that the area that needs to be rendered by the server is the area excluding the characters in the N+1th frame (an example of area A), it sends a rendering request B to the graphics rendering system of the server. The rendering request B can carry the rendering data of area A, such as the normals, vertices, maps, textures, materials and other data of objects such as the TV and sofa in area A.

[0270] S110 . The graphics rendering system of the server calls the GPU of the server to render area A in the N+1th frame.

[0271] As a possible implementation method, after receiving the rendering request B, the server starts the corresponding cloud rendering task, and the data storage retrieval service in the graphics rendering system of the server calls the GPU through the RHI to perform rendering.

[0272] Still following the above example, if the rendering request B of the mobile phone carries the rendering data of area A, such as the normals, vertices, maps, textures, materials and other data of objects such as the TV and sofa in area A, the server can render area A according to the rendering data of area A.

[0273] S111. The GPU of the server returns the rendering result of area A.

[0274] As a possible implementation manner, the GPU of the server returns the rendering result of area A to the graphics rendering system of the server, and the graphics rendering system of the server returns the rendering result of area A to the graphics rendering system of the mobile phone.

[0275] S112. The graphics rendering system of the mobile phone merges the rendering result of area A and the rendering result of area B to obtain the N+1th frame (game screen).

[0276] For example, Fig. 9 The mobile phone merges the locally rendered character area B (an example of the first area) with the server-rendered background area A (an example of the second area). The bottom layer is the background and the upper layer is the character, and the N+1th frame of the game screen (an example of the first screen) is obtained.

[0277] For another example, the mobile phone renders the basic outlines of each object in the N+1th frame, and the server renders the dynamic global illumination and reflection effects in the N+1th frame. The mobile phone superimposes and fuses the dynamic global illumination and reflection effects onto the basic outlines of each object to obtain the N+1th frame of the game screen.

[0278] S113. The graphics rendering system of the mobile phone sends data of the N+1th frame to the display device.

[0279] For example, the mobile phone will obtain the following after fusion: Fig. 9 The data of the N+1th frame of the game screen is sent to the display screen.

[0280] S114. The display device displays the N+1th frame.

[0281] Still following the above example, after the display screen receives data of the N+1 frame of game screen, it displays the N+1th frame of game screen.

[0282] The above description takes the rendering of the N+1th frame as an example. The rendering process of other frames of the game screen can refer to the rendering process of the N+1th frame mentioned above.

[0283] In an embodiment of the present application, a flexible end-cloud collaboration strategy can be configured so that tasks during the operation of a 3D application can be assigned to the terminal and server as needed, and the terminal and server work together to support the operation of the 3D application. For example, only the rendering tasks of some high-order low-frequency areas (such as areas with effects such as global illumination and reflection) are assigned to the server. For another example, only the rendering tasks can be assigned to the server for execution, and other tasks other than the rendering tasks (such as payment, account login) are performed by the terminal. For another example, only some rendering tasks are assigned to the server for execution. In this way, the GPU and CPU resources of the terminal can be fully utilized to avoid wasting the computing resources of the terminal.

[0284] The above description uses the example that the mobile phone renders a partial area in the N+1th frame and the server renders another partial area in the N+1th frame. In other embodiments, the rendering task of the complete game screen of the N+1th frame can be handed over to the server for execution, and the mobile phone performs other tasks besides rendering during the game, such as the mobile phone performing payment tasks.

[0285] The above description is based on the example of the terminal receiving the rendering result returned by the server. In other embodiments, the terminal may not receive the rendering result from the server. For example, when the network quality between the terminal and the server is poor, if the terminal does not receive the rendering result from the server within a certain period of time, the terminal can present the 3D game screen according to the local rendering result. Fig. 10A (a), the terminal renders the area where the character is located in the Mth frame of the game screen, and the terminal sends a rendering request to the server. Within a certain period of time, the terminal receives the rendering result of the background area in the Mth frame from the server, and then the terminal merges the rendering result of the background area and the rendering result of the character area to obtain the Mth frame. Fig. 10A (b), the terminal continues to render the area where the character is located in the M+1 frame of the game screen, and the terminal sends a rendering request to the server. Within a certain period of time, the terminal does not receive the rendering result (an example of the fourth rendering result) of the background area (an example of the fourth area) in the M+1 frame from the server, then the terminal can obtain the M+1 frame based on the local rendering result (an example of the third rendering result) of the area where the character is located (an example of the third area). As a possible implementation method, the terminal can fuse the rendering result of the background area in the Mth frame and the rendering result of the area where the character is located in the M+1 frame to obtain the M+1 frame.

[0286] In this method, the terminal renders the area of ​​the game screen that has a higher real-time requirement (such as the area where the character is located), or it is understood that the terminal renders the game screen in the area that has a higher impact on the smoothness of the screen or user operation or user vision. In this way, even if the network quality between the terminal and the server is not good, the terminal cannot receive or cannot receive the rendering result of the server in time, it will not affect the terminal's generation of the game screen containing this part of the area (such as the area where the character is located), so as to meet the demand for real-time generation and presentation of the game screen containing this part of the area, and improve the smoothness of the game screen display.

[0287] The present application also provides a rendering method, whereby the mobile phone can determine whether to hand over the rendering task to the server for execution based on the network quality. Fig. 10B , the method may include:

[0288] For the specific implementation of S201-S205, please refer to the above S101-S105.

[0289] S206. The graphics rendering system of the mobile phone determines that the network quality meets the condition.

[0290] The network quality meets the preset conditions, which may mean that the network quality meets at least one of the following conditions: the packet loss rate is less than or equal to the packet loss rate threshold (first threshold), the bandwidth between the mobile phone and the server is greater than or equal to the bandwidth threshold (second threshold), the transmission delay between the terminal and the server is less than or equal to the third threshold, and the transmission rate between the mobile phone and the server is greater than or equal to the rate threshold (fourth threshold). Of course, the preset conditions can also be other conditions, such as the throughput meets a certain threshold. The embodiments of the present application do not limit the preset conditions, as long as they can be used to judge the quality of the network to determine whether the terminal and the server are co-rendering.

[0291] In the embodiment of the present application, if the network quality meets the conditions, the following S207-S210 can be executed, and the server performs the rendering task of the N+1th frame. The network quality meets the conditions, which means that the network quality between the mobile phone and the server is good, and the server can return the rendering results to the mobile phone in a timely manner after rendering as much as possible, thereby improving the smoothness of the game operation.

[0292] On the contrary, if the network quality does not meet the conditions, it means that the network quality between the mobile phone and the server is poor. In this case, in order to avoid the server performing rendering and failing to return the rendering results to the mobile phone in time, the mobile phone does not send a rendering request to the server, and the mobile phone performs rendering on the N+1 frame locally. Later, when the network quality meets the conditions, the rendering tasks of subsequent frames can be handed over to the server.

[0293] S207: The graphics rendering system of the mobile phone sends a rendering request for the N+1th frame to the graphics rendering system of the server.

[0294] In some embodiments, the rendering request includes rendering data of the N+1th frame.

[0295] For example, when the bandwidth between the mobile phone and the server is high, the mobile phone sends a message to the server such as Fig. 9 The rendering request for the N+1th frame is shown, and the rendering request carries rendering data of objects such as a person, a television, a sofa, etc. in the N+1th frame.

[0296] S208. The graphics rendering system of the server calls the GPU of the server to render the N+1th frame.

[0297] S209. The GPU of the server returns the rendering result of the N+1th frame.

[0298] S210. The graphics rendering system of the mobile phone sends the rendering result of the N+1th frame to the display device.

[0299] S211. The display device displays the N+1th frame according to the rendering result of the N+1th frame.

[0300] In some embodiments, Figure 8 The method shown can be used with Fig. 10B For example, before rendering the N+1th frame of the game screen, the mobile phone can first determine the network quality, such as Fig.11 If the network quality between the mobile phone and the server is good, the mobile phone can hand over the rendering task of some areas in the N+1 frame (such as static area A) to the server, and the rendering task of other areas in the N+1 frame (such as dynamic area B) is performed by the mobile phone itself. On the contrary, if the network quality is poor, the mobile phone renders the N+1 frame locally.

[0301] The present application also provides a rendering method. When a mobile phone uploads rendering data to a server through a rendering request, the mobile phone can only upload the index of the rendering data that has been uploaded to the server in the past, so as to reduce the transmission overhead. Figure 8 The method shown uses the scheme for reducing transmission overhead as an example, such as Fig.12 , the method may include:

[0302] For the specific implementation of S301 - S308 , please refer to the above S101 - S108 .

[0303] S309 . The graphics rendering system of the mobile phone sends a rendering request B for area A to the graphics rendering system of the server. The rendering request B includes an index of the rendering data of area A.

[0304] In the embodiment of the present application, the mobile phone calculates the index of the rendering data of area A and uploads the index of the rendering data to the server. As a possible implementation method, the mobile phone calculates the hash value of the rendering data and uses the hash value as the index of the rendering data.

[0305] S310: The graphics rendering system of the server determines the rendering data that has been uploaded, and obtains other rendering data that has not been uploaded to the server from the terminal.

[0306] The rendering data may include more fine-grained data. Exemplarily, the rendering data includes an action library, a mesh, and a texture. The action library may be used to represent the posture of an object (such as a person).

[0307] After the graphics rendering system of the server receives the index of the rendering data of area A, it determines whether the rendering data has been uploaded before according to the index of the rendering resource. If the rendering data has been uploaded before, there is no need for the mobile phone to upload the rendering data again. The server can query the corresponding rendering data in the server according to the index uploaded by the mobile phone. On the contrary, if the rendering data has not been uploaded before, the server notifies the mobile phone to upload the rendering data. Exemplarily, after receiving the rendering request, the server locally queries the hash value of the mesh of the object (such as character A) that needs to be rendered in area A. After querying, the mesh of character A has been uploaded, and the action library corresponding to the mesh is a standing posture. If the server needs to render the character A in a sitting posture this time, and the mobile phone has not uploaded the action library of the sitting posture, the server can return a notification to the mobile phone, notifying the mobile phone to upload the action library of the sitting posture. After the mobile phone uploads the action library of the sitting posture according to the notification of the server, the server performs rendering on character A according to the action library of the sitting posture uploaded by the mobile phone this time, as well as the rendering data such as the mesh of character A. Only two fine-grained rendering data, action library and mesh, are listed here. In the specific implementation, the server can compare other more or fewer fine-grained rendering data, and determine whether the mobile phone needs to upload the corresponding rendering data based on the hash value.

[0308] As a possible implementation, the server stores the received rendering data in a resource library, calculates a hash value of the rendering data, and stores the hash value of the rendering data. For example, Table 1 shows the rendering data stored in the server and the hash value corresponding to each rendering data. Each color block on the clothing 1 may correspond to a map, and the map may be used for other clothing, such as reusing a map on the clothing. Figure 1 or paste Figure 2 To describe the color of the corresponding area on other clothes.

[0309] Table 1

[0310] Rendering Data Hash value Stickers on clothes 1 Figure 1 xxxxx Stickers on clothes 1 Figure 2 yyyyy Material 1 zzzzz mesh1 aaaaa … …

[0311] As a possible implementation method, after the server receives the hash value of the rendering data of area A from the mobile phone, the server verifies the hash value and queries the server's resource library for the rendering data corresponding to the hash value. If the rendering data corresponding to the hash value is found in the resource library, it means that the rendering data has been uploaded before and the mobile phone does not need to re-upload the rendering data.

[0312] Optionally, the rendering data of area A may be one or more rendering data. For example, the rendering data of area A includes: the background texture in the N+1th frame of the game screen. For another example, the rendering data of area A includes: the background texture in the N+1th frame of the game screen, and the rendering data of the global illumination in the background.

[0313] Optionally, one or more rendering data may be rendering data that has been uploaded to the server in the past, or some rendering data may have been uploaded to the server, while some rendering data may not have been uploaded to the server. This embodiment of the present application does not limit this.

[0314] In the embodiment of the present application, if rendering data A (such as tables and chairs in the game background) has been uploaded in the past, the server can notify the mobile phone to upload rendering data other than rendering data A in area A (such as other rendering data other than the rendering data of tables and chairs in area A). The mobile phone does not need to repeatedly upload rendering data A to the server, which can reduce the upload overhead.

[0315] On the contrary, if the rendering data A has not been uploaded in the past, the server may notify the mobile phone to upload the rendering data of the area A (including the rendering data A corresponding to the upload index and other rendering data).

[0316] S311. The graphics rendering system of the server calls the GPU rendering area A of the server.

[0317] In some embodiments, if the server determines that a certain rendering data required for rendering area A has been uploaded in the past, the server can obtain the rendering data from the resource library. If the server determines that a certain rendering data required for rendering area A has not been uploaded in the past, the server can request, notify, or instruct the mobile phone to upload the rendering data.

[0318] After the server obtains all the rendering data required for rendering area A, it performs rendering of area A.

[0319] Compared to the method of submitting rendering data synchronously every time the terminal submits a rendering request to the cloud side, which results in excessive data transmission and long upload time, the above method, on the one hand, can use the method of calculating the index to establish a globally unique index for the rendering data to be uploaded. In this way, the rendering data only needs to be uploaded for the first time to form a globally unique index on the server. In the subsequent rendering process, when the server needs the rendering data (such as a background map), the terminal only needs to notify the server of the index corresponding to the rendering data. The terminal does not need to repeatedly upload the rendering data. The server can find the rendering data that has been uploaded before through the index. It can be seen that if the server needs a lot of rendering data to perform rendering, this method can avoid repeated uploading of rendering data and save upload bandwidth.

[0320] For example, when the mobile phone renders the Nth frame, it uploads the background map required for the Nth frame to the server, and the server stores the background map in the resource library. Later, when rendering the lower N+1 frame, since the background between the two frames does not change, the mobile phone uploads the index of the background map. The server queries the corresponding background map in the resource library based on the index, and reuses the background map to render the corresponding area in the lower N+1 frame. It can be seen that in some scenarios, the server can reuse the rendering data (such as maps) that the terminal has previously uploaded, without the terminal having to repeatedly upload the rendering data.

[0321] S312-S315 can refer to the above-mentioned S111-S114.

[0322] The present application also provides a rendering method that can improve the processing efficiency of cloud game related tasks during the operation of cloud games. Fig.13 , the method comprises the following steps:

[0323] S401, the game application displays the game screen.

[0324] The game screen is rendered by the server. For example, when the network quality between the server and the mobile phone is good, the server renders the game screen, which can reduce the rendering consumption of the mobile phone without affecting the rendering efficiency. Alternatively, the game screen is rendered by the server and the mobile phone in collaboration. Alternatively, the game screen is rendered by the mobile phone. For example, when the network quality between the server and the mobile phone is poor, the mobile phone renders the game screen, which can avoid rendering delays when the mobile phone loses network.

[0325] S402: The game application receives a control instruction B, where the control instruction B is used to trigger a payment task.

[0326] For example, if a player wants to purchase gaming equipment while playing a cloud game, he or she can click the purchase button in the interface (an example of the first operation).

[0327] S403: The game application sends a control instruction B to the game running logic module.

[0328] After receiving the control instruction B from the player clicking the purchase button, the game application sends the control instruction B to the game running logic module.

[0329] S404: The game running logic module launches the payment application in response to the control instruction B.

[0330] S405: The payment application displays a payment interface.

[0331] In some embodiments, the game running logic module responds to the control instruction B, pulls up the local payment application (an example of the second application) of the mobile phone, and the payment application displays the payment interface. After that, the user pays through the payment interface, and after the payment is completed, the mobile phone can display the payment completion interface.

[0332] Since most tasks such as mobile phone payment rely on local mobile phone resources, compared to the related art where all game logic runs on the server, the server cannot directly use the corresponding resources of the mobile phone, and the payment task processing efficiency is low, in the embodiment of the present application, during the game running process, after the game running logic module receives the control instruction B, it can directly call the local mobile phone resources to process the payment task. For example, the game running logic module can directly pull up the local payment application, and players can pay through the interface of the payment application to purchase game equipment. It can be seen that in this method, during the game running process, tasks such as payment that rely on the functional support in the terminal are processed by the terminal. Therefore, the terminal can quickly process the payment task through the functions in the terminal, thereby improving the processing efficiency of the payment task.

[0333] Here, during the running of a 3D game application (an example of a first application), the terminal directly launches a local payment application to quickly implement a payment function. In other embodiments, during the running of the 3D game application, the terminal can also directly launch other local applications according to the current scenario or user needs, so as to quickly and timely implement corresponding functions / execute corresponding tasks.

[0334] The above method splits and decouples the CPU computing logic and the GPU computing logic. The cloud can perform some GPU computing (such as rendering) while the client side retains some GPU computing functions (such as rendering). In particular, since the cloud has strong computing power, computing tasks that consume a lot of computing power can be handed over to the cloud for execution. For example, cloth simulation and fluid simulation are performed by the cloud.

[0335] It should be noted that the above-mentioned multiple embodiments can be combined and the combined scheme can be implemented. Optionally, some operations in the process of each method embodiment are optionally combined, and / or the order of some operations is optionally changed. In addition, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps. There can also be other execution orders between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed. A person of ordinary skill in the art will think of a variety of ways to reorder the operations described herein. In addition, it should be noted that the process details involved in a certain embodiment of this article are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0336] In addition, some steps in the method embodiment may be equivalently replaced by other possible steps. Alternatively, some steps in the method embodiment may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added in the method embodiment. Alternatively, the execution subject (such as a functional module) of some steps in the method embodiment may be replaced by other execution subjects.

[0337] Furthermore, the above method embodiments may be implemented separately or in combination.

[0338] Some other embodiments of the present application provide a device, which may be the above-mentioned terminal (such as a mobile phone, etc.) or a server. The device may include: a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the device may execute the various functions or steps executed by the mobile phone in the above-mentioned method embodiment. The structure of the device may refer to Figure 3 Device shown.

[0339] Among them, the core structure of the device can be expressed as Fig.14 As shown in the structure, the device includes: a processing module 1301, an input module 1302, a storage module 1303 and a display module 1304.

[0340] The processing module 1301 may include at least one of a central processing unit (CPU), an application processor (AP) or a communication processor (CP). The processing module 1301 may perform operations or data processing related to the control and / or communication of at least one of the other elements of the user device. Specifically, the processing module 1301 may be used to control the content displayed on the main screen according to certain trigger conditions. The processing module 1301 is also used to process the input instructions or data and determine the display style according to the processed data.

[0341] The input module 1302 is used to obtain the instructions or data input by the user and transmit the obtained instructions or data to other modules of the device. Specifically, the input mode of the input module 1302 may include touch, gesture, approaching the screen, etc., or voice input. For example, the input module may be the screen of the device, obtain the user's input operation and generate an input signal according to the obtained input operation, and transmit the input signal to the processing module 1301.

[0342] The storage module 1303 may include a volatile memory and / or a non-volatile memory. The storage module is used to store at least one instruction or data related to other modules of the user equipment.

[0343] The display module 1304 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display, and is used to display content (e.g., text, images, videos, icons, symbols, etc.) that can be viewed by a user.

[0344] Optionally, a communication module 1305 is also included to support personal devices (via a communication network) to communicate with other personal devices. For example, the communication module can be connected to a network via wireless communication or wired communication to communicate with other personal devices or a network server. Wireless communication can use at least one of cellular communication protocols, such as long term evolution (LTE), advanced long term evolution (LTE-A), code division multiple access (CDMA), wideband code division multiple access (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro) or global mobile communication system (GSM). Wireless communication may include, for example, short-range communication. Short-range communication may include at least one of wireless fidelity (Wi-Fi), Bluetooth, near field communication (NFC), magnetic stripe transmission (MST) or GNSS.

[0345] It should be noted that each functional module of the device can execute one or more steps in the above method embodiment.

[0346] The present application also provides a chip system, such as Fig.15 As shown, the chip system includes at least one processor 1401 and at least one interface circuit 1402. The processor 1401 and the interface circuit 1402 can be interconnected via lines. For example, the interface circuit 1402 can be used to receive signals from other devices (such as the memory of the device). For another example, the interface circuit 1402 can be used to send signals to other devices (such as the processor 1401). Exemplarily, the interface circuit 1402 can read the instructions stored in the memory and send the instructions to the processor 1401. When the instructions are executed by the processor 1401, the device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0347] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned device, the device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

[0348] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the mobile phone in the above method embodiment.

[0349] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0350] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0351] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0352] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0353] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0354] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A rendering method, characterized in that: Applied to a terminal, the terminal having a first application installed therein, the method comprising: Perform rendering on a first area of ​​a first screen of the first application to obtain a first rendering result of the first area; Sending a first rendering request to a server, wherein the first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area; Obtaining the second rendering result from the server; Merging the first rendering result and the second rendering result to obtain the first picture; The first screen is displayed.

2. The method according to claim 1, characterized in that Also includes: Performing rendering on a third area of ​​the second screen of the first application to obtain a third rendering result of the third area; Sending a second rendering request to the server, where the second rendering request is used to request the server to render a fourth area of ​​the second picture to obtain a fourth rendering result; the fourth area is an area of ​​the second picture excluding the third area; When the fourth rendering result is not obtained from the server, the second picture is presented according to the third rendering result.

3. The method according to claim 1 or 2, characterized in that: The first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture.

4. The method according to any one of claims 1 to 3, characterized in that: Send the first rendering request to the server, including: Determining the network quality between the terminal and the server; If the network quality between the terminal and the server meets a preset condition, the first rendering request is sent to the server.

5. The method according to claim 4, characterized in that The network quality between the terminal and the server meets the preset condition, which includes satisfying at least one of the following conditions: The packet loss rate between the terminal and the server is less than or equal to a first threshold, the bandwidth between the terminal and the server is greater than or equal to a second threshold, the transmission delay between the terminal and the server is less than or equal to a third threshold, and the transmission rate between the terminal and the server is greater than or equal to a fourth threshold.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: Acquire data of the first screen of the first application; According to a rendering strategy, the first area and the second area in the first picture are determined.

7. The method according to claim 6, characterized in that The rendering strategy includes: whether to enable the end-cloud collaboration capability when the first application is running, the preset conditions for enabling the end-cloud collaboration capability, and the screen area that needs to be rendered by the server when the first application is running.

8. The method according to claim 6 or 7, characterized in that: The rendering strategy is a strategy preset by a developer in the first application.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: Determine information of a first rendering resource corresponding to the second area; The first rendering request includes information about the first rendering resource; The information is used by the server to search whether the first rendering resource is stored.

10. The method according to any one of claims 1 to 9, characterized in that: The first application includes a 3D application, and the 3D application includes at least one of the following applications: a 3D game application, a 3D social application, and a 3D live broadcast application.

11. The method according to any one of claims 1 to 10, characterized in that: Performing rendering on a first area of ​​a first screen of the first application includes: Rendering is performed on a first area of ​​a first screen of the first application by a graphics processor GPU.

12. A rendering method, characterized in that: Applied to a server, the method comprises: receiving a first rendering request from a terminal, wherein the first rendering request is used to request the server to render a second area of ​​the first picture to obtain a second rendering result; the second area is an area of ​​the first picture other than the first area; The second rendering result is sent to the terminal; the second rendering result is used to be merged with the first rendering result obtained by the terminal rendering the first area to obtain the first picture.

13. The method according to claim 12, characterized in that The first area is a dynamic picture area in the first picture, and the second area is a static picture area in the first picture; or, the first area is a low-order high-frequency area in the first picture, and the second area is a high-order low-frequency area in the first picture.

14. A terminal, characterized in that: include: Processor and memory; The memory is used to store computer-executable instructions. The processor is coupled to the memory. When the terminal is running, the processor executes the computer-executable instructions stored in the memory, so that the terminal executes the method according to any one of claims 1 to 11.

15. A server, characterized in that: include: Processor and memory; The memory is used to store computer-executable instructions. The processor is coupled to the memory. When the server is running, the processor executes the computer-executable instructions stored in the memory to enable the server to execute the method according to claim 12 or 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions. When the computer instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 11, or the computer is caused to execute the method according to claim 12 or 13.

17. A chip system, characterized in that: The method comprises one or more processors. When the one or more processors execute instructions, the one or more processors execute the method according to any one of claims 1 to 11, or execute the method according to claim 12 or 13.

18. A rendering system, characterized in that: It comprises the terminal as claimed in claim 14 and the server as claimed in claim 15.

Images