Interactive picture rendering method and device, electronic equipment and storage medium

Abstract

The invention discloses an interactive picture rendering method and device, electronic equipment and a storage medium, and relates to the technical field of cloud, and the method comprises the steps: in the rendering process of a target interactive picture corresponding to a target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive picture, the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed; when a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started, so that the layer synthesis thread executes layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized, and a target interaction picture is obtained; the target layer is any one of the at least one layer which detects the rendering completion signal at present. By using the technical scheme provided by the invention, the timeliness of picture rendering and the fluency of the picture can be improved on the basis of reducing the system resource consumption in the rendering process.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to an interactive screen rendering method, device, electronic device and storage medium. Background Art

[0002] With the development of Internet technology, various applications have shown explosive growth, and the functions of various applications have been continuously enriched and upgraded, which has also enriched the application interaction screens.

[0003] In the related art, the interactive screen of the application needs to be rendered using a GPU (graphics processing unit), and then the layer synthesis is performed in combination with a layer synthesis thread; however, in the related art, in order to save system resources, the layer synthesis thread is completed by relying on fixed-interval pipeline control, that is, the layer synthesis thread is not always running, but runs once at a preset time interval; however, this will result in the inability to perform interactive screen rendering in time after the layer rendering is completed, resulting in problems such as screen rendering delay and screen freeze on the user side. Summary of the invention

[0004] The present application provides an interactive screen rendering method, apparatus, device, storage medium and computer program product, which can improve the timeliness of application interactive screen rendering and improve the smoothness of the screen on the basis of reducing system resource consumption during the rendering process.

[0005] In one aspect, the present application provides an interactive screen rendering method, the method comprising:

[0006] During the rendering process of the target interactive screen corresponding to the target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen, wherein the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed;

[0007] When a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen; the target layer is any layer in the at least one layer for which a rendering completion signal is currently detected.

[0008] On the other hand, an interactive screen rendering device is provided, the device comprising:

[0009] A rendering completion signal detection module is configured to execute, during the rendering process of a target interactive screen corresponding to a target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen, wherein the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed;

[0010] The layer synthesis processing module is configured to start the layer synthesis thread when a rendering completion signal corresponding to the target layer is detected, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen; the target layer is any layer in the at least one layer for which a rendering completion signal is currently detected.

[0011] In an optional embodiment, the layer synthesis processing module includes:

[0012] The first state updating unit is configured to execute, when a rendering completion signal corresponding to the target layer is detected, updating the layer synthesis thread from a ready state to a running state, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer of the at least one layer is synthesized to obtain the target interactive screen.

[0013] In an optional embodiment, the layer synthesis thread is a thread that runs according to a preset cycle; the device further includes:

[0014] a synthesis recognition module, configured to perform synthesis recognition on the rendered image corresponding to the at least one layer to obtain a synthesis recognition result when the current running time corresponding to the layer synthesis thread is reached and the layer synthesis thread is in the ready state;

[0015] The first state updating unit is further configured to execute, when the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered images corresponding to the at least one layer, and when a rendering completion signal corresponding to any unsynthesized image is detected within a preset time period, updating the layer synthesis thread from the ready state to the running state;

[0016] The duration corresponding to the preset time period is shorter than the duration corresponding to the preset cycle, and the target layer includes a layer corresponding to any unsynthesized image for which a rendering completion signal is currently detected.

[0017] In an optional embodiment, the device further comprises:

[0018] The first thread state control module is configured to control the layer synthesis thread to continue to be in the ready state when the synthesis recognition result indicates that the rendered images corresponding to the at least one layer have been synthesized.

[0019] In an optional embodiment, the device further comprises:

[0020] The second thread state control module is configured to control the layer synthesis thread to continue to be in the ready state when the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered image corresponding to the at least one layer, and no rendering completion signal corresponding to any unsynthesized image is detected within the preset time period.

[0021] In an optional embodiment, the device further comprises:

[0022] A parsing and processing module is configured to, when receiving an interaction instruction corresponding to the target application, parse the interaction instruction to obtain parsing information, wherein the parsing information includes rendering instruction information corresponding to each of the at least one layer;

[0023] The layer image rendering module is configured to execute a call to the GPU and render the image corresponding to the at least one layer based on the rendering indication information corresponding to each of the at least one layer.

[0024] In an optional embodiment, the device further comprises:

[0025] An image storage module is configured to store the rendered image corresponding to the target layer into a display memory of the GPU when the image rendering corresponding to the target layer is completed;

[0026] The layer synthesis processing module is specifically configured to start the layer synthesis thread when a rendering completion signal corresponding to the target layer is detected, so that the layer synthesis thread obtains the rendered image corresponding to the target layer from the video memory, and performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen.

[0027] In an optional embodiment, when the target application is a cloud application, the device further includes:

[0028] The target interaction screen push module is configured to push the target interaction screen to the terminal corresponding to the target application, so that the terminal displays the target interaction screen.

[0029] Another aspect provides an electronic device, including: a processor;

[0030] a memory for storing instructions executable by the processor;

[0031] Wherein, the processor is configured to execute the instructions to implement any of the above-mentioned interactive screen rendering methods.

[0032] On the other hand, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned interactive screen rendering methods.

[0033] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the interactive screen rendering method provided in the above various optional implementations.

[0034] The interactive screen rendering method, device, equipment, storage medium and computer program product provided by the present application have the following technical effects:

[0035] In the process of rendering the target interactive screen corresponding to the target application, the present application introduces a detection mechanism for the rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen; and when the rendering completion signal corresponding to any layer is detected, the layer synthesis thread is started, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the any layer, until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen, which can effectively avoid the invalid startup of the layer synthesis thread, and thus can effectively avoid the waste of system resources and the impact on system performance, and when the rendering completion signal generated when the rendering of the image corresponding to each layer is detected, the layer synthesis thread is started, which can reduce invalid waiting and effectively avoid the synthesis delay caused by the periodic startup of the layer synthesis thread, thereby greatly improving the rendering efficiency, the timeliness of the screen rendering and the smoothness of the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1It is a schematic diagram of an application environment of an interactive screen rendering method provided in an embodiment of the present application;

[0038] Figure 2 It is a flowchart of an interactive screen rendering method provided by an embodiment of the present application;

[0039] Figure 3 It is a flowchart of another interactive screen rendering method provided by an embodiment of the present application;

[0040] Figure 4 It is a schematic diagram of a layer synthesis process performed by a layer synthesis thread provided in an embodiment of the present application;

[0041] Figure 5 It is a schematic diagram of a cloud application corresponding to an interactive screen rendering process provided in an embodiment of the present application;

[0042] Figure 6 It is a schematic diagram of a timing path diagram of an existing industry solution in the process of rendering an interactive screen provided by an embodiment of the present application;

[0043] Figure 7 It is a schematic diagram of a timing path diagram of the process of rendering an interactive screen in the solution of the present application provided in an embodiment of the present application;

[0044] Figure 8 It is a structural schematic diagram of an interactive screen rendering device provided in an embodiment of the present application;

[0045] Fig. 9 is a block diagram of an electronic device for interactive screen rendering provided by an embodiment of the present application;

[0046] Fig.10 It is a block diagram of another electronic device for interactive screen rendering provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] The following are definitions of abbreviations and key terms used in the embodiments of the present application:

[0050] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0051] Cloud rendering is to place the rendering process of the application interactive screen in the cloud, use the server resources on the cloud for rendering, and then push the rendering results to the terminal, so as to reduce the requirements for hardware device configuration on the game terminal side.

[0052] Cloud applications are local applications installed in a sandbox on the cloud. Calculation and rendering are performed on the cloud, and then the rendering results are sent to the user end for display via network streaming.

[0053] Link: The entire process from cloud rendering to pushing to the terminal.

[0054] Layer synthesis: The process of combining layer images rendered by the GPU through related synthesizers. For example, there are two layers, one is a rectangular image and the other is a circular image. After the two layer images are superimposed and synthesized, a new image is formed.

[0055] Sandbox: refers to placing related applications in an independent environment to achieve resource isolation. It is widely used in cloud deployment services.

[0056] See also Figure 1 , Figure 1 It is a schematic diagram of an application environment of an interactive screen rendering method provided in an embodiment of the present application, and the application environment may include at least a server 100 and a terminal 200.

[0057] In an optional embodiment, the server 100 can be used for interactive screen rendering processing. The server 100 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0058] In an optional embodiment, the terminal 200 can be used to display an interactive screen and provide interactive services to the user. Specifically, the terminal 200 may include but is not limited to smart phones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, vehicle terminals, smart TVs and other types of electronic devices; it may also be software running on the above electronic devices, such as applications, applets, etc. The operating system running on the electronic device in the embodiment of the present application may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0059] In addition, it should be noted that Figure 1 What is shown is only an application environment of an interactive screen rendering method. For example, the rendering process of the interactive screen can also be executed in a terminal. The embodiments of this specification are not limited to the above.

[0060] In the embodiments of this specification, the server 100 and the terminal 200 may be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitation thereto.

[0061] The following describes an interactive screen rendering method of the present application. Figure 2 It is a flowchart of an interactive screen rendering method provided by an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method may include:

[0062] S201: During the rendering process of a target interactive screen corresponding to a target application, a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen is detected.

[0063] In a specific embodiment, during the rendering process of the target interactive screen corresponding to the target application, the rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen can be detected in real time; the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed. In a specific embodiment, the target interactive screen can be the application screen that currently needs to be rendered. The target interactive screen may include a screen (image) corresponding to at least one layer; for example, in a game application, the target interactive screen may include a screen corresponding to a text layer and a screen corresponding to a game character layer, the text layer refers to a layer that displays text content, and the game character layer may be a layer that displays a game character image.

[0064] In a specific embodiment, the target application may be any application that is currently running interactively. Optionally, the target application may run on the terminal side or on a cloud server, such as cloud applications such as cloud games.

[0065] In an optional embodiment, the above method may further include:

[0066] When receiving the interaction instruction corresponding to the target application, the interaction instruction is parsed to obtain parsing information, where the parsing information includes rendering instruction information corresponding to at least one layer;

[0067] The GPU is called to render images corresponding to the at least one layer based on the rendering instruction information corresponding to each of the at least one layer.

[0068] In a specific embodiment, the user can trigger interaction instructions through the terminal based on actual interaction needs; optionally, taking rendering processing on the server side as an example, the terminal can send interaction instructions to the server; optionally, taking the target application as a cloud application running on a cloud server as an example, the target application can run in a sandbox on the cloud server, and accordingly, the interaction instructions can be received through the sandbox.

[0069] In a specific embodiment, the interaction instructions can be parsed in combination with the parsing rules corresponding to different applications to convert the interaction instructions into information to be rendered by the GPU (i.e., parsed information including rendering indication information corresponding to at least one layer); specifically, the rendering indication information corresponding to any layer can be the rendering content information of the image corresponding to the layer; further, the GPU can render the image corresponding to each layer separately in combination with the rendering indication information corresponding to each layer.

[0070] In the above embodiment, when an interaction instruction corresponding to a target application is received, the interaction instruction is parsed and processed to obtain parsed information including rendering indication information corresponding to at least one layer. Then, the GPU can be called to render an image corresponding to at least one layer based on the rendering indication information corresponding to at least one layer, thereby achieving the rendering of at least one layer image corresponding to the target interaction screen.

[0071] S203: When a rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen.

[0072] In a specific embodiment, when the rendering of the image corresponding to each layer is completed, a rendering completion signal will be generated; specifically, the rendering completion signal can be a signal for calling back the frame output function when the GPU has rendered the image corresponding to a layer, and specifically, the frame output function can be frameReady().

[0073] In a specific embodiment, the target layer may be any layer of the at least one layer for which a rendering completion signal is currently detected.

[0074] In a specific embodiment, Figure 3 As shown, the above method may also include:

[0075] S205: When the image rendering corresponding to the target layer is completed, the rendered image corresponding to the target layer is stored in the video memory of the GPU;

[0076] Correspondingly, when the rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized, and obtaining the target interactive screen may include:

[0077] When a rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started to enable the layer synthesis thread to obtain the rendered image corresponding to the target layer from the video memory, and perform layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen.

[0078] In a specific embodiment, each layer is rendered separately, and the rendering result (image) is saved in the video memory of the GPU. When the rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started. Accordingly, the layer synthesis thread can obtain the rendered image corresponding to the target layer from the video memory, and then perform layer synthesis processing until the rendering completion signal corresponding to at least each layer in the layer is detected, and the rendered image corresponding to each layer in at least one layer is synthesized to obtain the above-mentioned target interactive screen.

[0079] In a specific embodiment, there may be multiple rendering completion signals, that is, after the rendering of the image corresponding to each layer is completed, a signal will be generated, and then the layer synthesis thread will be notified to synthesize. The rendering of each layer is processed in parallel and controlled by the application logic. For example, one layer is always displaying 3D text, and another layer displays a continuously moving character. When multiple rendering completion signals are detected and synthesis is triggered, the layer synthesis thread will process the rendered images corresponding to the multiple layers in sequence according to the position order of the multiple layers. The position order of the multiple layers is controlled by the application logic.

[0080] In a specific embodiment, during the layer synthesis process, the color information of the changed areas of the rendered images corresponding to all layers will be calculated. Then, in order to ensure that the synthesized interactive screen is more realistic, the color information of the changed areas can be combined to recalculate more realistic color information (such as transparency), and the images of the changed areas can be superimposed in combination with the recalculated color information. After superimposing the rendered images corresponding to each layer in at least one layer corresponding to the target interactive screen, the above-mentioned target interactive screen is obtained.

[0081] In a specific embodiment, Figure 4 As shown, Figure 4 It is a schematic diagram of a layer synthesis processing process of a layer synthesis thread provided in an embodiment of the present application; specifically, the image corresponding to each layer (layer image) will be rendered separately, and when the rendering is completed and the frame output function is called back, a rendering completion signal is generated. Furthermore, when the rendering completion signal is generated, the layer synthesis thread can be notified to start.

[0082] In the above embodiment, when the rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started, the rendered image corresponding to the target layer is obtained from the video memory, and the layer synthesis processing is performed. The layer synthesis processing can be performed in a timely manner, and the thread is only started when the layer image is rendered. The waste of system resources can also be effectively avoided. On the basis of reducing the consumption of system resources during the rendering process, the timeliness of the rendering of the application interactive screen can be improved, and the smoothness of the screen can be improved.

[0083] In an optional embodiment, when a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized, and obtaining the target interactive screen may include:

[0084] When the rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is updated from the ready state to the running state, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen.

[0085] In a specific embodiment, before being started, the layer composition thread may be in a ready state. Accordingly, the layer composition thread may be started by updating the layer composition thread from the ready state to the running state.

[0086] In an optional embodiment, the layer synthesis thread may be a thread that runs according to a preset cycle; the above method may further include:

[0087] When the current running time corresponding to the layer synthesis thread is reached and the layer synthesis thread is in a ready state, performing synthesis recognition on a rendered image corresponding to at least one layer to obtain a synthesis recognition result;

[0088] Accordingly, when the rendering completion signal corresponding to the target layer is detected, updating the layer synthesis thread from the ready state to the running state may include:

[0089] When the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered images corresponding to at least one layer, and when a rendering completion signal corresponding to any unsynthesized image is detected within a preset time period, the layer synthesis thread is updated from a ready state to a running state;

[0090] In a specific embodiment, the duration corresponding to the preset time period is less than the duration corresponding to the preset cycle, and the target layer includes a layer corresponding to any unsynthesized image for which a rendering completion signal is currently detected. Specifically, the preset time period can be set in combination with actual application requirements.

[0091] In a specific embodiment, the preset period may be a pre-set period for automatically triggering the start of the layer synthesis thread. Accordingly, the current running time of a new period is reached every preset time interval (preset period); accordingly, the current running time may be the start trigger time corresponding to the layer synthesis thread.

[0092] In the above embodiment, by keeping the layer synthesis thread running according to the preset cycle, it can be compatible with historical solutions, greatly reducing the difficulty and cost of development, and by performing synthesis identification on the rendered image corresponding to at least one layer when the layer synthesis thread in the ready state reaches the current running time, and when the synthesis identification result indicates that there is at least one unsynthesized image in the rendered image corresponding to at least one layer, and when the rendering completion signal corresponding to any unsynthesized image is detected within the preset time period, the layer synthesis thread is updated from the ready state to the running state to start the layer synthesis thread. On the basis of effectively taking into account the historical solutions, the rationality of starting the layer synthesis thread after introducing the detection of the rendering completion signal can be improved, and then on the basis of effectively avoiding the waste of system resources and the impact on system performance, the timeliness of the rendering of the application interactive screen and the smoothness of the screen can be improved.

[0093] In an optional embodiment, the above method may further include:

[0094] When the synthesis identification result indicates that all rendered images corresponding to at least one layer have been synthesized, the layer synthesis thread is controlled to continue to be in a ready state.

[0095] In an optional embodiment, the above method may further include:

[0096] When the synthesis identification result indicates that there is at least one unsynthesized image in the rendered images corresponding to at least one layer, and no rendering completion signal corresponding to any unsynthesized image is detected within a preset time period, the layer synthesis thread is controlled to continue to be in a ready state.

[0097] In the above embodiment, when the synthesis recognition result indicates that the rendered images corresponding to at least one layer have been synthesized, or the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered images corresponding to at least one layer, and the rendering completion signal corresponding to any unsynthesized image is not detected within a preset time period, the layer synthesis thread is controlled to continue to be in a ready state, which can avoid invalid startup of the layer synthesis thread, and further effectively avoid waste of system resources and impact on system performance.

[0098] In an optional embodiment, when the target application is a cloud application, the method further includes:

[0099] The target interactive screen is pushed to the terminal corresponding to the target application so that the terminal displays the target interactive screen.

[0100] In a specific embodiment, taking the target application as a cloud application running in a sandbox of a cloud server as an example, optionally, at least one application may be running in the sandbox, and the at least one application includes the target application, such as Figure 5 As shown, Figure 5This is a schematic diagram of the interactive screen rendering process corresponding to a cloud application provided in an embodiment of the present application. Specifically, when the sandbox receives an interactive instruction, it can enter the interactive instruction parsing process; then, the GPU can perform layer rendering in combination with the parsing information; further, when a rendering completion signal corresponding to any layer is detected, the layer synthesis thread can be started to perform layer synthesis processing until the rendered image of each layer corresponding to the interactive instruction is synthesized to obtain the target interactive screen, and finally, the target interactive screen can be pushed to the terminal for display.

[0101] In the above embodiment, in the cloud rendering scenario, a rendering completion signal detection mechanism is introduced, and when the rendering completion signal is detected, the layer synthesis thread is started to perform layer synthesis. This can effectively avoid the waste of system resources and performance impact caused by the invalid start of the cloud layer synthesis thread, and can reduce the delay of the entire interactive link of the cloud application, thereby greatly improving the timeliness of the corresponding interactive screen rendering of the cloud application and the smoothness of the screen.

[0102] In a specific embodiment, in order to verify the effect of the present application's scheme for controlling the start of the layer synthesis thread by detecting the rendering completion signal, the existing industry scheme for periodically starting the layer synthesis thread and the scheme based on the present application for controlling the start of the layer synthesis thread by detecting the rendering completion signal can be tested separately under the same communication environment and combined with the dot timing method; specifically, the total time from triggering the interaction instruction to the terminal displaying the corresponding target interaction page in the existing industry scheme is 64ms, while the total time from triggering the interaction instruction to the terminal displaying the corresponding target interaction page in the present application scheme is 50ms, that is, by adopting the scheme of the present application, the interaction rendering time can be reduced by 14ms, thereby greatly reducing the rendering delay, thereby improving the timeliness of the application interaction screen rendering and the smoothness of the screen.

[0103] In another specific embodiment, in order to verify the effect of the present application's scheme of controlling the start of the layer synthesis thread by detecting the rendering completion signal, the timing path diagram of the existing industry scheme in the rendering process of the interactive screen and the timing path diagram of the present application scheme in the rendering process of the interactive screen can be respectively captured, such as Figure 6 As shown, Figure 6 This is a schematic diagram of a timing path diagram of an existing industry solution in the process of rendering an interactive screen provided by an embodiment of the present application, from Figure 6 It can be seen that from the completion of rendering to the start time of the next cycle of the layer synthesis thread, it takes 16ms to wait; however, combined with Figure 7 As shown, Figure 7 This is a schematic diagram of a timing path diagram of the process of rendering an interactive screen in the present application solution provided by an embodiment of the present application, from Figure 7It can be seen that there is no waiting from the completion of rendering to the start time of the next cycle of the layer synthesis thread. Directly triggering a synthesis process can greatly reduce the rendering delay, thereby improving the timeliness of the application interactive screen rendering and the smoothness of the screen.

[0104] It can be seen from the technical solutions provided in the above embodiments of this specification that, in the process of rendering the target interactive screen corresponding to the target application, this specification introduces a detection mechanism for the rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen; and when the rendering completion signal corresponding to any layer is detected, the layer synthesis thread is started, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to any layer, until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen, which can effectively avoid invalid startup of the layer synthesis thread, and thus can effectively avoid waste of system resources and impact on system performance, and when the rendering completion signal generated when the rendering of the image corresponding to each layer is detected, the layer synthesis thread is started, which can reduce invalid waiting and effectively avoid synthesis delay caused by periodic startup of the layer synthesis thread, thereby greatly improving rendering efficiency, timeliness of screen rendering and smoothness of the screen.

[0105] The present application also provides an interactive screen rendering device, such as Figure 8 As shown, the above device comprises:

[0106] The rendering completion signal detection module 810 is configured to execute, during the rendering process of the target interactive screen corresponding to the target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen, wherein the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed;

[0107] The layer synthesis processing module 820 is configured to start the layer synthesis thread when a rendering completion signal corresponding to the target layer is detected, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen; the target layer is any layer in at least one layer for which a rendering completion signal is currently detected.

[0108] In an optional embodiment, the layer synthesis processing module 820 includes:

[0109] The first state update unit is configured to execute, when a rendering completion signal corresponding to the target layer is detected, updating the layer synthesis thread from a ready state to a running state, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in at least one layer is synthesized to obtain a target interactive screen.

[0110] In an optional embodiment, the layer synthesis thread is a thread that runs according to a preset cycle; the above-mentioned device also includes:

[0111] The synthesis recognition module is configured to perform synthesis recognition on the rendered image corresponding to at least one layer when the current running time corresponding to the layer synthesis thread is reached and the layer synthesis thread is in a ready state to obtain a synthesis recognition result;

[0112] The first state updating unit is further configured to execute, when the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered images corresponding to at least one layer, and when a rendering completion signal corresponding to any unsynthesized image is detected within a preset time period, updating the layer synthesis thread from a ready state to a running state;

[0113] The duration corresponding to the preset time period is shorter than the duration corresponding to the preset cycle, and the target layer includes a layer corresponding to any unsynthesized image for which a rendering completion signal is currently detected.

[0114] In an optional embodiment, the above device further includes:

[0115] The first thread state control module is configured to control the layer synthesis thread to continue to be in a ready state when the synthesis recognition result indicates that all rendered images corresponding to at least one layer have been synthesized.

[0116] In an optional embodiment, the above device further includes:

[0117] The second thread state control module is configured to control the layer synthesis thread to continue to be in a ready state when the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered image corresponding to at least one layer, and no rendering completion signal corresponding to any unsynthesized image is detected within a preset time period.

[0118] In an optional embodiment, the above device further includes:

[0119] A parsing and processing module is configured to, when receiving an interaction instruction corresponding to a target application, parse the interaction instruction to obtain parsing information, wherein the parsing information includes rendering instruction information corresponding to at least one layer;

[0120] The layer image rendering module is configured to execute the call to the GPU and render the image corresponding to at least one layer based on the rendering indication information corresponding to each of the at least one layer.

[0121] In an optional embodiment, the above device further includes:

[0122] The image storage module is configured to store the rendered image corresponding to the target layer into the video memory of the GPU when the image rendering corresponding to the target layer is completed;

[0123] The layer synthesis processing module 820 is specifically configured to start the layer synthesis thread when a rendering completion signal corresponding to the target layer is detected, so that the layer synthesis thread obtains the rendered image corresponding to the target layer from the video memory, and performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer in at least one layer is synthesized to obtain the target interactive screen.

[0124] In an optional embodiment, when the target application is a cloud application, the apparatus further includes:

[0125] The target interaction screen push module is configured to push the target interaction screen to the terminal corresponding to the target application so that the terminal displays the target interaction screen.

[0126] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0127] Fig. 9 is a block diagram of an electronic device for interactive screen rendering provided by an embodiment of the present application. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an interactive screen rendering method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0128] Fig.10 is a block diagram of another electronic device for interactive screen rendering provided by an embodiment of the present application. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The electronic device includes a processor, a memory and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an interactive screen rendering method is implemented.

[0129] Those skilled in the art will understand that Fig. 9 or Fig.10 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0130] In an exemplary embodiment, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the interactive screen rendering method as in the embodiment of the present disclosure.

[0131] In an exemplary embodiment, a computer-readable storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the interactive screen rendering method in the embodiment of the present disclosure.

[0132] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the interactive screen rendering method provided in the above various optional implementations.

[0133] It is understandable that in the specific implementation of this application, user-related data is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0135] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0136] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An interactive screen rendering method, characterized in that: The method comprises: During the rendering process of the target interactive screen corresponding to the target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen, wherein the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed; When a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen; the target layer is any layer in the at least one layer for which a rendering completion signal is currently detected.

2. The method according to claim 1, characterized in that When a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer in the at least one layer is synthesized, and the target interactive screen is obtained, including: When a rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is updated from the ready state to the running state, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen.

3. The method according to claim 2, characterized in that The layer synthesis thread is a thread that runs according to a preset cycle; the method further includes: When the current running time corresponding to the layer synthesis thread is reached and the layer synthesis thread is in the ready state, performing synthesis recognition on the rendered image corresponding to the at least one layer to obtain a synthesis recognition result; When a rendering completion signal corresponding to the target layer is detected, updating the layer synthesis thread from a ready state to a running state includes: When the synthesis recognition result indicates that there is at least one unsynthesized image in the rendered images corresponding to the at least one layer, and when a rendering completion signal corresponding to any unsynthesized image is detected within a preset time period, updating the layer synthesis thread from the ready state to the running state; The duration corresponding to the preset time period is shorter than the duration corresponding to the preset cycle, and the target layer includes a layer corresponding to any unsynthesized image for which a rendering completion signal is currently detected.

4. The method according to claim 3, characterized in that The method further comprises: When the synthesis identification result indicates that all rendered images corresponding to the at least one layer have been synthesized, the layer synthesis thread is controlled to continue to be in the ready state.

5. The method according to claim 3, characterized in that: The method further comprises: When the synthesis identification result indicates that there is at least one unsynthesized image in the rendered images corresponding to at least one layer, and no rendering completion signal corresponding to any unsynthesized image is detected within the preset time period, the layer synthesis thread is controlled to continue to be in the ready state.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When receiving the interaction instruction corresponding to the target application, parsing the interaction instruction to obtain parsing information, wherein the parsing information includes rendering instruction information corresponding to each of the at least one layer; The GPU is called to render images corresponding to the at least one layer respectively based on the rendering instruction information corresponding to each of the at least one layer.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the image rendering corresponding to the target layer is completed, the rendered image corresponding to the target layer is stored in the video memory of the GPU; When a rendering completion signal corresponding to the target layer is detected, a layer synthesis thread is started, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer in the at least one layer is synthesized, and the target interactive screen is obtained, including: When a rendering completion signal corresponding to the target layer is detected, the layer synthesis thread is started so that the layer synthesis thread obtains the rendered image corresponding to the target layer from the video memory, and performs layer synthesis processing based on the rendered image corresponding to the target layer, until the rendered image corresponding to each layer of the at least one layer is synthesized to obtain the target interactive screen.

8. The method according to any one of claims 1 to 7, characterized in that: In the case where the target application is a cloud application, the method further includes: The target interaction screen is pushed to a terminal corresponding to the target application, so that the terminal displays the target interaction screen.

9. An interactive screen rendering device, characterized in that: The device comprises: A rendering completion signal detection module is configured to execute, during the rendering process of a target interactive screen corresponding to a target application, detecting a rendering completion signal corresponding to each layer in at least one layer corresponding to the target interactive screen, wherein the rendering completion signal corresponding to each layer is generated when the rendering of the image corresponding to each layer is completed; The layer synthesis processing module is configured to start the layer synthesis thread when a rendering completion signal corresponding to the target layer is detected, so that the layer synthesis thread performs layer synthesis processing based on the rendered image corresponding to the target layer until the rendered image corresponding to each layer in the at least one layer is synthesized to obtain the target interactive screen; the target layer is any layer in the at least one layer for which a rendering completion signal is currently detected.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the interactive screen rendering method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the interactive screen rendering method as described in any one of claims 1 to 8.

Images