Image rendering method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image rendering method, device, electronic equipment and storage medium, the method comprising: in the case of displaying an image frame corresponding to a first scene, when it is detected that a second scene loading condition is met, a pipeline creation thread creates a rendering pipeline associated with the second scene; in the process of creating the rendering pipeline associated with the second scene by the pipeline creation thread, a main thread processes business data related to the first scene to obtain a first processing result; and an image is rendered based on the first processing result by a rendering thread using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene. The essence is to assign a new rendering pipeline creation task to the pipeline creation thread to complete, so that the main thread and the rendering thread can continue to complete the work of rendering the image frame, and the problem of rendering frame rate jitter caused by the main thread or the rendering thread being busy creating a new rendering pipeline can be avoided.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image rendering method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the traditional gaming industry, when a game character enters a new scene, there is usually a loading screen that takes a long time to display. This not only interrupts the user's game progress, but also significantly reduces the user's immersive gaming experience. Summary of the Invention

[0003] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides an image rendering method, apparatus, electronic device and storage medium.

[0004] In a first aspect, this disclosure provides an image rendering method, including:

[0005] When displaying image frames corresponding to the first scene, if the loading conditions for the second scene are detected, the pipeline creation task associated with the second scene is assigned to the pipeline creation thread by the main thread.

[0006] A rendering pipeline associated with the second scene is created using a pipeline creation thread;

[0007] During the process of creating a rendering pipeline associated with the second scene in the pipeline creation thread, the main thread processes business data related to the first scene to obtain a first processing result; the rendering thread performs image rendering based on the first processing result using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene; and the image frame corresponding to the first scene is displayed.

[0008] Secondly, this disclosure also provides an image rendering apparatus, comprising:

[0009] The task allocation module is used to allocate pipeline creation tasks associated with the second scene to the pipeline creation thread through the main thread when the second scene loading conditions are detected, while displaying image frames corresponding to the first scene.

[0010] The pipeline creation module is used to create a rendering pipeline associated with the second scene through the pipeline creation thread;

[0011] The synchronization control module is used to process business data related to the first scene through the main thread during the process of the pipeline creation thread creating a rendering pipeline associated with the second scene, and obtain a first processing result; through the rendering thread, based on the first processing result, to perform image rendering using the rendering pipeline associated with the first scene, and obtain an image frame corresponding to the first scene; and to display the image frame corresponding to the first scene.

[0012] Thirdly, this disclosure also provides an electronic device, the electronic device comprising:

[0013] One or more processors;

[0014] Storage device for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image rendering method as described above.

[0016] Fourthly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the image rendering method described above.

[0017] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0018] The technical solution provided in this disclosure, when displaying an image frame corresponding to a first scene, detects that the loading conditions for a second scene are met, and allocates a pipeline creation task associated with the second scene to the pipeline creation thread via the main thread; the pipeline creation thread creates a rendering pipeline associated with the second scene; during the creation of the rendering pipeline associated with the second scene, the main thread processes business data related to the first scene to obtain a first processing result; the rendering thread, based on the first processing result, performs image rendering using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene; and the image frame corresponding to the first scene is then displayed. Essentially, this adds a pipeline creation thread to the image rendering system. When a new rendering pipeline needs to be created, the new rendering pipeline creation task is assigned to the pipeline creation thread to complete it. This allows the main thread and the rendering thread to continue rendering the image frame corresponding to the first scene, avoiding the problem of frame rate stuttering caused by the main thread or rendering thread being busy creating new rendering pipelines and thus unable to complete the image rendering work normally. This also reduces the waiting time when actually switching scenes. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of an image rendering method provided in an embodiment of this disclosure;

[0022] Figure 2 A schematic diagram of an image rendering method provided in an embodiment of this disclosure;

[0023] Figure 3 This is a schematic diagram of the structure of an image rendering apparatus according to an embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0025] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0026] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0027] Before introducing the embodiments of this application, some terms or concepts involved in the embodiments of this application will be explained first. It should be understood that this application does not specifically limit the naming of the following terms. The following terms may have other names. The renamed terms still satisfy the following related terminology interpretations.

[0028] The rendering pipeline, also known as the rendering pipeline, pixel pipeline, or pixel pipeline, is a parallel processing unit within the GPU that processes signals from independent graphics units. The main purpose of the rendering pipeline is to generate or render a two-dimensional image given conditions such as a virtual camera, 3D objects, light sources, lighting modes, and textures.

[0029] As described in the background section, in the traditional gaming industry, when a game character enters a new scene, there is usually a lengthy loading screen, which not only interrupts the user's gameplay but also significantly reduces the user's immersive gaming experience. After thorough research, the applicant discovered that a major reason for this phenomenon is that when a game character enters a new scene, a new scene rendering pipeline needs to be created beforehand. Only after the new scene rendering pipeline is created can the image of the new scene be rendered and displayed. In existing technologies, the creation of the new scene rendering pipeline is often handled by the main thread. Because the process of creating the new scene rendering pipeline is time-consuming, the main thread and the rendering thread cannot cooperate to complete the normal image rendering work during the pipeline creation process, easily leading to frame rate stuttering. To avoid this stuttering, games usually display a loading screen to ensure that necessary initialization work is completed in the background, thus ensuring that the new scene can be smoothly presented to the user after loading. While this method is effective, it also results in a long wait, affecting the user's immersion and gaming experience.

[0030] Figure 1 A flowchart illustrating an image rendering method provided in this embodiment of the disclosure. Figure 2 This is a schematic diagram illustrating an image rendering method provided in an embodiment of this disclosure. This embodiment is applicable to situations where image rendering is performed on a client side. The method can be executed by an image rendering device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a terminal, specifically including but not limited to smartphones, PDAs, tablets, wearable devices with displays, desktop computers, laptops, all-in-one computers, smart home devices, and extended reality devices. Alternatively, this embodiment is applicable to situations where image rendering processing is performed on a server side. The method can be executed by an image rendering device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server. Alternatively, this embodiment is applicable to situations where the server and terminal cooperate in image rendering processing.

[0031] The extended reality devices described in this application may include, but are not limited to, the following types:

[0032] The computer-based augmented reality device uses a PC to perform calculations and output data related to augmented reality functions, while the external computer-based augmented reality device uses the data output from the PC to achieve the augmented reality effect.

[0033] Mobile extended reality devices support setting up mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connections with the mobile terminal, the mobile terminal performs calculations related to extended reality functions and outputs data to the mobile extended reality device, such as watching extended reality videos through an app on the mobile terminal.

[0034] like Figure 1 and Figure 2 As shown, the method may specifically include:

[0035] S110. When displaying an image frame corresponding to the first scene, if the loading conditions for the second scene are detected, the pipeline creation task associated with the second scene is assigned to the pipeline creation thread by the main thread.

[0036] A scene can be a specific environment formed by various scene elements. Scene elements refer to objects, characters, animals, plants, etc., used to create a specific scene. For example, in some embodiments, in games, a scene typically refers to a specific game screen environment, including the overall screen composed of scene elements such as terrain, buildings, and characters. For instance, an adventure game might include forest scenes and city scenes. A forest scene includes scene elements such as dense trees, grasslands, and streams. A city scene includes scene elements such as skyscrapers, streets, and pedestrians. The game character controlled by the user can switch between forest and city scenes to explore. In other embodiments, a scene can be understood as an application page. For example, when a document editing application is opened, the document editing interface is one scene, and the font formatting tools and text input area content in the document editing interface are all environmental elements. When a data statistics application is opened, the data statistics interface is another scene. The charts and data labels in the data statistics interface are all environmental elements.

[0037] Meeting the second scene loading conditions means that the user may or needs to enter the second scene, requiring the pre-creation of a rendering pipeline associated with the second scene. The rendering pipeline associated with the second scene could be, for example, a rendering pipeline used to render scene elements within the second scene. This application does not limit the specific nature of the second scene loading conditions. For example, in a game, if the game character controlled by the user is detected to have reached a certain location in the first scene, it is determined that the second scene loading conditions are met. Alternatively, if the user is detected to have selected (e.g., clicked) an option representing the second scene, it is determined that the second scene loading conditions are met. Or, if the user is detected to have completed the task necessary to enter the second scene, it is determined that the second scene loading conditions are met.

[0038] In this application, the image rendering process involves a main thread, a pipeline creation thread, and a rendering thread. The main thread is typically responsible for handling the main logic and business processes of the application, resulting in a processing output. For example, in a game, the main thread might handle user input and game state management. The rendering thread is primarily responsible for rendering based on the processing output and the rendering data of scene elements, using the rendering pipeline corresponding to the scene elements to obtain the image frame to be displayed. The pipeline creation thread is used to create the rendering pipeline.

[0039] S120. Create a rendering pipeline associated with the second scene through a pipeline creation thread.

[0040] S130. During the process of creating the rendering pipeline associated with the second scene in the pipeline creation thread, the main thread processes the business data related to the first scene to obtain the first processing result; the rendering thread performs image rendering based on the first processing result using the rendering pipeline associated with the first scene to obtain the image frame corresponding to the first scene; and displays the image frame corresponding to the first scene.

[0041] The above technical solution, when displaying image frames corresponding to the first scene and detecting that the loading conditions for the second scene are met, allocates a pipeline creation task associated with the second scene to the pipeline creation thread via the main thread. The pipeline creation thread then creates a rendering pipeline associated with the second scene. During the creation of the rendering pipeline associated with the second scene, the main thread processes business data related to the first scene to obtain a first processing result. Based on the first processing result, the rendering thread uses the rendering pipeline associated with the first scene to perform image rendering, obtaining the image frame corresponding to the first scene. Finally, the image frame corresponding to the first scene is displayed. Essentially, this adds a pipeline creation thread to the image rendering system. When a new rendering pipeline needs to be created, the new rendering pipeline creation task is assigned to the pipeline creation thread. This allows the main thread and the rendering thread to continue rendering the image frames corresponding to the first scene, avoiding the problem of the main thread or rendering thread being busy creating new rendering pipelines and unable to complete the rendering work normally, resulting in frame rate stuttering. This also reduces the waiting time when actually switching scenes.

[0042] Based on the above technical solution, optionally, after the rendering pipeline associated with the second scene is created, the main thread processes the business data related to the second scene to obtain a second processing result; the rendering thread then uses the rendering pipeline associated with the second scene to render the image based on the second processing result, obtaining the image frame corresponding to the second scene; and finally, the image frame corresponding to the second scene is displayed. Essentially, this setup transfers control of the rendering pipeline associated with the second scene to the rendering thread after its creation, ensuring that the image frame corresponding to the second scene can be rendered correctly. Simultaneously, the pipeline creation thread can wait for new pipeline creation tasks or perform new rendering pipeline creation work, ensuring a reduced probability of rendering frame rate stuttering during multiple consecutive scene transitions.

[0043] In practice, optionally, within a preset time period after the rendering pipeline associated with the second scene is created, the rendering thread simultaneously renders image frames corresponding to the first scene and image frames corresponding to the second scene; based on a preset smooth transition strategy, the display effects of the image frames corresponding to the first scene and the image frames corresponding to the second scene are adjusted to achieve a smoother and more natural scene transition. The smooth transition strategy can be, for example, a gradient transition or a blur transition. See, for an example... Figure 2 At frame N-1, if the loading conditions for the second scene are not met, only image frame R1 corresponding to the first scene needs to be rendered and displayed. If the loading conditions for the second scene are met during the time period corresponding to frame N, a rendering pipeline associated with the second scene is created through the pipeline creation thread starting from frame N. The rendering pipeline associated with the second scene is completed at frame N+2. During the creation of the rendering pipeline associated with the second scene (corresponding to frames N and N+1), image frame R1 corresponding to the first scene is rendered and displayed. At frames N+2 and N+3 (i.e., within a preset time after the rendering pipeline associated with the second scene is created), image frame R1 corresponding to the first scene and image frame R2 corresponding to the second scene are rendered and displayed simultaneously. Image frame R2 corresponding to the second scene is displayed at frame N+4. Specifically, at frames N+2 and N+3, image frame R1 corresponding to the first scene can be set to gradually become transparent from opaque, while image frame R2 corresponding to the second scene can gradually become opaque from transparent.

[0044] Furthermore, after the pipeline creation thread creates the rendering pipeline associated with the second scene, the method may further include: storing the executable file of the rendering pipeline associated with the second scene to a shared storage space through the pipeline creation thread; "using the rendering thread to perform image rendering based on the second processing result and utilizing the rendering pipeline associated with the second scene to obtain an image frame corresponding to the second scene" includes: using the rendering thread to read the executable file from the shared storage space, and using the rendering pipeline associated with the second scene to perform image rendering based on the second processing result and obtain an image frame corresponding to the second scene.

[0045] Shared storage space can be, for example, a storage space accessible to all threads in the rendering system. The creation of the rendering pipeline essentially involves compiling code files into executable files and storing them in shared storage space. This facilitates the smooth transfer of control of the created rendering pipeline from the pipeline creation thread to the rendering thread, thereby enabling the rendering thread to perform image rendering using the rendering pipeline associated with the second scene.

[0046] In practice, there can be multiple rendering pipelines associated with the second scene; different rendering pipelines associated with the second scene are used to render different scene elements in the second scene. For example, the second scene is a forest, and among the multiple rendering pipelines associated with the second scene, some rendering pipelines are used to render terrain, some rendering pipelines are used to render leaves, and some rendering pipelines are used to render water.

[0047] Furthermore, "by using the rendering thread and based on the second processing result, performing image rendering using the rendering pipeline associated with the second scene to obtain an image frame corresponding to the second scene" may include: obtaining rendering data of scene elements of the second scene to be rendered; determining the target rendering pipeline from the rendering pipeline associated with the second scene based on the rendering data of the scene elements to be rendered; and rendering the scene elements to be rendered using the target rendering pipeline by using the rendering thread and based on the second processing result.

[0048] Rendering data for scene elements is a collection of information used to describe and present the visual characteristics of scene elements. It includes data describing the appearance, shape, material, and other features of scene elements. Through processing by the rendering thread, this data can be transformed into a visual image displayed on the screen. For example, rendering data includes at least one of the following: object model data, material properties, and texture information. Model data determines the shape of the object, material properties determine the object's appearance characteristics, such as color and gloss, and texture information adds rich details to the object.

[0049] The target rendering pipeline could be, for example, a rendering pipeline that can be used to render scene elements of a second scene to be rendered.

[0050] By setting rendering data based on the scene elements to be rendered, the target rendering pipeline is determined from the rendering pipeline associated with the second scene. Through the rendering thread, based on the second processing result, the scene elements to be rendered are rendered using the target rendering pipeline. Essentially, when rendering the image frame corresponding to the second scene, the appropriate rendering pipeline is selected for rendering according to the scene elements that need to be displayed at the moment, so that the rendered scene elements are natural.

[0051] Based on the above technical solutions, the method may optionally include: acquiring rendering data of multiple different application windows, wherein the rendering data of the application windows includes image rendering data of the application windows and position information of the application windows on the screen; and performing unified rendering based on the rendering data of multiple different application windows through the same rendering thread to obtain the image frame to be displayed.

[0052] Image rendering data can be, for example, a set of information used to describe and present the visual features of things in an image. In other words, image rendering data can, for example, include a set of information consisting of rendering data of scene elements included in the image.

[0053] This setup essentially means that individual application threads are not responsible for handling their own image rendering tasks. Instead, the image rendering system's threads uniformly handle the image rendering tasks of multiple applications. Compared to traditional rendering, where each application's thread is responsible for its own image rendering task, creating multiple images, and then combining the images rendered by each application to obtain the final image frame to be displayed, this approach uses a single rendering thread to perform unified rendering based on rendering data from multiple different application windows to obtain the image frame to be displayed. This allows for more efficient use of hardware resources, avoids resource waste and coordination problems between different processing units (such as those specifically responsible for vertex or pixel processing), and improves rendering speed and rendering quality.

[0054] Furthermore, the cloud server obtains rendering data from multiple different application windows deployed on the terminal; the cloud server performs unified rendering based on the rendering data of multiple different application windows through the same rendering thread to obtain the image frame to be displayed; the cloud server sends the image frame to be displayed to the terminal so that the terminal can display the image frame. In essence, this setup means that the terminal is responsible for rendering the image, while the cloud server is responsible for rendering the image, which can effectively reduce the computing burden on the terminal device and improve rendering efficiency and quality.

[0055] Furthermore, the second scene is associated with multiple rendering pipelines; the creation of the rendering pipeline associated with the second scene through the pipeline creation thread also includes: determining the creation priority of the rendering pipeline associated with the second scene; and creating the rendering pipeline associated with the second scene sequentially through the pipeline creation thread based on the creation priority of the rendering pipeline.

[0056] The creation priority of the rendering pipeline indicates the order in which rendering pipelines are created. In practice, the creation priority of the rendering pipeline can be determined based on a variety of factors. For example, the creation priority of the rendering pipeline can be determined based on user operation information, can be preset, or can be determined based on the terminal's hardware parameters (such as the terminal's screen parameters).

[0057] By setting the creation priority based on the rendering pipeline, the rendering pipeline associated with the second scene is created sequentially. Essentially, the rendering pipeline is created according to its importance, which can ensure that the most important visual effects can be presented immediately and improve the user experience.

[0058] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0059] Figure 3 This is a schematic diagram of an image rendering apparatus according to an embodiment of this disclosure. The image rendering apparatus provided in this embodiment can be configured in a client or in a server. See also Figure 3 The image rendering apparatus specifically includes:

[0060] The task allocation module 310 is used to allocate pipeline creation tasks associated with the second scene to the pipeline creation thread through the main thread when the second scene loading conditions are detected, while displaying an image frame corresponding to the first scene.

[0061] Pipeline creation module 320 is used to create a rendering pipeline associated with the second scene through a pipeline creation thread;

[0062] The rendering module 330 is used to process business data related to the first scene through the main thread during the process of the pipeline creation thread creating a rendering pipeline associated with the second scene, and obtain a first processing result; and to perform image rendering using the rendering pipeline associated with the first scene based on the first processing result through the rendering thread, and obtain an image frame corresponding to the first scene; and to display the image frame corresponding to the first scene.

[0063] Furthermore, the rendering module is also used for:

[0064] After the rendering pipeline associated with the second scene is created, the main thread processes the business data related to the second scene to obtain the second processing result.

[0065] Through the rendering thread, based on the second processing result, image rendering is performed using the rendering pipeline associated with the second scene to obtain an image frame corresponding to the second scene;

[0066] Display the image frame corresponding to the second scene.

[0067] Furthermore, the pipeline creation module is used to store the executable file of the rendering pipeline associated with the second scene to the shared storage space after the pipeline creation thread creates the rendering pipeline associated with the second scene.

[0068] The step of rendering an image using the rendering pipeline associated with the second scene based on the second processing result through the rendering thread to obtain an image frame corresponding to the second scene includes:

[0069] The executable file is read from the shared storage space through the rendering thread, and image rendering is performed using the rendering pipeline associated with the second scene based on the second processing result to obtain an image frame corresponding to the second scene.

[0070] Furthermore, the number of rendering pipelines associated with the second scene is multiple; different rendering pipelines associated with the second scene are used to render different scene elements in the second scene; the rendering module is used for:

[0071] Obtain the rendering data of the scene elements of the second scene to be rendered;

[0072] Based on the rendering data of the scene elements to be rendered, the target rendering pipeline is determined from the rendering pipeline associated with the second scene.

[0073] Based on the second processing result, the scene elements to be rendered are rendered using the target rendering pipeline through the rendering thread.

[0074] Furthermore, the rendering module is used for:

[0075] Obtain rendering data for multiple different application windows, wherein the rendering data of the application windows includes the image rendering data of the application windows and the position information of the application windows on the screen;

[0076] Using the same rendering thread, based on the rendering data of the multiple different application windows, a unified rendering is performed to obtain the image frame to be displayed.

[0077] Furthermore, the soft membrane module is used for:

[0078] Control the cloud server to obtain rendering data of multiple different application windows deployed on the terminal;

[0079] The cloud server is controlled to perform unified rendering based on the rendering data of the multiple different application windows through the same rendering thread to obtain the image frame to be displayed;

[0080] The cloud server is controlled to send the image frame to be displayed to the terminal, so that the terminal can display the image frame to be displayed.

[0081] Furthermore, the second scene is associated with multiple rendering pipelines; the pipeline creation module is used for:

[0082] Determine the creation priority of the rendering pipeline associated with the second scene;

[0083] By creating threads through the pipeline, rendering pipelines associated with the second scene are created sequentially based on the creation priority of the rendering pipeline.

[0084] The image rendering apparatus provided in this disclosure can execute the steps performed by the client or server in the image rendering method provided in this disclosure, and has the execution steps and beneficial effects, which will not be described in detail here.

[0085] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 4 The diagram illustrates a structural schematic suitable for implementing the electronic device 1000 in the embodiments of this disclosure. The electronic device 1000 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0086] like Figure 4 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003 to implement the image rendering method as described in the embodiments of this disclosure. The RAM 1003 also stores various programs and information required for the operation of the electronic device 1000. The processing device 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0087] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to exchange information with other devices wirelessly or via wired communication. Although Figure 4 An electronic device 1000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0088] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the image rendering method as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.

[0089] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include information signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated information signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0090] In some implementations, clients and servers may communicate using any known or future network protocol, such as HTTP (Hypertext Transfer Protocol), and may interconnect with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or future network.

[0091] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0092] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0093] When displaying image frames corresponding to the first scene, if the loading conditions for the second scene are detected, the pipeline creation task associated with the second scene is assigned to the pipeline creation thread by the main thread.

[0094] A rendering pipeline associated with the second scene is created using a pipeline creation thread;

[0095] During the process of creating a rendering pipeline associated with the second scene in the pipeline creation thread, the main thread processes business data related to the first scene to obtain a first processing result; the rendering thread performs image rendering based on the first processing result using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene; and the image frame corresponding to the first scene is displayed.

[0096] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0097] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0099] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0100] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0101] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0103] One or more processors;

[0104] Memory, used to store one or more programs;

[0105] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the image rendering methods provided in this disclosure.

[0106] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements an image rendering method as described in any of the present disclosure.

[0107] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the image rendering method described above.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image rendering method, characterized in that, include: When displaying image frames corresponding to the first scene, if the loading conditions for the second scene are detected, the pipeline creation task associated with the second scene is assigned to the pipeline creation thread by the main thread. A rendering pipeline associated with the second scene is created using a pipeline creation thread; During the process of the pipeline creation thread creating the rendering pipeline associated with the second scene, the main thread processes the business data related to the first scene to obtain the first processing result. Based on the first processing result, the rendering thread performs image rendering using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene. Display the image frame corresponding to the first scene.

2. The method according to claim 1, characterized in that, Also includes: After the rendering pipeline associated with the second scene is created, the main thread processes the business data related to the second scene to obtain the second processing result. Through the rendering thread, based on the second processing result, image rendering is performed using the rendering pipeline associated with the second scene to obtain an image frame corresponding to the second scene; Display the image frame corresponding to the second scene.

3. The method according to claim 2, characterized in that, After the pipeline creation thread creates the rendering pipeline associated with the second scene, it also includes: The pipeline creation thread stores the executable file of the rendering pipeline associated with the second scene to the shared storage space; The step of rendering an image using the rendering pipeline associated with the second scene based on the second processing result through the rendering thread to obtain an image frame corresponding to the second scene includes: The executable file is read from the shared storage space through the rendering thread, and image rendering is performed using the rendering pipeline associated with the second scene based on the second processing result to obtain an image frame corresponding to the second scene.

4. The method according to claim 2, characterized in that, The number of rendering pipelines associated with the second scene is multiple; different rendering pipelines associated with the second scene are used to render different scene elements in the second scene; The step of rendering an image using the rendering pipeline associated with the second scene based on the second processing result through the rendering thread to obtain an image frame corresponding to the second scene includes: Obtain the rendering data of the scene elements of the second scene to be rendered; Based on the rendering data of the scene elements to be rendered, the target rendering pipeline is determined from the rendering pipeline associated with the second scene. Based on the second processing result, the scene elements to be rendered are rendered using the target rendering pipeline through the rendering thread.

5. The method according to claim 1, characterized in that, Also includes: Obtain rendering data for multiple different application windows, wherein the rendering data of the application windows includes the image rendering data of the application windows and the position information of the application windows on the screen; Using the same rendering thread, a unified rendering is performed based on the rendering data of the multiple different application windows to obtain the image frame to be displayed.

6. The method according to claim 5, characterized in that, The cloud server retrieves rendering data from multiple different application windows deployed on the terminal. The cloud server uses the same rendering thread to perform unified rendering based on the rendering data of the multiple different application windows to obtain the image frame to be displayed. The cloud server sends the image frame to be displayed to the terminal so that the terminal can display the image frame.

7. The method according to claim 1, characterized in that, The second scene is associated with multiple rendering pipelines; the step of creating the rendering pipeline associated with the second scene through a pipeline creation thread further includes: Determine the creation priority of the rendering pipeline associated with the second scene; By creating threads through the pipeline, rendering pipelines associated with the second scene are created sequentially based on the creation priority of the rendering pipeline.

8. An image rendering apparatus, characterized in that, include: The task allocation module is used to allocate pipeline creation tasks associated with the second scene to the pipeline creation thread through the main thread when the second scene loading conditions are detected, while displaying image frames corresponding to the first scene. The pipeline creation module is used to create a rendering pipeline associated with the second scene through the pipeline creation thread; The rendering module is used to process business data related to the first scene through the main thread during the process of the pipeline creation thread creating a rendering pipeline associated with the second scene, and obtain a first processing result. Based on the first processing result, the rendering thread performs image rendering using the rendering pipeline associated with the first scene to obtain an image frame corresponding to the first scene. Display the image frame corresponding to the first scene.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images