Asset Recognition Computing Architecture for Graphics Processing

Abstract

To provide a method for executing a game by a computing system that uses a central processing unit (CPU) and graphics processing unit (GPU) for generating video frames.SOLUTION: A draw call is generated for a video frame by a CPU. At writing of GPU commands by the CPU using a GPU API, asset aware data (AAD) is written to a command buffer, and loading of one or more level of detail (LOD) data from an asset store to system memory is requested. The GPU executes the draw call for the frame using LOD data written to the system memory, the GPU using at least a minimum of LOD data based on the AAD. Additionally, the GPU uses information regarding the LOD load state when executing the draw call, in order to avoid access to LODs not yet loaded.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to computing system architectures, and more particularly, to a computing system architecture having an asset recognition component including a central processing unit and a graphics processing unit that recognize assets when generating video frames for games or other interactive applications.

Background Art

[0002] Video games provide gameplay in a virtual game world. By executing the corresponding game, various scenes in the game world can be created. The scenes can be generated using various types of assets such as textures and models. These assets are used by the graphics processor to generate the scenes. Since the assets are too large to fit into the system memory all at once, the assets are usually stored on optical media, hard drives, or solid-state devices and loaded into the system memory as needed. As a result, the user may experience long delays when loading assets, and managing the loading of assets into the system memory and freeing up the system memory to receive additional assets can be a significant burden for game developers. Thus, game developers can reduce the complexity of the scenes to reduce user latency and required asset management, but as a result, the immersive experience obtained by the player also decreases.

[0003] Embodiments of the present disclosure have been made in such a background.

Summary of the Invention

[0004] Embodiments of the present disclosure relate to a computing system architecture including a central processing unit and a graphics processing unit configured to perform asset recognition when processing textures, models, or other assets via a graphics pipeline, and a method for implementing the same. Asset recognition provides, in the graphics pipeline, identification of assets required for processing and tracking of loading of assets to be used. Also, asset recognition enables knowing when loading of minimal assets (e.g., lower LOD) for processing via the graphics pipeline is completed, thereby reducing the impact of any stalls in the graphics pipeline, especially when transitioning between scenes in a virtualized game world.

[0005] Embodiments of the present disclosure disclose an asset recognition computing architecture configured to manage loading of assets to system memory when the graphics pipeline requires an asset and manage release of system memory when the asset is no longer needed. The computing architecture includes an asset store composed of memory devices providing high-speed access, thereby enabling loading of assets for a scene rendered in a video frame to system memory in a few milliseconds or tens of milliseconds. Specifically, using a bind-time trigger, an asset is loaded from the asset store to system memory. In this way, loading of assets for one or more draw calls used for rendering a video frame is triggered when an asset is bound to the corresponding draw call (e.g., at bind time) and when a central processing unit (CPU) executing the game constructs one or more command buffers for draw calls to be executed by a graphics processing unit (GPU) implementing the graphics pipeline using a graphics application programming interface (API). To recognize assets, the computing architecture enables the GPU to be executable using the minimum assets required for one or more draw calls of a video frame without waiting for the required assets to be fully loaded. The asset store controller is configured to control the management of assets in the asset store and the distribution of assets across the entire asset recognition computing architecture. In one embodiment, the asset store controller can be software that runs on the CPU with assistance from other on-chip units. Alternatively, the asset store controller can be an on-chip complex that includes a coprocessor such as an additional CPU. Both the CPU and the GPU can be modified for integration and / or interface with the asset store controller. The asset store includes game textures, models, and other data. The asset store is much larger than system memory. The asset store includes the content of the asset store with assets identified and / or tagged using asset identifiers (asset IDs), as well as recognizes the content and / or details of the assets (e.g., the level of detail of the corresponding assets). The asset store is updated in various ways when an asset is recognized, such as when the asset is locally generated, or when the asset is accessed from mass storage within a rack, or when the asset is accessed from within a data center, or when the asset is delivered over a network. Conventionally, games running on the CPU use the GPU API to bind one or more assets (e.g., textures, level-of-detail - LOD, models, etc.) to draw calls, enabling the assets to be used for the objects that the draw calls render. In an asset-aware computing architecture, the GPU API receives additional information from a game running on the CPU. For example, for each draw call, additional Asset Aware Data (AAD) includes the asset ID of the asset used in the draw call, the minimum LOD at which the asset should be loaded before the draw call can be executed in the graphics pipeline, and the priority of the asset's load compared to the load of other assets used in the draw call. The GPU API is configured to place at least a part of the AAD (e.g., asset ID, minimum LOD, etc.) in the command buffer of the corresponding draw call and pass at least a part, but not all, of the AAD to the asset store controller. Further, the GPU API can also be used to notify the asset store controller when each video frame of rendering is completed, for example, by placing a command in the command buffer. Thus, in an asset-aware architecture, the asset store controller receives an AAD stream such as at least all of the assets required for a video frame, along with prioritization information, and a notification of the end of rendering using these assets, etc. When the asset store controller receives the AAD, it starts loading the asset from the asset store into the system memory. The loading process uses the load priority. For example, the "minimum LOD" of all textures and / or models and / or other assets can be loaded, and then, based on the asset load priority information, higher LODs can be loaded. When the load is complete, the GPU is notified (e.g., using the asset ID) that the asset load is complete or that the asset is being loaded and to what extent it is loaded (e.g., which LOD of the asset has been loaded). In some embodiments, there may be an "urgent load" of an asset if rendering (e.g., execution of the corresponding draw call in the graphics pipeline) is started and the load of the asset required for that draw call has not yet been completed. In other embodiments, conditional commands within a command buffer can be used to query the load status of LOD data and change the command flow. For example, if the load of at least the minimum LOD data is not complete, draw calls can be completely skipped (i.e., GPU stalls are avoided), or rendering can be performed using a proxy asset, or customized rendering based on the loaded LOD becomes possible. In yet other embodiments, the GPU can use AAD to make these same decisions. For example, if the load of at least the minimum LOD data is not complete, draw calls can be completely skipped (i.e., GPU stalls are avoided). The asset store controller is also configured to free up unnecessary memory, such as when an asset is not used during the rendering of a frame, so the memory space storing that asset can be freed. The GPU can be modified to recognize assets. For example, when an object is rendered, the GPU recognizes whether the load of the required LOD of the asset is complete. The GPU can issue an "emergency load" request for the required LOD and stall until the required LOD is loaded. Further, the GPU recognizes the highest LOD loaded into memory for a particular asset and uses up to that highest LOD, but does not use LODs beyond that highest LOD to render the corresponding video frame. As part of the loading and freeing of system memory, operations on the page table, CPU cache, and GPU cache may be required, and thus, the asset store controller, CPU, and GPU can be configured to accelerate these operations. Thus, from the perspective of a game running on the CPU, the assets required for the corresponding draw calls "magically" appear in system memory when they are used. That is, the game only needs to make draw calls using the GPU API, and the required assets (e.g., minimum LOD) are ready to be used in the graphics pipeline.

[0006] In one embodiment, a method for executing a game is disclosed by a computing system that uses a central processing unit and a graphics processing unit to generate video frames. The method includes generating, by the CPU, a draw call for one of the video frames of the video frames. The method includes writing, at bind time, one or more commands of the draw call to a command buffer. The method includes writing, at bind time, the asset recognition data (AAD) of the draw call to the command buffer using the GPU API and starting, at bind time, the loading of one or more levels of detail (LOD) data from the asset store to the system memory used by the computing system. The method includes executing, by the GPU, the draw call of the frame using the LOD data written to the system memory, and the GPU uses at least a minimum amount of LOD data based on the AAD.

[0007] In other embodiments, a computing system for executing a game to generate video frames is disclosed. The computing system includes a central processing unit configured to execute the game. The CPU generates a draw call for one of the frames of the video frames, and the draw call includes commands. The computing system includes a command buffer configured to store the asset recognition data and the commands of the draw call, and the AAD and the commands are written to the command buffer by the CPU using the GPU API at bind time. The computing system includes an asset store configured to store a plurality of assets of the application, which includes one or more levels of detail of the assets used by the draw call. The computing system includes a system memory configured to store one or more levels of detail data used by draw calls, the loading of LOD data from the asset store is triggered by the CPU at bind time, and the loading of the LOD data into the system memory is initiated while writing AAD to the command buffer. The computing system includes a graphics processing unit configured to execute commands for draw calls of a frame using the LOD data written to the system memory, and the GPU uses at least a minimum amount of LOD data based on the AAD.

[0008] In yet other embodiments, a non-transitory computer-readable medium storing a computer program for executing a game by a computing system that uses a central processing unit and a graphics processing unit to generate video frames is disclosed. The non-transitory computer-readable medium includes program instructions for generating, by the CPU, a draw call for one of the video frames of the video frame. The non-transitory computer-readable medium includes program instructions for writing, at bind time, one or more commands of a draw call to a command buffer. The non-transitory computer-readable medium includes program instructions for writing, at bind time, asset recognition data (AAD) of a draw call to the command buffer using the GPU API, and for initiating, at bind time, the loading of one or more levels of detail (LOD) data from an asset store into the system memory used by the computing system. The non-transitory computer-readable medium includes program instructions for executing, by the GPU, a draw call of a frame using the LOD data written to the system memory, and the GPU uses at least a minimum amount of LOD data based on the AAD.

[0009] Other aspects of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate embodiments of the principles of the present disclosure.

[0010] This disclosure will be best understood by reference to the following description, taken in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0011]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 7

Embodiments for Carrying Out the Invention

[0012] The following embodiments for carrying out the invention include a number of specific details for illustrative purposes, but those skilled in the art will understand that many variations and modifications to the following details are included within the scope of the present disclosure. Accordingly, the aspects of the present disclosure described below are set forth without detracting from the generality of the claims following the embodiments for carrying out the invention and without imposing limitations on the claims.

[0013] Generally, various embodiments of the present disclosure describe a computing system architecture including asset recognition components such as a central processing unit and a graphics processing unit that generate video frames via a graphics pipeline, and a method for implementing the same. Asset recognition by the CPU and GPU provides identification and tracking of assets used in the graphics pipeline, thereby enabling the use of a minimum number of assets (e.g., minimum level of detail - LOD) in the processing via the graphics pipeline when generating corresponding video frames. Specifically, game code executed on the CPU can predict when an asset is needed in order to timely load the required assets into system memory. For example, an asset can be a texture and / or a model that is processed through a shader program of a graphics pipeline to generate a video frame that is displayed through the pixels of a display. For example, in a forest scene, a texture of a tree can be received as input and processed through a graphics pipeline. If there is not enough time to load a large amount of LOD data of the texture and / or the model into the system memory for processing, to avoid stalling of the GPU, the GPU embodiments of the present disclosure provide asset recognition, whereby the GPU is configured to track the loading of the identified asset and generate a scene using at least a minimal LOD of the asset. Initially, the scene can be rendered at a low resolution, but in subsequent frames, the scene can be rendered using more complete LOD data. In this way, a large and complex scene of a virtualized game world can be efficiently generated through a graphics pipeline, and at the same time, the possibility of waiting for the loading of assets and stalling the GPU can be reduced. This is particularly useful when the game encounters a scene cut within the game (e.g., a scene change in a cinematic flow, or the start of interactive gameplay after a series of menus) where the game transitions between two video frames from one scene to another. Instead of waiting for the entire LOD data to be loaded and a new scene to be generated, minimal LOD data can be loaded for each asset required for processing through the graphics pipeline, and the corresponding video frame can be generated. For example, by reducing the stalling of the GPU even while rendering a complex scene, the immersive experience of the user playing the game is significantly improved.

[0014] With the above general understanding of the various embodiments, exemplary details of the embodiments are now described with reference to the various drawings.

[0015] Throughout this specification, references to "game" or "video game" or "game application" are meant to represent any kind of interactive application that is directed through the execution of input commands. By way of mere example, interactive applications include applications such as games, word processing, video processing, video game processing, and the like. Further, the terms introduced above are used in the same meaning.

[0016] FIG. 1A is a diagram of a system 100A for providing a game according to an embodiment of the present disclosure. As shown, the game is executed locally on a client device 110 (e.g., a game console) of a corresponding user playing the game. Specifically, an instance of the game is executed by a game title processing engine 111. The client device 110 may operate in a single-player mode for the user. Game logic 115 (e.g., executable code) for implementing the game is stored in the client device 110 and is used to execute the game. By way of example, the game logic 115 can be delivered to the client device 110 via portable media (e.g., optical media) or via a network (e.g., downloaded from a game provider via the Internet).

[0017] In one embodiment, the game title processing engine 111 of the client device 110 includes basic processor-based functions for executing games and services associated with the game application. For example, the processor-based functions include 2D or 3D rendering, physics, physical simulation, scripting, audio, animation, graphic processing, lighting, shading, rasterization, ray tracing, shadowing, culling, transformation, artificial intelligence, and the like. Furthermore, the services of the game application include memory management, multi-thread management, quality of service (QoS), bandwidth testing, social networking, social network friend management, communication with friends' social networks, communication channels, text transmission, instant messaging, chat support, and so on. Specifically, the client device 110 is configured as a computing system architecture including asset recognition components such as a central processing unit and a graphics processing unit that generate video frames via a graphics pipeline. In this way, when generating the corresponding video frame, the computing system architecture is configured to identify, track, and use at least the minimum LOD data of the assets used in the graphics pipeline (for example, the execution of drawing calls).

[0018] The client device 110 can receive inputs from various types of input devices such as game controllers, tablet computers, keyboards, gestures captured by video cameras, mice, touch pads, etc. The client device 110 can be any type of computing device having at least a memory and a processor module, and is configured to generate a rendered image executed by the game title processing engine 111 and display the rendered image on a display (for example, the display 11, or a display 11 including a head-mounted display - HMD, etc.). For example, the rendered image can be associated with an instance of a game executed locally on the client device 110 to implement the corresponding user's game play, such as via input commands used to drive the game play. Some examples of the client device 110 include personal computers (PCs), game consoles, home theater devices, general-purpose computers, mobile computing devices, tablets, phones, or any other type of computing device capable of executing an instance of a game.

[0019] FIG. 1B is a diagram of a system 100B for providing a game having a backend cloud game network that provides support services, according to one embodiment of the present disclosure. As shown, the game is executed locally on a client device 110 (e.g., a game console) of a corresponding user playing the game, and the client device 110 has already been introduced in FIG. 1A. Specifically, an instance of the game is executed by the game title processing engine 111 as described above. That is, the game title processing engine 111 of the client device 110 includes basic processor-based functions for executing games and services related to the game application. Game logic 115 (e.g., executable code) for implementing the game is stored in the client device 110 and is used to execute the game. As described above, the client device 110 includes asset recognition components such as a central processing unit and a graphics processing unit that generate video frames via a graphics pipeline. Thus, a computing system architecture including a CPU and a GPU is configured to identify, track, and use at least a minimum amount of LOD data of assets used in the graphics pipeline (e.g., execution of drawing calls) when generating corresponding video frames.

[0020] The game server 160 may be configured to support one or more local computing devices corresponding to a plurality of users, and each local computing device can execute an instance of a corresponding game, such as in a single-player mode or a multiplayer mode. For example, in the multiplayer mode, while the game is being executed locally, the cloud game network 190 simultaneously receives information (e.g., game state data) from each local computing device via the network 150, and appropriately distributes the information across one or more of the local computing devices, whereby each user can interact with other users (e.g., through corresponding characters in the game) in the virtual game world of the multiplayer game. In this way, the cloud game network 190 coordinates and merges the gameplay of each user within the multiplayer game environment by passing information via the network 150. The network 150 may include one or more communication technologies. In some embodiments, the network 150 may include fifth generation (5G) network technology having an advanced wireless communication system.

[0021] Also, in either the single-player mode or the multiplayer mode, the cloud game network can provide additional services to the client device 110. For example, the cloud game network 190 can store the game state of one or more points in the user's gameplay for purposes such as backup of gameplay, providing highlights of gameplay, generating mini-games based on recorded gameplay, and providing information based on artificial intelligence (AI) related to gameplay (e.g., support for gameplay).

[0022] FIG. 1C is a diagram of a system 100C for providing a game via the cloud game network 190 according to an embodiment of the present disclosure, where the game is executed remotely from the client device 110' (e.g., a sync client) of the corresponding user playing the game. The system 100C can provide game control to one or more players playing one or more games via the cloud game network 190 via the network 150 in either the single-player mode or the multiplayer mode. In some embodiments, the cloud gaming network can be a cloud gaming network 190 that includes a plurality of virtual machines (VMs) running on a hypervisor of a host machine, and one or more of the virtual machines are configured to execute a game processor module utilizing the hardware resources available to the host's hypervisor. Network 150 can include one or more communication technologies. In some embodiments, network 150 can include 5th generation (5G) network technology with an advanced wireless communication system.

[0023] As shown, cloud gaming network 190 includes a game server 160 that provides access to a plurality of games. Game server 160 can be any type of server computing device available within the cloud and can be configured as one or more virtual machines running on one or more hosts. For example, game server 160 can manage a virtual machine that supports a game processor that instantiates an instance of a user's game. Thus, the multiple game processors of game server 160 associated with multiple virtual machines are configured to execute multiple instances of one or more games associated with multiple users' game play. In this way, backend server support provides streaming of game play media (such as video, audio, etc.) of multiple game applications to multiple corresponding users. That is, game server 160 is configured to stream and return data (such as a rendering image and / or frame of the corresponding game play) to the corresponding client device 110' via network 150. In this way, computationally complex game applications can be executed on the backend server in response to controller inputs received and forwarded by client device 110'. Each server can render images and / or frames, which are then encoded (e.g., compressed), streamed to the corresponding client device, and displayed.

[0024] For example, multiple users can access the cloud game network 190 via the network 150 using corresponding client devices 110'. The client device 110' can be configured similarly to the client device 110 in FIGS. 1A-1B (including, for example, the game title processing engine 111), or can be configured as a thin client that provides an interface to a backend server configured to provide computing capabilities (including, for example, the game title processing engine 111').

[0025] For ease of illustration, in FIG. 1C, a single client device 110' that supports a corresponding user is shown. Specifically, the client device 110' of a corresponding user (not shown) is configured to request access to a game via a network 150 such as the Internet, and to render an instance of the game that is executed by the game server 160 and delivered to a display device associated with the corresponding user. For example, a user can interact with an instance of a game executed on the game processor of the game server 160 via the client device 110'. More specifically, the instance of the game is executed by the game title processing engine 111'. The corresponding game logic (e.g., executable code) 115' that implements the game application is stored and accessible via a data store (not shown) and is used to execute the game. The game title processing engine 111' can support multiple games using multiple game logics, and each of the games can be selectable by the user.

[0026] For example, the client device 110’ is configured to interact with the game title processing engine 111’ in relation to the game play of the corresponding user, such as via input commands used to drive the game play. Specifically, the client device 110’ can receive inputs from various types of input devices such as game controllers, tablet computers, keyboards, gestures captured by video cameras, mice, touch pads, etc. The client device 110’ can be any type of computing device having at least a memory and a processor module, and can be connected to the game server 160 via the network 150. The backend game title processing engine 111’ is configured to generate a rendered image, and the generated rendered image is distributed via the network 150 and displayed on the corresponding display associated with the client device 110’. For example, via a cloud-based service, the rendered image can be distributed by an instance of the corresponding game executed on the game title processing engine 111’ of the game server 160. That is, the client device 110’ is configured to receive the rendered image and display the rendered image on the display 11. In one embodiment, the display 11 includes an HMD (e.g., for displaying VR content). In some embodiments, the rendered image can be streamed wirelessly or wired to a smartphone or tablet directly from the cloud-based service or via the client device 110’ (e.g., PlayStation (registered trademark) Remote Play).

[0027] In one embodiment, the game server 160 and / or the game title processing engine 111' includes basic processor-based functions for executing games and services associated with the game application. For example, the processor-based functions include 2D or 3D rendering, physics, physical simulation, scripting, audio, animation, graphic processing, lighting, shading, rasterization, ray tracing, shadowing, culling, transformation, artificial intelligence, and the like. Further, the services of the game application include memory management, multi-thread management, quality of service (QoS), bandwidth testing, social networking, social network friend management, communication with friends' social networks, communication channels, text transmission, instant messaging, chat support, and the like. Specifically, the game server 160 and / or the game title processing engine 111' is configured as a computing system architecture that includes asset recognition components such as a central processing unit and a graphic processing unit that generate video frames via a graphics pipeline. In this way, when generating the corresponding video frame, the computing system architecture is configured to identify, track, and use at least the minimum LOD data of the assets used in the graphics pipeline (e.g., execution of drawing calls).

[0028] In one embodiment, the cloud game network 190 is a distributed game server system and / or architecture. Specifically, a distributed game engine that executes game logic is configured as a corresponding instance of a corresponding game. Generally, the distributed game engine is responsible for each of the functions of the game engine and distributes these functions to be executed by a plurality of processing entities. Each individual function can be further distributed across one or more processing entities. The processing entities can be configured in various configurations including physical hardware and / or as virtual components or virtual machines and / or as virtual containers, where a container is different from a virtual machine that virtualizes an instance of a game application running on a virtualized operating system. The processing entities can utilize and / or rely on servers (computing nodes) and their underlying hardware on one or more servers of the cloud game network 190, and the servers can be arranged in one or more racks. The coordination, allocation, and management of the execution of these functions for various processing entities are performed by a distributed synchronization layer. In this way, the execution of these functions is controlled by the distributed synchronization layer, and in response to controller input by the player, generation of media (such as video frames, audio, etc.) of the game application becomes possible. The distributed synchronization layer can efficiently execute these functions across the distributed processing entities (e.g., through load distribution), whereby important game engine components / functions are distributed and then recombined for more efficient processing.

[0029] The game title processing engines 111 and 111' of FIGS. 1A-1C include a CPU and a GPU group configured to execute a multi-tenancy GPU function. In one embodiment, one CPU and GPU group can implement the graphics and / or rendering pipelines of multiple games. That is, the CPU and GPU group are shared among the multiple games being executed. The CPU and GPU group can be configured as one or more processing devices. In other embodiments, multiple GPU devices are combined to execute the graphics processing of a single application running on the corresponding CPU.

[0030] FIG. 2 shows a computing architecture 200 including asset recognition components such as a central processing unit 702 and a graphics processing unit 716 that recognize assets when generating video frames while the CPU is executing a game, according to one embodiment of the present disclosure. Specifically, the video frames are generated using a graphics pipeline implemented by the GPU 716 and controlled by the CPU 702. Asset recognition by the CPU 702 and the GPU 716 provides identification and tracking of the assets used in the graphics pipeline, thereby allowing the use of minimal assets (e.g., minimal level of detail - LOD) in the processing through the graphics pipeline when generating the corresponding video frames. The computing architecture 200 can be implemented in the client devices 110 and / or the cloud game network 190 (e.g., the game server 160 and / or the game title processing engines 111 and 111') of FIGS. 1A-1C.

[0031] Computing architecture 200 includes a CPU 702 configured to execute games. Specifically, the CPU 702 generates a draw call for rendering one of the video frames, and the draw call includes instructions. Further, while the draw call is being executed in the graphics pipeline, assets may be requested by the instructions of the draw call. For a particular video frame, there can be multiple draw calls generated by the CPU 702 and executed by the GPU 716. Due to the improved processing power, the game is designed to render even more complex scenes during gameplay. As a result, the game pushes increasingly more draw calls per video frame to generate the scenes (such as a forest scene, a sea scene, etc.) used during the gameplay of the game. The game code executed on the CPU 702 predicts when an asset is needed (such as when needed by one or more draw calls of a video frame), and adjusts the loading of the asset into the system memory 220 before it is actually used so that the asset is available at an appropriate time for the GPU 716 to render the scene via the graphics pipeline.

[0032] The GPU API 250 can be used to communicate between components of the computing architecture 200 or between applications executed on components of the computing architecture 200. For example, the GPU API 250 can be executed on the CPU 702 or called by the CPU 702 to communicate with another component such as a graphics processing unit, system memory, command buffer, asset store, etc. The GPU API 250 can be executed on the asset store controller 230 or on a processor within the GPU 716 to communicate with other components, and in some embodiments, the GPU API 250 can be partially implemented as fixed function hardware (instead of a general-purpose CPU) for this communication purpose.

[0033] For asset recognition, the game executed on CPU 702 generates additional information regarding the assets used in each draw call. For example, this Asset Recognition Data (AAD) may include the identifier of the corresponding asset (e.g., asset ID), the minimum LOD that should be loaded into system memory before the corresponding draw call can be executed in the graphics pipeline, and the priority of the load of the said asset compared to the load of other assets used in other draw calls or within this draw call. At least a part of the AAD of the draw call (e.g., asset ID) is delivered from the game executed on CPU 702 to the corresponding command buffer holding the commands of the draw call using GPU API 250. Further, at least a part (if not all) of the AAD of the draw call can be delivered from the game executed on CPU 702 to asset store controller 230 using GPU API 250 to adjust the load of the assets used in the graphics pipeline for the draw call. In some embodiments, all of the AAD is delivered to asset store controller 230.

[0034] Computing architecture 200 includes GPU 716 configured to implement a graphics pipeline. For example, the graphics pipeline may be implemented by GPU 716 to execute a shader program on the vertices of the objects in a scene to generate texture values for the pixels of the display, and operations may be executed in parallel via GPU 716 to increase efficiency. Generally, the graphics pipeline receives input geometry (e.g., the vertices of the objects in the game world). The vertex shader constructs the polygons or primitives that make up the objects in the scene. A vertex shader or other shader program can perform lighting, shading, shadowing, and other operations on polygons. A vertex shader or other shader program can perform depth or Z-buffering to identify objects visible from a corresponding viewpoint within the scene being rendered. Rasterization is performed to project objects within the three-dimensional world onto a two-dimensional plane defined by the viewpoint. For an object, pixel-sized fragments are generated, and one or more fragments can contribute to the color of the corresponding pixel when the image is displayed. The fragments are merged and / or blended to identify the combined color of each pixel, which is stored in the frame buffer and displayed.

[0035] Furthermore, controlled by the CPU 702 that executes the game, the game can use various types of assets for graphic processing. These assets can be used by the GPU 716 in the graphics pipeline to generate scenes within the game. For example, in a forest scene, a texture of a tree and a model of a tree can be received as input and processed through the graphics pipeline. As the game evolves, the virtualized game world created through the execution of the corresponding game becomes more complex, and it may become necessary to execute the operations of the shader program for an increasing number of vertices and / or textures of the objects or models within the scene.

[0036] More specifically, the GPU 716 of the computing architecture 200 is configured to execute draw call commands to generate video frames using LOD data. More specifically, the GPU 716 uses at least minimal LOD data based on the asset recognition data notified to the components of the asset recognition computing architecture 200. As described above, a video frame may require the execution of one or more draw calls, and each draw call may require one or more assets (such as textures, models, etc.). These assets can only be used by the GPU 716 when they are loaded into the system memory 220. However, these assets are too large to fit into the system memory 220 all at once, so they are stored outside the system memory 220, for example, on a hard drive or a solid state device (SSD). While the game is being executed by the CPU 702, the game code running on the CPU 702 predicts when an asset is needed (e.g., when needed by one or more draw calls of a video frame), and uses the GPU API to load the asset into the system memory 220 at an appropriate timing such that the asset is available for the GPU 716 to render the scene through the graphics pipeline before it is actually used. As the scene becomes more complex and larger texture data is required, the time to fully load the necessary assets can become longer. Thus, the graphics pipeline may attempt to use an asset before its load is complete.

[0037] In an embodiment of the present disclosure, the asset-aware computing architecture 200 can make an asset available for processing in the graphics pipeline without stalling the GPU 716 while waiting for the asset to be fully loaded into the system memory 220. Stalling can occur, especially in a conventional computing architecture, when there are too many inputs (such as textures) provided for rendering a scene. For example, before the corresponding rendering call is executed by the graphics pipeline, the texture of the video frame or the asset data of the model may not be fully loaded. This is particularly true when there is a scene cut in the game (e.g., a scene change in a cinematic flow, or the start of interactive gameplay after a series of menus) where the game transitions between two video frames from one scene to another. In conventional computing architectures, the graphics pipeline stalls to wait for the asset to be fully loaded before continuing execution. On the other hand, the asset recognition computing architecture 200 of the present embodiment is configured to start the execution of the graphics pipeline even if the asset (e.g., texture and / or model) is not fully loaded. Specifically, (when executed by the CPU) all the game has to do is call the GPU API 250 for the rendering call (as it would normally always do). Since the asset recognition computing architecture 200 can load the available assets in a timely manner, game developers do not have to focus on adjusting the game's scenes (e.g., reducing the complexity of the scenes) in order to avoid stalls or performance degradation that can occur when rendering particularly complex scenes or switching scenes during scene cuts, as would be the case on more conventional computing architectures.

[0038] The asset recognition computing architecture 200 includes an asset store 210 configured to store a plurality of assets of the game. In some embodiments, the asset store 210 can be configured as a hard drive or using an SSD. As memory technology evolves, other memory devices that provide faster access than ever before can be used for the asset store 210. Thus, all the assets used to render a scene can be loaded from the asset store 210 at speeds of several milliseconds or tens of milliseconds or more.

[0039] In an embodiment, various communication and / or bus interfaces may be used to communicate with the asset store 210. For example, PCI Express (PCIe) may be used to access the asset store 210. In one embodiment, Non-Volatile Memory Express or NVM Express (NVMe) defines the specification of an open logical device interface for accessing non-volatile storage media connected via a PCIe bus, and access to the asset store 210 is implemented via NVMe to improve the access performance to the memory devices within the asset store 210.

[0040] When loading a game to execute, assets may be loaded into the asset store 210. Specifically, the asset store 210 holds the LOD data 290 for the corresponding draw calls and holds all the LOD data for all draw calls. Each asset includes one or more LODs that can be used by the corresponding draw call. For example, a texture asset may be represented by a chain of mipmaps or texture maps having one or more LODs, and each texture map (LOD) or each mipmap (LOD) provides a different resolution or representation of the texture. For example, a texture map or mipmap may be an image mapped onto the surface of an object or polygon. Texture mapping may be used to reduce the number of polygons of an object that need to be rendered. Further, a model asset may represent an object. An asset may include one or more models (e.g., LODs) of various resolutions for representing that object. Thus, the necessary model with the appropriate resolution of the asset may be selected for rendering via the graphics pipeline.

[0041] Furthermore, the asset store 210 includes asset recognition data (AAD) for each asset, and all or part of the asset recognition data also resides in the system memory 220. For example, each asset can be tagged with a corresponding identifier (e.g., asset ID), and the identifier can be stored in the asset store 210 in association with the corresponding asset. Also, each asset can include other data providing details of the asset. For example, the AAD of each asset can include information describing the LOD set of that asset. This representative information can be stored in association with the corresponding asset within the asset store. The AAD data can be updated in various ways in which the asset is recognized, such as when the asset is locally generated by the CPU 702 or GPU 716, or when the asset is accessed from the mass storage within the rack, or when the asset is accessed from within the data center, or when the asset is accessed via the network. That is, the asset store 210 can be updated in the manner in which the asset is recognized from the rack, or from other locations within the data center, or via the network, or from the game.

[0042] The asset recognition computing architecture 200 includes a system memory 220 configured to store one or more LOD data 290 used by corresponding draw calls. Further, all the LOD data of all the draw calls used to generate the corresponding video frames is loaded into the system memory 220. Specifically, in one embodiment, when a game running on the CPU 702 uses the GPU API 250 to bind an asset (e.g., texture, LOD data, minimum LOD data, etc.) to a corresponding draw call at bind time, the CPU 702 also requests the asset store controller 230 to load the LOD data 290 from the asset store 210 into the system memory 220. This request to load the LOD data into the system memory 220 can be implemented as an extension of the GPU API 250 or by a separate API.

[0043] The system memory 220 may include a plurality of command buffers 240. Each command buffer is configured to store an AAD 260 and a corresponding draw call command, and the AAD and the command are stored at the time of binding. The GPU 716 can be made to recognize the AAD 260 when accessing the command buffer to execute the corresponding command. Specifically, the CPU 702 uses the GPU API 250 to write the AAD and the command of the draw call into the corresponding command buffer. As shown in FIG. 2, the command buffer 240a is used to generate a video frame - F0 (for example, frame 0). The command buffer 240a includes draw call commands 241 (for example, commands 241a to 241n) and the AAD of the corresponding draw call. As described above, the asset recognition data of the corresponding asset may include one or more of the asset IDs 261 of the required assets (such as textures, models, etc.), and a pointer 262 to the asset ID 261 in the asset store 210 and / or the system memory 220, the minimum LOD 263 of the required asset ID (and the corresponding asset), the load state 264 of the asset ID, and the load priority 265 of each asset ID (for example, the load priority of the asset with respect to other assets required by the corresponding draw call or other draw calls). For example, the priority of the asset used for an object close to the camera may be high or may be the highest. Subsequent video frames are rendered using command buffers configured similarly. For example, the command buffer 240b is used to generate a video frame - F1 (for example, frame - 1), ···, and the command buffer 240n is used to generate a video frame - Fn (for example, frame - n).

[0044] Furthermore, the command is generated by the CPU 702, included in the command buffer, and can provide a notification when the corresponding frame has finished rendering. In one embodiment, the notification is provided when the corresponding asset of the video frame has finished rendering. Thus, the notification is generated by the GPU 716 that executes the draw call commands in the graphics pipeline and is delivered to the asset store controller 230 using the GPU API 250.

[0045] The asset-aware computing architecture 200 includes an asset store controller 230 configured to manage the storage and access of assets in the asset store 210, and more specifically, to manage the flow of asset-aware information across the entire computing architecture 200. In one embodiment, the asset store controller 230 is configured as an "on-chip complex" or unit that includes one or more coprocessors. In other embodiments, the asset store controller 230 is configured as an application that runs on the CPU 702 with the assistance of other on-chip units.

[0046] As described above, the asset store controller 230 is configured to receive a stream of AAD information from the CPU 702 using the GPU API 250 for one or more draw calls used to generate video frames, either separately or using the API. For example, the AAD includes the identification (e.g., asset ID) of all assets required for the rendering and / or rendering of the corresponding video frame, and the load priority of each of those assets. For example, the load priority of an asset is given in comparison to the load of other assets required by that draw call. The asset store controller can further prioritize the minimum LOD required for the asset when executing draw calls in the graphics pipeline.

[0047] Upon receiving the asset information, the asset store controller 230 begins to load one or more assets (e.g., LOD data) of the rendering call from the asset store 210 to the system memory 220 based on the AAD (e.g., the load priority of the asset). Specifically, in one embodiment, the request to load the LOD data of the asset of the rendering call occurs substantially simultaneously with the writing of the command of the rendering call and the AAD of the rendering call to the command buffer. That is, the request to load the LOD data of the asset of the rendering call occurs at bind time. By simultaneously triggering the loading of the LOD data of one or more assets of the rendering call when one or more assets are bound, at least the minimum LOD can be loaded to timely execute the rendering call in the graphics pipeline by the GPU 716. Further, the bind-time trigger to load at least the minimum LOD of the assets required for the rendering call reduces the occurrence of GPU stalls where the GPU 716 waits for the loading of the required assets. The request to load the LOD data does not have to occur at bind time. In other embodiments, the AAD is written to the command buffer together with the rendering call command and, separately and at another time, the AAD is sent to the asset store controller 230 to request the loading of the asset.

[0048] In one embodiment, the loading is performed according to the priority. For example, the "minimum LOD" of all textures is loaded first, and then, based on the priority information, higher LODs are loaded. Also, with respect to loading, some assets have a higher priority than other assets, so the higher-priority assets (e.g., minimum LOD or higher LOD) are loaded first, and then the lower-priority assets (e.g., their minimum LOD or higher LOD) are loaded. In one embodiment, when the loading of one or more assets is completed, the corresponding asset being loaded and the degree to which the asset is loaded (e.g., the loading status of the corresponding LOD) are notified to the GPU 716 (using the asset ID). In other embodiments, for example, when a draw call command is executed by the GPU 716, the GPU 716 is configured to access the loading status.

[0049] The loading status can be identified by the asset store controller 230 configured to request and track the loading of the LOD of the corresponding asset. In this way, the loading state of the corresponding asset can be communicated from the asset store controller 230 to the GPU 716, for example, directly using the GPU API 250, or via a command buffer, or via fetching the loading state by the GPU 716. Since the AAD includes information related to the minimum LOD and optionally the LOD set of a particular asset, as well as the loading state of the asset into the system memory 220, the GPU 716 is configured to start executing draw calls using at least the minimum LOD of a given asset. More specifically, the GPU 716 can identify the highest LOD that has been loaded for the corresponding asset of a particular draw call based on the AAD (e.g., the loading state). Thus, the GPU 716 can execute draw calls using the highest loaded LOD without using higher LODs (e.g., unloaded LODs).

[0050] Furthermore, since asset recognition includes the loading state of the corresponding asset of a draw call, the GPU 716 is configured to determine whether the required LOD (e.g., the minimum LOD) of the corresponding asset has been fully loaded into the system memory. If the minimum LOD has not been fully loaded, the GPU can issue an "urgent load" request to load, for example, the minimum LOD or a proxy asset (if there is no proxy asset in the system memory yet). The GPU can stall until the minimum LOD or the loading of the proxy asset is completed. The determination of an emergency situation can be made before or during the execution of a draw call. For example, the asset store controller 230 and / or the GPU 716 can make the determination while a previous draw call is being executed in the graphics pipeline or before any draw call of a video frame including the use of an asset is completed by the GPU 716. If the loading of the minimum LOD is not completed, the asset store controller 230 or the GPU 716 can issue an "emergency load" request at that time.

[0051] The asset store controller 230 is also configured to release the allocation of system memory. Specifically, as described above, a notification can be generated when the rendering of a video frame is completed. Further, the AAD can include a notification from the command buffer regarding the completion of the rendering of a specific asset. The notification can be delivered from the GPU that executes the draw call commands in the command buffer to the asset store controller 230 using the GPU API 250, and the command buffer can include commands for generating and delivering the notification. In this way, by being able to target the assets of that video frame, the memory space storing these assets can be freed. Further, the asset store controller is configured to track the usage status of the assets loaded in the system memory (for example, the usage status of LOD data including the number of times a texture is read, etc.). Therefore, by being able to target the assets not used for the rendering of a video frame, the memory space storing these assets can be freed. An asset may be released in its entirety (all LODs) or only a subset of the LODs. To assist in making these decisions, in some embodiments, the GPU 716 is configured to track the usage of an asset, and the usage of an asset includes, for example, in the case of a texture, the minimum and maximum LODs used, as well as the total number of accesses or the number of accesses per LOD, and in the case of a draw call, the number of pixels written as a result of the call and execution of the draw call or their absence (some calls may be conditionally executed), and in the case of a model, the LOD used and the number of vertices generated. In some embodiments, using the usage of the loaded LODs of partially loaded assets, the loading of the highest LOD of these assets may be prioritized. For example, if the loading of LOD0 of two assets is not yet complete, and the LOD1 of one asset is expected to be used frequently, while the LOD1 of the other asset is not, then the LOD0 of the asset with frequently used LOD1 must be prioritized.

[0052] As part of the loading and releasing of system memory, operations on the page table, CPU cache, and GPU cache may be required. Therefore, the asset store controller, CPU, and GPU may be configured to accelerate these operations. For example, the GPU 716 may be configured to accelerate the loading and releasing of the system memory 220 by the asset store controller 230. The loading and releasing of memory by the asset store controller 230 requires operations on the page table and the GPU cache 718. Thus, the GPU 716 may be modified to accelerate these processes. Additionally, the CPU 702 may be configured to speed up the loading and releasing of the system memory 220 by the asset store controller 230. The loading and releasing of memory by the asset store controller 230 requires operations on the page table and the CPU cache (not shown). Thus, the CPU 702 may be modified to accelerate these processes.

[0053] In one embodiment, the CPU 702 is configured to query the asset store controller 230 regarding which assets are currently in the system memory 220 or which assets are used for a particular frame of graphics. The CPU 702 is also configured to query the asset store controller 230 regarding the content of the asset store.

[0054] In other embodiments, the CPU 702 is configured to provide useful advice to the asset store controller 230 regarding the loading of the corresponding assets. The useful advice is not necessarily related to any particular draw call being generated. For example, the useful advice may be configured as a request to load the LOD data of the assets that may be used for the predicted draw calls with a particular priority.

[0055] FIG. 3 shows a rendering pipeline of a computing system including the CPU 702 and the GPU 716 that recognize assets when generating video frames while the CPU 702 executes a game according to an embodiment of the present disclosure. The frame update is executed to meet a predetermined frame rendering speed. For example, the embodiments of the present disclosure can execute frame rendering at 60 frames per second. Other embodiments are suitable for executing frame rendering at a slower or faster speed. The rendering pipeline 301 can be implemented alone or in combination within the client device 110 and the cloud game network 190 as described above.

[0056] Specifically, the rendering pipeline includes a CPU 702, a GPU 716, and a memory that can be accessed by both of them (for example, an asset store 210 for storing rendered video frames delivered to a display or the like, a system memory 220, a shared memory, a vertex buffer, an index buffer, a depth or Z buffer, a frame buffer). The rendering pipeline (or graphics pipeline) exemplifies a general process for rendering an image, such as when using a 3D (three-dimensional) polygon rendering process. For example, the rendering pipeline 301 of a rendered image outputs corresponding color information for each pixel on the display, and the color information can represent a texture and shading (such as color, shadowing, etc.).

[0057] The CPU 702 can generally be configured to perform scene generation and / or object animation of video frames of a game executed on the CPU 702. The CPU 702 receives input geometry corresponding to objects in the 3D virtual environment. The input geometry can be represented as vertices in the 3D environment and information corresponding to each vertex. For example, an object in the 3D virtual environment can be represented as a polygon (e.g., a triangle) defined by vertices, and then the surface of the corresponding polygon is processed through the rendering pipeline 301 to achieve a final effect (e.g., color, texture, etc.). Also, texture and model information can be provided as inputs to the graphics pipeline. The operation of CPU 702 is well known and will be described schematically herein. Generally, CPU 702 performs object animations from frame to frame, which can be done by manual video recording, capture from human actors, and / or simulation according to forces applied to the object and / or forces applied by the object (e.g., external forces such as gravity and internal forces of the object that induce movement). For example, CPU 702 executes physical simulations of objects and / or other functions in a 3D virtual environment. Next, CPU 702 issues draw calls for the vertices of the polygons to be executed by GPU 716.

[0058] Specifically, the results of the animations generated by CPU 702 can be stored in a vertex buffer, and then the vertex buffer is accessed by GPU 716 configured to perform projection of the vertices of the polygons onto the display or tessellation of the polygons projected for rendering the vertices of the polygons. That is, GPU 716 can be further configured to construct polygons and / or primitives that make up the objects within the 3D virtual environment, which includes performing calculations for polygon lighting, shadowing, and shading, which depend on the lighting of the scene. Further operations can be performed, such as clipping to identify or ignore primitives outside the view frustum and rasterization to project the objects within the scene onto the display (e.g., project the objects onto the image plane associated with the user's viewpoint). At a simple level, rasterization involves examining each primitive and identifying the pixels affected by that primitive. Using primitive fragmentation, the primitive can be divided into pixel-sized fragments, each fragment corresponding to a pixel in the reference plane associated with the display and / or rendering viewpoint. One or more fragments of one or more primitives can contribute to the color of the pixels when rendering the frame on the display. For example, for a given pixel, the fragments of all primitives in the 3D virtual environment are combined with the pixel for display. That is, all the texture and shading information of the corresponding pixel is combined to output the final color value of the pixel. These color values can be stored in the frame buffer, scanned when displaying the corresponding image of the scene for each frame, and captured by the corresponding pixel.

[0059] As shown in FIG. 3, the rendering pipeline is shown to include operations performed in sequence by the CPU 702, GPU 716, and raster engine to scan out the rendered frame to the display 710. By way of example, rendering pipeline sequences 391-395 for rendering frames F0-F4 are shown. Due to space constraints, other pipeline sequences, such as sequences of frames beyond F4, are not shown. In the example shown in FIG. 3, each of the components of the rendering pipeline 301 operates at the same frame generation frequency. For example, the CPU 702 and GPU 716 can operate at the same frame generation frequency to render frames at a predefined frame rate through the rendering pipeline.

[0060] For example, during a first frame period indicated as time t-0, the CPU 702 performs physics and other simulations on the objects and distributes polygon primitives to the GPU 716 along with one or more draw calls for the video frame (e.g., F0). During a second frame period indicated as time t-1, the GPU 716 performs primitive assembly and other tasks to generate the rendered frame (F0). The frame F0 is scanned out during a third frame period indicated as time t-2. Similarly, the pipeline sequence 392 renders the frame F1 during the second and third frame periods and scans out the frame F1 during a fourth frame period indicated as time t-3. Also, pipeline sequence 393 renders frame F2 during the third and fourth frame periods and scans out frame F2 during the fifth frame period indicated as time t-4. Similarly, pipeline sequence 394 renders frame F3 during the fourth and fifth frame periods and scans out frame F3 during the sixth frame period indicated as time t-5. Also, pipeline sequence 395 renders frame F4 during the fifth and sixth frame periods and scans out frame F4 during the seventh frame period indicated as time t-6.

[0061] Figure 4 shows the use of asset recognition data by a computing system including a CPU and a GPU when generating video frames while the CPU is executing a game according to an embodiment of the present disclosure. Specifically, a portion of the rendering and / or graphics pipeline that renders video frames is shown, which includes operations executed in sequence by CPU 702 and GPU 716. As shown, the rendering pipeline is referenced to timeline 490. Rendering time 402 is shown, which spans the period from when GPU 716 starts executing a draw call until GPU 716 finishes processing the draw call. Rendering time 402 separates frame period 1 and frame period 2.

[0062] At operation 410, CPU 702 processes video frame 1 via execution of the game within and / or starting prior to frame period 1. Specifically, one or more draw calls for the video frame are generated, and the draw calls are executed by GPU 716. At binding time 401, the CPU 702 binds the assets required for one or more drawing calls to render a portion of the video frame. More specifically, at binding time 401, the CPU 702 writes the command 241 (non-filled box) of each drawing call to the corresponding command buffer (240). As shown, at operation 415, the CPU 702 writes the command 241 of the drawing call and the AAD 260 (filled box) to the command buffer 240 using the GPU API 250 from binding time 401. The commands 241 and AAD 260 can be written in a regular manner (e.g., in order) with respect to each other, or in an irregular manner (e.g., interleaved) with respect to each other.

[0063] Furthermore, the asset 417 required for one or more drawing calls to render the video frame is loaded from binding time 401 into the system memory 220. The asset 417 includes one or more LOD data 290, which includes the minimum LOD that can be used when rendering the video frame using the corresponding asset. The loading of the LOD data 290 into the system memory 220 is started without waiting for the CPU 702 to complete writing all the commands required for the generation of the video frame to the command buffer 240. However, a delay may occur before the LOD data of the asset 417 is loaded into the system memory 220 if the asset store controller 230 is busy performing other loading operations such as loading higher-priority LOD data. The deallocation 411 of the system memory may need to be executed before loading the LOD data 290 of the asset 417.

[0064] In an asset recognition computing architecture including CPU 702 and GPU 716, as described above, the loading of LOD data can be tracked. Thus, GPU 716 and / or asset store controller 230 can be configured to determine whether the loading of the minimum LOD of the corresponding asset is completed. In the case of an affirmative determination, GPU 716 can proceed to render the video frame using the minimum LOD of the corresponding asset. In the case of a negative determination, GPU 716 and / or the asset store controller can issue an emergency load request for the minimum LOD data. The determination of the load status and the request for emergency loading can be made at draw time 402, or during the execution of the previous draw call, or at the start of the GPU processing of frame 1 shown in operation 420, or at another time. As described above, the asset store controller can notify the GPU of the load status of the asset for the corresponding draw call, or the GPU can be configured to query and / or fetch the load status from the asset store controller. In some embodiments, conditional commands in the command buffer can be used to query the load status of the LOD data of asset 417 and change the command flow. For example, if the loading of the minimum LOD is not completed, the draw call can be completely skipped (i.e., GPU stalls are avoided), or rendering can be performed using a proxy asset, or customized rendering based on the loaded LOD becomes possible. In still other embodiments, the GPU can use AAD to make these same decisions. For example, if the loading of the minimum LOD data is not completed, the draw call can be completely skipped (i.e., GPU stalls are avoided).

[0065] The usage of LOD data can also be controlled using the LOD clamp 403. So that the GPU 716 does not attempt to use LOD data that has not yet been fully loaded into the system memory, at the time of rendering 402 or earlier, the GPU caches the number of the highest LOD that has been loaded (e.g., the LOD clamp), and uses only the LOD data identified as existing in the system memory before the rendering call is initiated, that is, the access to the LOD data is clamped by the LOD clamp 403. For example, the LOD clamp can be cached at the time of rendering 402, or during the execution of one or more previous rendering calls, or at the start of the entire rendering frame.

[0066] In operation 420, the GPU 716 renders a video frame (e.g., frame 1) within and / or starts before frame period 2. As described above, in operation 420a, even before a rendering call is issued, there may be any early processing by the GPU 716 to determine whether the minimum LOD of the specific asset of the corresponding rendering call has been fully loaded, based on the AAD and load tracking of that asset.

[0067] FIG. 5 is an illustration of the implementation of the level of detail data of the corresponding assets used when generating a video frame of a game executed by a central processing unit, and the load clamp while loading the LOD data, according to an embodiment of the present disclosure. The terms LOD data and LOD can be used in the same meaning, such as corresponding to an asset and / or a plurality of assets. Specifically, the LOD data of the corresponding asset is shown. As described above, the LOD data may include LOD0, LOD1, LOD2, and LOD3 of various resolutions. According to convention, LOD0 has the highest level of detail, and LOD1, 2, and 3 gradually have lower levels of detail. For example, in the case of a texture asset, the LOD data can be the mipmap chain of the texture. In the case of an asset, the minimum LOD is defined as LOD2, and during the draw calls executed in the graphics pipeline, the minimum LOD can be used for the asset, that is, the draw call can proceed without the higher LOD1 or 0, but cannot be executed without LOD3 and 2 in the system memory.

[0068] To ensure that the GPU 716 does not attempt to use LOD data that has not yet been fully loaded into the system memory, LOD clamping can be implemented. At draw time 402 or earlier, the GPU caches the number representing the highest loaded LOD. The cached value of the LOD clamp is used by the GPU when executing the draw call. Thus, the GPU clamps the GPU access to the LOD data (e.g., texture and / or model) of the corresponding asset to this cached value of the LOD clamp and does not access what does not exist in memory beyond this value. That is, LOD clamping is used to ensure that the GPU does not attempt to use LODs that have not yet been fully loaded (e.g., at draw time 402). Further, LOD clamping is used to ensure that the same set of LOD data is used throughout the execution of the draw call. Thus, the LOD data that arrives for loading into the system memory after draw time 402 (e.g., during the execution of the draw call) is not used for the execution of the draw call.

[0069] In FIG. 5, an example of an LOD clamp is shown by line 403a. In this example, at drawing time 402, LOD3 and LOD2 have been fully loaded, but since LOD1 is only partially loaded, the LOD clamp is set to 2 (representing LOD2) for the LOD value. The GPU 716 is configured not to use an LOD value higher than the LOD clamp. For example, even if a primitive using a texture is near the camera and would normally be rendered using LOD1 or 0, these higher LODs are not used if the LOD clamp is set to 2.

[0070] Also in FIG. 5, another example of an LOD clamp is shown by line 403b. In this example, at drawing time 402, LOD1 has been fully loaded, and thus the LOD clamp is set to 1 for the LOD value, and LOD1 is used by the GPU 716 when executing a draw call. In some embodiments, the LOD clamp is set at drawing time, and in other embodiments, the LOD clamp is set at a point in time prior to drawing time within the GPU processing 420 of the frame, or even earlier, such as at the start of the GPU processing 420 of the frame.

[0071] Along with the detailed description of the various client devices 110 and / or cloud game network 190 (e.g., game server 160 and / or game title processing engines 111 and 111') of FIGS. 1A - 1C and FIG. 2, the flowchart 600A of FIG. 6A illustrates a game execution method by an asset recognition computing system that uses a central processing unit and a graphics processing unit to generate video frames according to one embodiment of the present disclosure.

[0072] At 610, the method includes generating, by the CPU, a draw call for one of the video frames of the video frame. The draw call is generated during the execution of the game by the CPU. As described above, one or more draw calls can be generated in association with the rendering of the video frame.

[0073] For example, draw calls are generated during a cut scene that defines a transition between a previous scene and a new scene. To draw the new scene, it may be necessary to load many assets into system memory. However, an asset-aware computing architecture can render the new scene using minimal LOD data, thus reducing the likelihood of GPU stalls.

[0074] At 620, the method includes writing commands for the draw call to a command buffer at bind time. Specifically, a game running on the CPU binds, at bind time using the GPU API, one or more assets required for the draw call when the draw call is processed through the graphics pipeline, i.e., writes commands related to the assets to the command buffer using the GPU API.

[0075] At 630, the method includes writing asset-aware data (AAD) for the draw call to the command buffer and starting the loading of one or more LOD data from the asset store to system memory. Specifically, when the game is running on the CPU, for each draw call, AAD can be generated or accessed and / or identified. For example, AAD can include an identifier (e.g., asset ID) for each asset required for the corresponding draw call, the minimal LOD for each asset that can be used for rendering the draw call, and the load priority of each asset to system memory when compared to the priorities of other assets required for the corresponding draw call.

[0076] When the CPU writes the draw call commands and AAD to the command buffer (using the GPU API), the CPU also requests the asset store controller to load one or more LOD data for the draw call into the system memory used by the computing system. When the request is issued to the asset store controller, the loading of the LOD data is started, but the actual loading of one or more LOD data may not necessarily have occurred yet. For example, the first plurality of LOD data of the asset begins to be loaded into the system memory, the asset is used when processing a draw call, and the first plurality of LOD data includes at least the minimum LOD. Specifically, the asset store controller is configured to receive a stream of information including the AAD of one or more assets required for a draw call. The AAD includes an identifier of the asset required for rendering a video frame, a load priority of the asset, and a notification of the end of rendering of the video frame using these assets. The LOD data is loaded from the asset store to the system memory by the asset store controller. As described above, the CPU can request the asset store controller to load the LOD data using the GPU API at bind time, and in one embodiment, for example, send a request having the AAD. Also, when the AAD is received by the asset store controller, a request for the LOD data can be made using the AAD, and thus, in other embodiments, the reception of the AAD triggers the loading of one or more LOD data from the asset store to the system memory.

[0077] The asset store controller is configured to adjust the loading of the asset into the system memory and, in one embodiment, notify the GPU of the loading status by some means, such as writing the loading status to the AAD of the command buffer. Also, conditional commands can be placed in the command buffer and executed, whereby the GPU queries and / or fetches the loading status of the assets (e.g., LOD data) used in a draw call from the asset store controller.

[0078] Furthermore, the asset store controller is configured to manage the deallocation of system memory. Specifically, the CPU and / or the asset store controller are notified by the GPU that the execution of a draw call has been completed. For example, a command that generates a notification indicating that a draw call has completed rendering a video frame and delivers the notification to the CPU and / or the asset store controller can be placed in the command buffer of the draw call. In this way, the memory space storing one or more assets used by the draw call can be freed. In other embodiments, the asset store controller can track the usage of assets stored in system memory. If an asset is not being used to render the current video frame in the graphics pipeline, the asset store controller can determine that the asset has become unnecessary for the processing of the draw call and can free the corresponding memory space.

[0079] At 640, the method includes the GPU executing a draw call for a frame using the LOD data written to system memory. The CPU issues a draw call to be executed by the GPU at the next draw (e.g., within the current frame period), and the draw call was generated by the CPU during the previous frame period. In this way, the video frame can be rendered in time to be scanned out in the next frame period. The GPU executes the draw call using at least a minimum amount of LOD data based on the AAD. For example, each asset has a minimum LOD, which can be used to process the corresponding draw call that requires the corresponding asset. Since the asset-aware computing architecture (e.g., the CPU and / or the asset store controller operating in conjunction with the GPU) can track the loading of assets (e.g., identify the loading state of each asset), the GPU can determine which LOD to use for the corresponding asset when the draw call that requires the corresponding asset is executed. For example, in the assets introduced above, the loading of the first plurality of LOD data into the system memory is tracked by the CPU and / or the asset store controller. In this way, the CPU and / or the asset store controller that performs the tracking can call the GPU API or otherwise notify the GPU of the loading state of the asset (e.g., to what extent the loading of the first plurality of LOD data is completed). Further, the GPU recognizes the first plurality of LOD data associated with the asset. In one embodiment, in order to ensure that higher LODs are utilized only when they are loaded into the system memory, the cached value of the maximum or highest LOD data loaded into the system memory is used, which is more fully explained in connection with FIG. 6B below.

[0080] FIG. 6B is a flow diagram 600B showing taking into account the load status of the LOD data of an asset when executing corresponding draw calls to generate a video frame, according to one embodiment of the present disclosure. As described above, the LOD data of the draw call is loaded into the system memory. Specifically, at 650, the LOD data is loaded according to the priority based on the AAD. For example, the LOD data of the corresponding asset can be prioritized to load the LOD data in a specific order. For example, in the order based on the priority of the AAD, the minimum LOD data from the plurality of LOD data of all assets is first loaded, and then the remaining LOD data is loaded from the lowest to the highest level of detail again in the order based on the priority of the AAD.

[0081] The GPU and / or the command buffer can execute one of different actions according to the load state, as will be described later. That is, the GPU and / or the command buffer has conditional execution control, whereby it is possible to execute different actions based on the LOD load state. Specifically, some calls and / or commands within the corresponding draw call can be conditionally execution-controlled based on the LOD load status. For example, the command buffer can have branch instructions or their equivalents, which, when executed, perform different actions based on the LOD load state. For example, the LOD load state can be checked using these branch instructions, and when a specific LOD is loaded, one set of commands is executed, but if not, another set of commands is executed.

[0082] In the determination step 655, a comparison is made between the highest LOD data that has been fully loaded into the system memory and the minimum LOD data required for the corresponding asset based on the AAD. This comparison can be made during or before rendering, as described above. For example, the comparison can be made during the execution of one or more previous draw calls or at the start of the entire rendering frame.

[0083] The GPU is configured to use at least the minimum LOD of the asset and can use the highest LOD that is loaded and higher than the minimum LOD when processing draw calls for rendering the corresponding video frame. When using an LOD higher than the minimum LOD, the GPU utilizes "LOD clamping", which is the cached value of the highest LOD that was fully loaded into the system memory at the time of execution (e.g., at draw time) or the previous time. In this case, the GPU is configured to ensure that the minimum LOD data is loaded and stored in the system memory. Also, to ensure that a higher LOD (e.g., an LOD higher than the minimum LOD data) is only utilized when it is loaded in the system memory, LOD clamping is used, i.e., the GPU uses LOD clamping to ensure that higher LOD data that was not loaded at the time when the LOD clamp was identified is not accessed during the execution of the corresponding draw call. For example, at 656, the LOD clamp is set to the value representing the highest LOD data for which the load has been completed. That is, the LOD clamp is the captured LOD load state. In this way, the GPU avoids attempting to access LOD data that has not yet been fully loaded into the system memory. Thus, at 657, the GPU uses the LOD data clamped to the LOD clamp (e.g., using the highest loaded LOD data) to execute the frame draw call.

[0084] On the other hand, at the determination step 655, if the highest loaded LOD data is lower than the minimum LOD of the corresponding asset, various operations can be performed at 660. If the GPU determines that the minimum LOD has not been fully loaded into the system memory, at 661, the GPU and / or the asset store controller can request to load the default (i.e., proxy) asset into the system memory for use when processing the corresponding draw call (if the default asset does not yet exist in the system memory). At 662, the default asset is used when the GPU executes the draw call. Alternatively, at 663, an emergency load of the minimum LOD data can be performed, and the GPU stalls until the minimum LOD data is successfully loaded. In some embodiments, the emergency load can be triggered early, such as during the previous draw call or at the start of frame rendering. At 664, the minimum LOD data is used when the GPU executes the draw call. Alternatively, at 665, the GPU can decide to skip the execution of the draw call as described above (i.e., to avoid stalling the GPU).

[0085] FIG. 7 shows the components of an exemplary device 700 that can be used to execute aspects of various embodiments of the present disclosure. For example, FIG. 7 shows a central processing unit and a graphics processing unit to enable minimal LOD data to be used for one or more assets required for a draw call to render a video frame, according to one embodiment of the present disclosure, an exemplary hardware system suitable for executing a game by an asset-aware computing system that generates a video frame using. This block diagram shows a device 700 that can incorporate or be a personal computer, a server computer, a game console, a mobile device, or other digital devices, each of which is suitable for implementing embodiments of the present invention. The device 700 includes a central processing unit (CPU) 702 that operates a software application and optionally an operating system. The CPU 702 can be composed of one or more homogeneous or heterogeneous processing cores.

[0086] According to various embodiments, the CPU 702 is one or more general-purpose microprocessors having one or more processing cores. Further embodiments can be implemented using one or more CPUs having a microprocessor architecture particularly adapted for high-parallel and compute-intensive applications, such as media and interactive entertainment applications, or applications configured to perform graphics processing during the execution of a game. The asset store controller 230 may exist as an independent unit, or the functions of the controller may be executed by the CPU 702.

[0087] Memory 704 stores applications and data used by CPU 702, asset store controller 230, and GPU 716. Storage 706 provides non-volatile storage and other computer-readable media for applications and data. Storage 706 may include a fixed disk drive, removable disk drive, flash memory device, and CD-ROM, DVD-ROM, Blu-ray (registered trademark), HD-DVD, or other optical storage device, as well as signal transmission and storage media. User input device 708 communicates user input from one or more users to device 700. Examples of user input device 708 may include a keyboard, mouse, joystick, touch pad, touch screen, still or video recorder / camera, and / or microphone. Network interface 714 enables device 700 to communicate with other computer systems via an electronic communication network and may include wired or wireless communication via a local area network and a wide area network such as the Internet. Audio processor 712 is adapted to generate analog or digital audio output from instructions and / or data provided by CPU 702, memory 704, and / or storage 706. The components of device 700 including CPU 702, the graphics subsystem including GPU 716 and GPU cache 718, asset store controller 230, asset store 210, memory 704, system memory 220, data storage 706, user input device 708, network interface 714, and audio processor 712 are connected via one or more data buses 722.

[0088] The asset store 210 may also be included to store assets required for the game, for example when rendering a scene. The asset store may be part of the memory 704 and / or the storage 706. The system memory 220 stores assets used in draw calls, for example during the execution of the game and when rendering a scene.

[0089] The graphics subsystem 714 is further connected to the data bus 722 and components of the device 700. The graphics subsystem 714 includes a graphics processing unit (GPU) 716 and a graphics memory 718. The graphics memory 718 includes a display memory (e.g., a frame buffer) used to store pixel data for each pixel of the output image. The graphics memory 718 may be integrated into the same device as the GPU 716, connected to the GPU 716 as a separate device, and / or implemented within the memory 704. Pixel data may be provided directly from the CPU 702 to the graphics memory 718. Alternatively, the CPU 702 provides data and / or instructions defining the desired output image to the GPU 716, and the GPU 716 generates pixel data for one or more output images based thereon. The data and / or instructions defining the desired output image may be stored in the memory 704 and / or the graphics memory 718. In an embodiment, the GPU 716 includes a 3D rendering function that generates pixel data for the output image from instructions and data defining the geometry, lighting, shading, texturing, motion, and / or camera parameters of the scene. The GPU 716 may further include one or more programmable execution units capable of executing shader programs.

[0090] The graphics subsystem 714 periodically outputs pixel data of an image from the graphics memory 718 to be displayed on the display device 710 or projected by the projection system 740. The display device 710 can be any device that can display visual information in response to a signal from the device 700, including CRT, LCD, plasma, and OLED displays. The device 700 can provide an analog or digital signal to the display device 710, for example.

[0091] Other embodiments for optimizing the graphics subsystem 714 can include multi-tenancy GPU operation where GPU instances are shared among multiple applications and a distributed GPU supports a single game. The graphics subsystem 714 can be configured as one or more processing devices.

[0092] For example, the graphics subsystem 714 can be configured to perform multi-tenancy GPU functions. In one embodiment, one graphics subsystem can implement the graphics and / or rendering pipelines of multiple games. That is, the graphics subsystem 714 is shared among multiple games being executed.

[0093] In other embodiments, the graphics subsystem 714 includes multiple GPU devices that are combined to perform the graphics processing of a single application running on the corresponding CPU. For example, multiple GPUs can perform alternate frame rendering, where GPU1 renders the first frame, GPU2 renders the second frame, and so on in sequential frame periods until the last GPU is reached, at which point the first GPU renders the next video frame (e.g., if there are only two GPUs, GPU1 renders the third frame). That is, when rendering a frame, the GPU circulates. The rendering operations may be repeated, and before GPU1 finishes rendering the first frame, GPU2 may start rendering the second frame. In another embodiment, different shader operations may be assigned to multiple GPU devices in the rendering and / or graphics pipeline. The master GPU performs the main rendering and composition. For example, in a group including three GPUs, master GPU1 can perform the main rendering (e.g., the first shader operation) and composition of the outputs from slave GPU2 and slave GPU3, slave GPU2 can perform the second shader (e.g., fluid effects such as rivers) operation, slave GPU3 can perform the third shader (e.g., particle smoke) operation, and master GPU1 composes the results from GPU1, GPU2, and GPU3 respectively. In this way, different GPUs can be assigned to execute different shader operations (e.g., twinkling flags, wind, smoke generation, fire, etc.) to render video frames. In still other embodiments, each of the three GPUs can be assigned to different objects and / or parts of a scene corresponding to a video frame. In the above embodiments and implementations, these operations may be executed (simultaneously in parallel) within the same frame period, or may be executed (sequentially in parallel) within different frame periods.

[0094] Therefore, the present disclosure describes a computing system architecture including asset recognition components such as a central processing unit and a graphics processing unit for generating video frames through a graphics pipeline in various embodiments, and a method for implementing the same. Asset recognition by the CPU and GPU provides identification and tracking of assets used in the graphics pipeline, thereby enabling the use of a minimum number of assets (e.g., minimum level of detail - LOD) in the processing through the graphics pipeline when generating the corresponding video frames.

[0095] It should be understood that the various embodiments defined herein can be combined or assembled into specific embodiments using the various features disclosed herein. Thus, the provided examples are only a few of the possible examples and are not limited to the various embodiments made possible by combining various elements to define more embodiments. In some examples, some embodiments may include fewer elements without departing from the spirit of the disclosed embodiments or equivalent embodiments.

[0096] Embodiments of the present disclosure can be implemented in various computer system configurations including, for example, handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers. Embodiments of the present disclosure can also be implemented in a distributed computing environment where tasks are performed by remote processing devices linked via a wired or wireless network.

[0097] With the foregoing embodiments in mind, it should be understood that embodiments of the present disclosure can use various computer-implemented operations involving data stored in a computer system. These operations are operations that require physical manipulation of physical quantities. Any of the operations described herein that form part of embodiments of the present disclosure are useful machine operations. Embodiments of the invention also relate to devices or apparatuses for performing these operations. The apparatus can be specially constructed for the required purposes, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Specifically, various general-purpose machines can be used with a computer program written according to the teachings herein, or it may be more convenient to construct a more specialized apparatus for performing the required operations.

[0098] The present disclosure can also be embodied as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data that can later be read by a computer system. Examples of computer-readable media include hard drives, network attached storage (NAS), read-only memory, random access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and other optical and non-optical data storage devices. The computer-readable media can include computer-readable tangible media distributed over a network-connected computer system so that the computer-readable code is stored and executed in a distributed fashion.

[0099] Although the method operations have been described in a particular order, it should be understood that other maintenance operations may be performed between operations, or the operations may be adjusted so that they occur at slightly different times, or the operations may be distributed within the system to allow the processing operations to occur at various processing-related intervals, as long as the processing of the overlay operations is performed in the desired manner.

[0100] Although the foregoing disclosure has been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications can be practiced within the scope of the appended claims. Accordingly, the embodiments should be regarded as illustrative rather than restrictive, and the embodiments of the present disclosure are not limited to the details provided herein and may be modified within the scope and equivalents of the appended claims.

Claims

1. A central processing unit (CPU) executes a video game to generate draw calls for rendering of image frames, binds a plurality of assets to the draw calls at binding time, the plurality of assets including asset recognition data, passes the asset recognition data to an asset store controller, the asset recognition data including at least one of a corresponding asset ID, a corresponding minimum level of detail (LOD), and a corresponding load priority for each of the plurality of assets, configures a command buffer for execution of the draw calls by a graphics processing unit (GPU), starts loading a plurality of minimum LODs for the plurality of assets from an asset store to system memory based on a plurality of load priorities for the plurality of assets when the assets are bound to the draw calls at the binding time, executes the command buffer by the GPU using at least the plurality of minimum LODs for the plurality of assets. A method.

2. Loads the asset recognition data into the command buffer. The method according to claim 1.

3. Based on the plurality of load priorities, after loading of the plurality of minimum LODs for the plurality of assets is completed, loads the next maximum level of LOD for each of the plurality of assets, wherein the first next maximum level of LOD for a first asset with a higher priority is loaded before the second next maximum level of LOD for a second asset with a lower priority. The method according to claim 1.

4. The asset store controller dynamically tracks a plurality of load states for the plurality of assets, communicates the plurality of load states to the GPU, and the GPU starts execution of the command buffer when the plurality of load states indicate that at least the plurality of minimum LODs have been loaded into the system memory. The method according to claim 1.

5. The GPU starts execution of the command buffer using a maximum LOD corresponding to each of the plurality of assets loaded into the system memory. The method according to claim 4.

6. The GPU starts the execution of the command buffer before all corresponding loads for each of the plurality of assets are loaded. The method according to claim 4. **Claim 7** Determine that the minimum LOD for an asset is not loaded in system memory, While a draw call used for rendering the previous image frame is being executed, the GPU issues an emergency load request to the asset store controller to load the minimum LOD for the asset. The method according to claim 1. **Claim 8** A computer system, A processor, A memory coupled to the processor and storing instructions that, when executed by the computer system, cause the computer system to execute a method, wherein A central processing unit (CPU) executes a video game and generates draw calls for rendering image frames, Bind a plurality of assets to the draw calls at bind time, the plurality of assets including asset recognition data, Pass the asset recognition data to an asset store controller, the asset recognition data including at least one of a corresponding asset ID, a corresponding minimum level of detail (LOD), and a corresponding load priority for each of the plurality of assets, Configure a command buffer for execution of the draw calls by a graphics processing unit (GPU), When the asset is bound to the draw call at bind time, start loading a plurality of minimum LODs for the plurality of assets from the asset store to system memory based on a plurality of load priorities for the plurality of assets, Execute the command buffer by the GPU using at least the plurality of minimum LODs for the plurality of assets. Computer system. **Claim 9** In the method, further, Load the asset recognition data into the command buffer. The computer system according to claim 8. **Claim 10** In the method, further, Based on the plurality of load priorities, after the loading of the plurality of minimum LODs for the plurality of assets is completed, load the next highest level of LOD for each of the plurality of assets. The first maximum level LOD for the first asset with the highest priority is loaded before the second maximum level LOD for the second asset with the lowest priority. The computer system according to claim 8.

11. In the method, further, The asset store controller dynamically tracks a plurality of load states for the plurality of assets, Communicates the plurality of load states to the GPU, and the GPU starts the execution of the command buffer when the plurality of load states indicate that at least the plurality of minimum LODs have been loaded into the system memory. The computer system according to claim 8.

12. In the method, The GPU starts the execution of the command buffer using the maximum LOD corresponding to each of the plurality of assets loaded into the system memory. The computer system according to claim 11.

13. In the method, The GPU starts the execution of the command buffer before all the corresponding loads for each of the plurality of assets are loaded. The computer system according to claim 11.

14. In the method, further, It is determined that the minimum LOD for the asset is not loaded into the system memory, While the drawing calls used for rendering the previous image frame are being executed, the GPU issues an emergency load request to the asset store controller to load the minimum LOD for the asset. The computer system according to claim 8.

15. A computer-readable medium having recorded thereon a computer program for executing a method, The central processing unit (CPU) has program instructions for executing a video game and generating drawing calls for rendering an image frame, Has program instructions for binding a plurality of assets to the drawing calls at the time of binding, and the plurality of assets include asset recognition data, Has program instructions for passing the asset recognition data to an asset store controller, and the asset recognition data includes at least one of a corresponding asset ID, a corresponding minimum level of detail (LOD), and a corresponding load priority for each of the plurality of assets. It has program instructions for configuring a command buffer for executing the drawing call by a graphics processing unit (GPU). It has program instructions for starting to load a plurality of minimum LODs for the plurality of assets from the asset store to the system memory based on a plurality of load priorities for the plurality of assets when the asset is bound to the drawing call at the time of the binding. It has program instructions for executing the command buffer by the GPU using at least the plurality of minimum LODs for the plurality of assets. A computer-readable medium. [

16. ] It has program instructions for loading the asset recognition data into the command buffer. The computer-readable medium according to claim 15. [

17. ] It has a program for loading the next maximum level of LOD for each of the plurality of assets after the loading of the plurality of minimum LODs for the plurality of assets is completed based on the plurality of load priorities, wherein the first next maximum level of LOD for the first asset with a higher priority is loaded before the second next maximum level of LOD for the second asset with a lower priority. The computer-readable medium according to claim 15. [

18. ] It has program instructions for dynamically tracking a plurality of load states for the plurality of assets by the asset store controller, communicating the plurality of load states to the GPU, and the GPU has program instructions for starting the execution of the command buffer when the plurality of load states indicate that at least the plurality of minimum LODs have been loaded into the system memory. The computer-readable medium according to claim 15. [

19. ] The GPU has program instructions for starting the execution of the command buffer using the maximum LOD corresponding to each of the plurality of assets loaded into the system memory, and the GPU starts the execution of the command buffer before all the corresponding loads for each of the plurality of assets are loaded. The computer-readable medium according to claim 18. [

20. ] It has program instructions for determining that the minimum LOD for an asset has not been loaded into the system memory. While the rendering calls used for rendering the previous image frame are being executed, the GPU issues program instructions for issuing an emergency load request to the asset store controller to load the minimum LOD for the asset. The computer-readable medium according to claim 15.

Images