System and method for efficient tracking and rendering of dynamic entities in video games

Abstract

A system and method for efficient tracking and rendering of dynamic entities in video games. In order to avoid costly computational loads associated with simulating physics for each dynamic entity independently, then rendering each entity, the system and method stores the pose of dynamic entities after physics calculations and amalgamates pose data into an optimized data structure to allow a graphics processing unit (GPU) to re-create the pose of and display a plurality of dynamic entities as part of a single rendering pass, referring to only a single set of reference data. The method achieves this by recording the position and rotation of a set of pre-defined bones of importance of a skeleton onto a single location, a single image file, which contains all the information necessary for the GPU to pose and display each of the dynamic entities simply by referencing some pre-defined definitions.

Description

SYSTEM AND METHOD FOR EFFICIENT TRACKING AND RENDERING OF DYNAMIC ENTITIES IN VIDEO GAMESFIELD

[0001] Certain aspects of the present disclosure generally relate to a computer -implemented methods for tracking and rendering a plurality of three-dimensional objects within a virtual space, specifically, for tracking and rendering dynamic interactable three-dimensional entities within a user-interactable game space.BACKGROUND

[0002] Computer-generated virtual spaces are used for many different purposes, such as in the production of static art, films, or video games. Such spaces are typically populated by various objects and entities, which are tracked and rendered in three dimensions for display. In certain applications, such as in video games, certain objects must also be interactable by users, forming the game world. The degree of interactivity of objects in such game worlds typically vary from static background objects, to dynamic actor objects. Dynamic actor objects are typically more important to the game experience, and are typically tracked and rendered with greater fidelity, while other background or environment objects are only displayed statically in order to optimize resource usage. In many cases, a dynamic actor object’s role may cease, such as when it has been removed from the gameplay as an active element. In such cases, the dynamic actor object is typically disabled, and may be replaced by a lower-fidelity static object, or is simply removed altogether.

[0003] This behavior is a characteristic of simulated, video game worlds, which differ from reality, and is typically implemented as a necessary concession due to limited computational resources. In the scenario of a video game, a typical dynamic actor may be a player character, or a non-player character, which may, during the course of gameplay progression, become removed from play, such as if the character dies, for example. In typical implementations, the active character model is typically tracked and rendered with high fidelity, as these characters are the most important gameplay elements. However, when a character dies, its gameplay role typically comes to an end. To visualize this change of gamestate for the character, a pre-determined death animation may play, or in other cases, allowing the corpse of the character to dynamically fall to the ground under simulated ragdoll physics. While pre-determined death animations reduce computational load, allowing the corpse to interact dynamically with the environment under simulated physics allows for more intricate interaction with other objects and terrain geometry. However, the computational load required for simulating physics for each corpse quickly becomes unfeasible as the number of dead characters increases, and thus, typically, corpses are deleted from the game state a set amount of time after character death. This, however, places some restriction on game design and gameplay interaction, and may be detrimental to immersion when bodies are abruptly removed from the environment.

[0004] Accordingly, it is desired to provide a new tracking and rendering system for persistent dead entities which offers improved computational efficiency over existing methods.SUMMARY

[0005] Without limiting the scope of the appended claims, some prominent features are described herein.

[0006] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Note that the relative dimensions of the following figures may not be drawn to scale.

[0007] In order to avoid costly computational loads associated with simulating physics for each dynamic entity independently, then rendering each entity, the present invention provides an optimized method of storing the pose of dynamic entities after physics calculations and amalgamating pose data into an optimized data structure to allow a graphics processing unit (GPU) to re-create the pose of and display a plurality of dynamic entities as part of a single rendering pass, referring to only a single set of reference data, and without needing to reference physics simulations. The method achieves this by recording the position and rotation of a set of pre-defined bones of importance of a skeleton onto a single location - a single image file, which contains all the information necessary for the GPU to pose anddisplay each of the dynamic entities simply by referencing some pre-defined definitions. Compared to traditional tracking and rendering methods which tracks, and then renders each entity individually, the present invention eliminates the need to retrieve information multiple times from multiple memory locations in order to have the complete information to pose and display multiple entities, and effectively allows multiple entities to be treated as a single, larger and more complex entity when being displayed.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. l is a schematic block diagram illustrating aspects of a device for performing methods herein.

[0009] FIG. 2 is a schematic illustration of a layout of the data structure of a render target image according to one aspect of the present disclosure.

[0010] FIG. 3 is a schematic illustration of how skin weight information is stored in UV channels, according to one aspect of the present disclosure.

[0011] FIG. 4 is a flow chart illustrating general steps in a method for efficient tracking and rendering as described herein.

[0012] FIG. 5 is a flow chart illustrating steps for retrieving and processing rotational information according to one aspect of the present disclosure.

[0013] FIG. 6 is a flow chart illustrating steps for sampling of data for a given vertex from a render target image according to one aspect of the present disclosure.

[0014] FIG. 7 is a screen capture of a simulated video game environment with diagnostics, showing a computer device rendering a plurality of entities using traditional rendering methods.

[0015] FIG. 8 is a screen capture of a simulated video game environment with diagnostics, showing the same computer device of FIG. 7 rendering a plurality of entities using an embodiment of a method for efficient tracking and rendering as described herein.DETAILED DESCRIPTION

[0016] Various aspects of the novel systems, apparatuses, and methods are described more fully hereinafter with reference to the accompanying drawings. The teachings disclosure can, however, be embodied in many different forms and should not be construed as limited toany specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the disclosure is intended to cover any aspect of the novel systems, apparatuses, and methods disclosed herein, whether implemented independently of or combined with any other aspect of the invention. For example, an apparatus can be implemented, or a method can be practiced using any number of the aspects set forth herein. In addition, the scope of the invention is intended to cover such an apparatus or method which is practiced using other structure, functionality, or structure and functionality in addition to or other than the various aspects of the invention set forth herein. Any aspect disclosed herein can be embodied by one or more elements of a claim.

[0017] Although aspects are described herein, many variations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of the preferred aspects are mentioned, the scope of the disclosure is not intended to be limited to benefits, uses, or objectives. The detailed description and drawings are merely illustrative of the disclosure rather than limiting.

[0018] Referring to FIG. 1, a device 100 for implementing game systems is illustrated. The device 100 may include, for example, a processor 102, which may comprise a first processor being a central processor 104 and a second processor being a graphical processor 106. In some embodiments, the central processor 104 may be a central processing unit (CPU) based on 80x86 architecture such as designed by Intel™ or AMD™, or based on reduced instruction set computer architecture as designed by ARM™, for example. The graphical processor 106 may be a separate processor or graphics processing unit (GPU) specialized for digital image processing, part of a separate device, or may be the central processor 104. The device 100 may also include a user interface 108 for receiving textual or directional commands such as a keyboard, mouse, controller, joystick or any other input device, one or more sensors 110 such as cameras, motion sensors, thermometers, or any other sensor, an input / output port 112 to enable communication between the device and a network 130, an audio and / or visual output 114 such as a display and / or speakers or headset, a volatile memory 116 such as random-access memory (RAM), or non-volatile memory 118 such as a hard drive, solid statedrive, or any other storage device. The audio and / or visual output 116 may be partially coupled to the graphics processor 106, such as a display being coupled to a graphics processing unit. The device may optionally be contained within a housing 120, such as a computer case, laptop, console, tablet, smart phone, or any other device housing, or may have portions external to the housing 120, such as having an external audio and / or visual output 116, and / or external user interface 108 devices such as a wireless controller, for example. The network 130 may comprise a network of electronic devices (not shown), including devices similar to device 100, or other, specialized devices such as server devices or client devices.

[0019] In accordance with a disclosed aspect, there is provided a system for efficient tracking and rendering of dynamic entities. In some embodiments, the dynamic entities may be 3- dimensional geometry made up of a plurality of polygons forming a mesh that can be animated, such as by a skeletal animation, wherein the position and orientation of 3- dimensional geometry is defined by a hierarchical set of interconnected reference points, or bones, having associated polygons, with respect to which other bones are positioned, can translate, or about which can rotate, for example. Together, the bones of a dynamic entity form a skeleton. In other embodiments, the reference points need not all be hierarchical or interconnected, but are nevertheless used to represent a description of the motion of the associated parts of the 3-dimensional geometry. The system comprises a first data structure being a skeleton data structure for associating a render target image data structure to a skeleton, and a second data structure being a mesh data structure of an associated skeleton. An example of a render target image data structure 200 is shown in FIG. 2, and stores the vector components of position and rotation information, and an additional value such as durability, associated with each predefined bone of a skeleton in the form of color channels (red, green, blue, alpha) of pixels of an image file, in the form of a 2-dimensional matrix, with a first direction of the matrix, or a column of pixels representing individual predefined bones of the skeleton, and a second, perpendicular, direction in the matrix, or rows being different instances of entities having such a skeleton. In some embodiments, as shown in FIG. 2, each pixel of the render target image may have a red, blue, green, and alpha channel, and the position coordinates, as a fixed-point value distance from the origin of the mesh,may be stored in the alpha, green, and blue channels of one pixel, while the first, second, third and fourth rotation quaternion normalized vector coefficients are stored in the red, green, blue and alpha channels respectively. In such an embodiment, there is a spare channel, red in the exemplary case detailed above, for each predefined bone as part of the position information. This additional channel can be used to store additional data associated with the predefined bone. In the embodiment of FIG. 2, the top half of the render target image stores the position information (and health) of each predefined bone, while the bottom half of the render target image stores information on the normalized quaternion rotation components. Use of quaternions to store rotational information has the added benefit of avoiding precision issues commonly encountered when using other traditional forms of rotational information storage such as Euler angles, such as gimbal locking when two of the three axes of rotation of a 3D object align resulting in a loss of a rotational degree of freedom.

[0020] The skeleton data structure defines the bone layout, replication behavior, and collision associated with a skeleton, and provides instructions on how the bones of the skeleton are arranged, acting as a guide when converting between a posed skeleton and the render target image. The mesh data structure defines a mesh associated with the skeleton data structure and how the actual three-dimensional object volume of portions of the mesh corresponds to the skeleton. In some embodiments, the skeleton, the mesh, the render target image data structure, and the mesh data structure may be predefined and stored within the non-volatile memory 118, and retrieved by the processor 102 as necessary. The processor 102 can then use the data structures.

[0021] In addition to the target image data structure and the skeleton data structure, there may also be a third data structure, or a bone weight data structure 300, as shown in FIG. 3, used to implement skin weighting, or defining a degree of association between a vertex and its two most closely associated bones. In a preferred embodiment, two bone weights are stored where the red channel refers to the bone 0 index, the green channel is the bone 0 weight, the blue channel is the bone 1 index, and the alpha channel is the bone 1 weight. In this preferred embodiment, an additional bone could be added without needing to store its bone weight, as the weight of the third bone could be determined by subtracting the sum of weights for bone 0 and bone 1 from a value of 1, as the weights should always be normalized to sum to 1.Alternative embodiments using 3 or 4 bone weights were considered and tested, but were demonstrated not to have appreciably different performance. In other embodiments, skin weights may be packed into a UV channel, which typically defines the correlation between a 3D surface (X, Y, and Z axes) and a 2D texture (U and V axes). In one exemplary embodiment, as shown in FIG. 3, UV channels of the mesh are used to determine bone positions. In this exemplary embodiment, UVs 0, 1, and 2 are used. This avoids requiring an additional texture or location on the render target image which must be accessed to determine the position of the bones. While the UV lookup must be repeated for each vertex, it is not a significant cost in terms of computational power - it is less computationally intensive to access UV channels than it is to access a texture on a random basis, as UV channels are mapped to vertices and already loaded into volatile memory, meaning no new resources need to be loaded in order to rapidly access this information.

[0022] In accordance with a disclosed aspect, there is provided a method 400 for the efficient tracking and rendering of dynamic entities. The method involves a grouping step 402, a transformation, modifying or baking step 404, a mapping step 406, and a rendering step 408.

[0023] The grouping step 402 comprises collecting, by a processor, a list of related or similar entities sharing identical or similar skeleton and mesh data, and forming a group of entities to be rendered. The similar entities need not be identical, but preferably share an identical skeleton, or share a similar skeleton with key bones which are identical or analogous. Key bones of a representative skeleton are identified as predefined bones when setting up the environment. The predefined bones need not be every bone in the skeleton of the entity. In an exemplary embodiment, only bones with associated data defining the shape of an object for physical collisions, or colliders, and can move and interact in simulated physics, are selected. In other exemplary embodiments, every bone in a skeleton can be a predefined bone. Additional information can be associated with the predefined bones. For example, each predefined bone may also have associated collision and navigation data for pathfinding, allowing for other characters and entities in the game world to interact with and navigate about an entity in the group of entities. In some embodiments, such additional associate information is to associate each vertex in a predefined source skeletal mesh with a predefined bone.

[0024] The transformation, modification or baking step 404 involves transforming the mapping data onto a mesh asset. The transformation, modification or baking step 404 involves defining and converting a skeletal mesh comprising a set of polygons forming a surface, and a hierarchical set of interconnected bones, which are used to animated the polygons, into a static mesh. The static mesh is then transformed into a static vertex animated mesh by appending additional data onto each vertex, namely, the indices of the two most closely- associated bones of the skeleton, in the form of vertex color, and the two said bones’ original location in their respective base pose positions, as stored in the UV channels. In a preferred embodiment, two bone weights are stored where the red channel refers to the bone 0 index, the green channel is the bone 0 weight, the blue channel is the bone 1 index, and the alpha channel is the bone 1 weight. In this preferred embodiment, an additional bone could be added without needing to store its bone weight, as the weight of the third bone could be determined by subtracting the sum of weights for bone 0 and bone 1 from a value of 1, as the weights should always be normalized to sum to 1. Alternative embodiments using 3 or 4 bone weights were considered and tested, but were demonstrated not to have appreciably different performance. In other embodiments, skin weights may be packed into a UV channel.

[0025] Initial bone positions must be stored so that the relative degree of deformation of the skin of the mesh, based on bone movement, can be determined from the positions stored in the render target image. In one exemplary embodiment, as shown in FIG. 3, UV channels of the mesh are used to determine bone positions. In this exemplary embodiment, UVs 0, 1, and 2 are used. This avoids requiring an additional texture or location on the render target image which must be accessed to determine the position of the bones. While the UV lookup must be repeated for each vertex, it is not a significant cost in terms of computational power - it is easier to access UV channels than it is to access a texture on a random basis.

[0026] In a preferred embodiment, the grouping step 402, and the transformation, modifying or baking step 404 are carried out by a first device, and used to generate pre-defined assets, which may be loaded by a second device which carries out the mapping step 406 and rendering step 408. For example, the pre-defined assets may be generated and defined by game developers, and packaged along with the game files, where the user device then usesthis data as reference information for the mapping 406 and rendering steps 408 carried out in real-time during gameplay. In other embodiments, all steps may be performed by a single device.

[0027] The mapping step 406 comprises mapping and indexing the position and rotation of the predefined bone of the skeleton of each entity in the group of entities onto an associated pixel of a render target image. In some embodiments, the image is split into two sections, a first section containing pixels indexed from bone 0 to n-1 where n is the total number of bones in order to store the position data of the bones, and a second section containing pixels indexed from n to n*2-l in order to store the rotation data, in quaternions, of each bone. Additionally, in some embodiments, skin weights for determining stretch factors associated with each predefined bone can also be stored in the render target image in order to facilitate rendering of skinned meshes. In some embodiments, as shown in FIG. 2, each pixel of the render target image may have a red, blue, green, and alpha channel, and the position coordinates, such as in cartesian coordinates, as a fixed-point value distance from the origin of the mesh, may be stored in the alpha, green, and blue channels of one pixel, while the first, second, third and fourth rotation quaternion normalized vector coefficients are stored in the red, green, blue and alpha channels respectively. In such an embodiment, there is a spare channel, red in the exemplary case detailed above, for each predefined bone as part of the position. This additional channel can be used to store additional data associated with the predefined bone. In the embodiment of FIG. 2, the top half of the render target image stores information on position (and health) of each predefined bone, while the bottom half of the render target image stores information on the normalized quaternion rotation components.

[0028] In one exemplary embodiment, this channel can be used to store a health value associated with the predefined bone in order to simulate durability of a specific portion of the entity. This allows for, for example, further interactivity with the entity, as in addition to movement and rotation, it is also possible to monitor the durability of a portion of the entity, and apply different visual and gameplay effects to said portion based on its remaining durability, for additional fidelity. In an embodiment, to efficiently update the health value of a limb of the entity when the entity is shot, for example, an additive blend mode can be applied to therender target image each time the limb is hit. This prevents the need for more complex implementation of texture layer blending.

[0029] In a simplified, exemplary, embodiment, a single entity can be represented by a column of pixels of the render target image, and additional entities are represented by subsequent columns of pixels. Each pixel in a row corresponds to either the position and durability, or rotation value associated with a particular predefined bone, which is the same predefined bone for each entity. Additional entities can easily be tracked by adding additional columns. The position and rotation of each predefined bone of each entity can, as a result, be efficiently tracked separately and individually.

[0030] The rendering step 408 involves using the render target image to draw each of the entities to form an image output for a display. In one exemplary embodiment, the rendering step leverages aspects of a particular engine, such as Unreal Engine™, making use of Hierarchical Instanced Static Meshes (HISMs), writing the RenderTargets via CPU, vertex shaders having access to said RenderTarget, and accessible replication APIs that allow fast replication of a large amount of data to clients. Analogous features exist in other engines, such as GPU instancing in Unity™, and a person skilled in the art would appreciate that the general principles described herein are applicable across different engines, and that analogous engine-specific features can be adapted to achieve similar results, and as such, the disclosure herein is not intended to be limiting to any particular engine.

[0031] The rendering step involves drawing by a GPU, each entity tracked in the render target image, in a single pass - namely, the position and rotation of each bone in each skeleton of each of the entities is known and accessed simultaneously, allowing multiple entities to be drawn on the screen as if they were part of a single, larger entity. The GPU retrieves and determines position and rotation data associated with the predefined bones of each tracked entity, and poses a predefined source mesh accordingly based on positioning and rotating the skeleton according to the position and rotation of each associated predefined bone as represented in the render target image in order. This allows the GPU to draw each entity collectively in a single pass and increases rendering efficiency by eliminating the need to process and render each entity individually, while retaining the ability to individually pose, and interact with each entity individually.

[0032] An example embodiment of retrieving the rotation data stored in the render target image then performing transformations is shown at 500 in FIG.5. The first step 502 involves conversion of the quaternion components stored in the render target image into a 3x3 matrix. An example code snippet for carrying out the step 502 of FIG.5 is shown below: / / Converts our Quat rotations to a 3x3 Matrix float3x3 QuatToMatrix ( f loat4 quat) { const float xx = quat.x * quat.x; const float yy = quat.y * quat.y; const float zz = quat . z * quat.z; const float xy = quat.x * quat.y; const float xz = quat.x * quat.z; const float yz = quat.y * quat.z; const float wx = quat.w * quat.x; const float wy = quat.w * quat.y; const float wz = quat.w * quat.z; return float3x3 ( l.Of - 2. Of (yy + zz) , 2. Of * (xy +wz ) , 2. Of * (xz - wy) , 2. Of * (xy - wz) , l.Of - 2. Of * (xx + zz) , 2. Of * (yz + wx) , 2. Of * (xz + wy) , 2. Of * (yz - wx) , l.Of -2. Of * (xx + yy) ) ;1

[0033] Once converted into a matrix, any position or normal of a bone corresponding to the index can be easily transformed through matrix multiplication 504, as shown in FIG. 5. In the shader, the render target image is sampled using the instance identifier and the current bone identifier, per vertex. Accordingly, the render target image is only sampled twice per skin weight - once to extract the location (and health), and again to extract the rotation quaternion. This process is then repeated per each skin weight as shown in the method 600of FIG. 6. An example code snippet for carrying out the method 600 as shown in FIG. 6 is shown below: / / Sample the Pose Texture , us ing our I nstance l D and Bone lD for S kinweightO for given vertex . const float4 SampleA = PoseData . Load ( uint3 ( Instance ID , Bone lD , 0 ) ) ; const float4 SampleB = PoseData . Load (uint3 ( Instance lD , Bone lD + NumBones , 0 ) ) ; / / Extract info from texture const float3 Pos O = SampleA . yzw; const float4 QuotO = SampleB . xyzw; Health = SampleA . x ;

[0034] For sampling additional levels of detail (LoDs), the same process can be repeated, adding the next weighted bone index from the UV channels. In some embodiments, only one skin weight is sampled instead of two, reducing the number of texture lookups from 4 to 2 for lower levels of detail. This further reduces computational cost, at the cost of an acceptable loss in visual fidelity as the entities are more distant.

[0035] In some embodiments, additional features may be included in the rendering step. For example, in an embodiment, the individual identifier of each entity in the group of entities may be hashed to generate a pseudo-random number, which can be used to further differentiate entities such as by assigning shader effects based on the pseudo-random number. In some embodiments, a time stamp can be associated with each entity in the group of entities in order to track the time elapsed, as an increment over every frame, since the entity was added to the group of entities and represented in the render target image, which can be used to determining time-based effects. In an illustrative example, the time stamp can be associate with a time of death of the entity, allowing various age-dependent effects to be applied to the corpse such as changes in color or animation, for example.

[0036] While designed for quick and efficient tracking of dynamic entities which are generated during the course of gameplay, the system and method disclosed herein is not limited to anyone particular scenario, and are widely applicable to many aspects of video game design. In other exemplary embodiments the method for efficient tracking and rendering can be applied to other game entities and at other stages of the game process. In some embodiments, the method can be used for pre-placed environmental assets, allowing for placement of terrainconforming, interactable elements within the virtual game space which are present upon initiation of the game instance. Such embodiments provide level designers with the ability to place efficient posable environmental assets which conform other environmental assets and terrain, without the need to engage artists to generate a new pre-rendered asset. As an additional benefit over pre-rendered assets which have a higher development and computational cost, or are non-interactive, these posable environmental assets improve immersion and interactivity, and provide new storytelling opportunities as such assets are dynamic, interactable and customizable while remaining computationally efficient.

[0037] In some embodiments, such as described above in relation to FIG. 1, a health value is stored in association with each predefined bone. In some such embodiments, additional data may be associated with each mesh which defines properties of how the health value represents durability. In such embodiments, the health value may be more or less affected by certain types of interaction or damage, where damage types representing explosives or burning may be more effective at depleting the health (and thus the durability) than other forms of damage like bunt damage or piercing damage from bullets, for example.

[0038] Additionally, the system and method for efficient tracking and rendering are beneficial in online multiplayer environments. By simplifying and reducing the tracked entities, it provides benefits to the efficiency of server-client interaction over computer networks, as less data needs to be tracked to pose each entity as all entities can be re-created across multiple clients synchronously from a single render target image which serves as a blueprint for the game engine to create and pose all the entities - each client can be provided the same data, such that each client can render the same group of entities. Furthermore, the data can be compressed, and only changes to the render target image need be communicated.

[0039] The system and method are designed to be implementable as a plugin, and the benefits are realizable in any interactive application where there are repeated instances of entities sharing skeletal data. The parameters associated with entities, such as position, rotation, durability,and age are common across many video game applications, and a person skilled in the art would be able to customize the flexible implementation specifics of the method to suit the specific needs of a particular application as illustrated in some of the exemplary embodiments described herein without deviating from the invention as disclosed. Various parameters are designed to be accessible without the need to customize the system or method, by providing various exposed call functions to adapt the behavior of corpses. In one illustrative example embodiment, when a character dies and a corpse is created, an identifier, a time of creation, and replicated data (bones, transforms, how it was destroyed) can be stored in various data structures of the system and method. Similarly, when said corpse is damaged, bone names, damage values, and the previous and new corpse health can be tracked and passed. Likewise, when a corpse is removed from the group of entities, analogous information to those when the corpse was first created (ID, time of removal, and replicated data) is made available. As described above, the data structures of the system and method are also able to store extra bone data. In some embodiments, for example, this provides the ability to change the health of each bone, such as for visualizing damage per limb on a corpse, and even for destruction of parts of the corpse (limbs blown off or gibbing) in certain embodiments.

[0040] In an illustrative example of the method 400 in practice, in a game scene is shown in FIG. 7 and FIG 8, there may be a plurality of active character entities, in this case, insectoid enemies or bugs 702. A player character 704 may interact with the bugs 702, such as by shooting at them, which depletes their health. When the health of a bug 702 is reduced to zero, it dies, leaving a corpse. Upon death, the method 400 adds the position and rotation of the skeleton of the bug 702, and tracks it as it deforms and animates under ragdoll physics, and becomes uniquely posed due to its orientation and interaction with other bugs 702. As shown in FIG. 7, the time required 704 for a device to display the plurality of bugs 702 using traditional rendering methods is shown on the upper right corner, along with the frame rate 706. As shown in FIG. 7, it takes the device 66.54 milliseconds to prepare and display a single frame, resulting in a frame rate of 15.03 frames per second. A typical target frame rate for a videogame is 60 or 120 frames per second in order to avoid perceivable stutter or jerkiness, and a framerate of approximately 15 frames per second will have a typically unacceptabledegree of perceptible stutter. As shown in FIG. 8, a roughly similar number of dead bugs 802 is displayed by the same device as in FIG. 7, using the method 400 instead of traditional rendering methods. As shown in the top left corner of FIG. 8, the time required 804 for the device to draw a frame is only 8.65 milliseconds to prepare and display a single frame, resulting in a frame rate 806 of 115.57 frames per second. This frame rate is within the typical target frame rate range of 60 to 120 frames per second, and would be perceived as smooth. Testing has shown that increasing the number of bugs 802 rendered using the method 400 does not appreciably affect the frame rate.

[0041] While the foregoing is directed to aspects of the present disclosure, other and further aspects of the disclosure can be devised without departing from the basic scope thereof.

Claims

WHAT IS CLAIMED IS:

1. A method for efficient tracking and rendering, comprising: grouping a plurality of similar entities to be rendered, defining assets for the group plurality of similar entities according to a first data structure having reference points with an associated 3 -dimensional geometry, storing position and rotation information of each reference point of each entity in the plurality of entities as a pixel on a render target image according to a second data structure, and rendering the plurality of similar entities by using the render target image of the second data structure to define the position and orientation of the associated 3- dimensional geometry associated with each reference point of each entity of the plurality of similar entities of the first data structure.

2. The method of claim 1, wherein the reference points of the first data structure are bones, and the associated 3-dimensional geometry is a mesh comprising a plurality of polygons, the bones determining the position, orientation of the polygons.

3. The method of claim 2, wherein the position and orientation information of each bone is stored as indexed pixels in the render target image of the second data structure, the render target image comprising a first section and a second section, wherein the positions are stored as color channels in the first section, and the rotations are stored as color channels in the second section.

4. The method of claim 3, where in the orientation is stored in the second data structure as rotational data in quaternions.

5. The method of claim 4, wherein rendering comprises retrieving the rotation information from the render target image of the second data structure, converting the quaternion into a 3x3 matrix, and transforming the matrix through matrix multiplication.

6. The method of claim 3 or claim 4, wherein the positions are stored in the second data structure as cartesian coordinates.

7. The method of claim 6, wherein the color channels of the pixels of the first section of the render target image additional stores an additional value in the second data structure.

8. The method of any one of claims 2 to 7, wherein defining assets in the first data structure comprises associating a set of polygons from the plurality of polygons with a bone in the plurality bones such that each polygon is associated with a bone.

9. The method of claim 8, wherein the defining assets in the first data structure further comprises defining a hierarchy of bones, each bone being associated with at least one other bone, and associating each vertex of each polygon with at least two bones, the bone associated with the polygon, and at least one other bone adjacent in the hierarchy.

10. The method of claim 9, wherein defining assets in the first data structure further comprises storing a degree of association between each vertex and its two most closely associated bones as colors in UV channels of the vertex.

11. The method of any one of claims 1 to 10, where in grouping the plurality of similar entities and defining assets according to the first data structure are performed by a processor.

12. The method of any one of claims 1 to 11, wherein rendering the plurality of similar entities is performed by a graphics processing unit.

13. The method of claim 12, wherein the graphics processing unit accesses the stored position and rotation information for each reference point of each entity of the plurality of entities according to the second data structure, and renders the plurality of entities collectively in a single rendering pass.

14. A system for efficient tracking and rendering, comprising: a first data structure for associating the render target image data structure with defined bones of a skeleton, comprising a skeleton data structure and a render target image data structure; a second data structure for associating polygons of a 3 -dimensional mesh with bones of the skeleton comprising a mesh data structure; a computer-readable memory configured to store information according to the first and second data structures; and a processor connected to the memory configured to generate an image based on information stored according to the first and second data structures.

15. The system of claim 14, wherein the skeleton data structure defines an arrangement of bones, replication behavior, and collision of a skeleton.

16. The system of claim 14 or 15, wherein the first data structure stores information regarding defined bones of a plurality skeletons in a 2-dimensional matrix, with successive bones of a skeleton extending in a first direction in the matrix, and analogous bones successive skeletons of the plurality of skeletons extending in a second direction in the matrix perpendicular to the first.

17. The system of claim 16, wherein the matrix corresponds to pixels of the render target image data structure, and vector components of the position and the rotation of bones of the skeleton are stored as values in color channels of a pixel.

18. The system of any one of claims 14-17, wherein the mesh data structure defines a mesh comprising a plurality of polygons arranged in 3 -dimensional space, and the relationship between portions of the mesh and the skeleton.

19. The system of any one of claims 14-18, further comprising a third data structure for storing further information on deformation of the 3 -dimensional mesh when the bones of the skeleton are moved, wherein the processor is further configured to store information according to the third data structure in the memory, and to generate an image based on information stored in the first, second and third data structures.

20. The system of any one of claims 14-19, wherein the processor comprises a first processor configured to store information according to the first data structure, and a second processor configured to store information according to the second data structure and to generate an image based on information stored according to the first and second data structures.

21. The system of claim 20, wherein the second processor is a graphics processing unit.

22. The system of claim 21, wherein the second processor is configured to generate the image based on information stored according to the first and second data structures in a single pass.

Images