Capturing computer game output in rendering process to access 2d to 3D conversion, access and other

Abstract

Once a complete image is made up, important information related to image generation, such as computer game video generation, may be lost. To avoid this, a customized capture script (600) is created for a computer game to specify (502) what is captured (504) for future use during rendering of game video frames, such as a frame buffer (400), a depth buffer (402), a template buffer (404), etc. The script also specifies when to record data during frame rendering. By capturing data at an intermediate frame, the relative distance between object boundaries and objects on the screen can be inferred. This information may be used (506) for 2D to 3D conversion, object coloring to increase contrast, object highlighting, and the like.

Description

Technical Field

[0001] This application generally relates to capturing the output of a computer game during the rendering process to achieve 2D to 3D conversion, accessibility, and other effects. Background Art

[0002] As understood herein, when rendering video frames for a computer simulation (such as a computer game), multiple layers can be rendered, and some data buffers are reused before the composition of the entire frame is completed. For example, a color buffer can be used to render a layer of an object's shape, a depth buffer can be used to render a layer representing the depth in the frame, and then multiple other layers, including lighting layers, transparent layers, opaque layers, etc., can be rendered by reusing the color and depth buffers. Summary of the Invention

[0003] As further understood herein, if the operating system cannot access a computer game running in the O.S. except for the commands to render video frames issued by the game to the graphics processing unit (GPU), information that may be valuable for the purposes discussed herein may be lost due to the reuse of buffers to render multiple frame layers.

[0004] Accordingly, an apparatus includes at least one computer storage device that is not a transitory signal and thus includes instructions that are executable by at least one processor to compose a first video frame of a computer simulation by at least partially rendering a first video layer using a first data buffer and rendering a second video layer using the first data buffer, and capturing data in the first data buffer during the rendering process before the composition of the first video frame is completed. The instructions are executable to change the appearance of at least one object associated with the computer simulation using the data captured from the first data buffer.

[0005] In an example implementation, the object is rendered in two dimensions (2D), the first data buffer includes a depth data buffer, and the instructions are executable to change the appearance of the object by at least partially rendering the object in 3D in at least one frame after the first frame and / or in the first frame.

[0006] In other examples, the object is rendered in two dimensions (2D), the first data buffer includes a depth data buffer, and the instructions are executable to change the appearance of the object by at least partially rendering at least one object in 3D in at least one frame of a remade version of the computer simulation.

[0007] In some implementations, an object is rendered in a first color in a first frame, a first data buffer includes a color data buffer, and the instructions are executable to change the appearance of the object by presenting the object in at least one frame after the first frame and / or in a second color different from the first color in the first frame.

[0008] In other implementations, an object is rendered in a first color in a first frame, a first data buffer includes a color data buffer, and the instructions are executable to change the appearance of the object by presenting the object in a second color different from the first color in at least one frame of a replicated version of the computer simulation.

[0009] In an example implementation, the instructions are executable to change the appearance of the object by highlighting the object in at least one frame after the first frame. Note that additionally or alternatively, the first frame itself may be modified and altered / replicated before display.

[0010] In other example implementations, the instructions are executable to change the appearance of the object by highlighting the object in at least one frame of a replicated version of the computer simulation.

[0011] If desired, a script may be used to identify the first frame and / or the first data buffer for data capture.

[0012] In another aspect, a device includes at least one processor programmed with instructions to render at least one video frame of a computer simulation. The instructions are executable to record first data from at least one buffer for rendering of the frame during the rendering process such that the data is recorded before completion of the composition of the frame and before the first data is replaced with second data during the composition of the frame.

[0013] In another aspect, a method includes: copying together video data representing a first video layer of a plurality of video layers to create a single video frame of a computer simulation, the single frame including at least one object; and using the video data to change the appearance of at least one object in the video simulation.

[0014] Details regarding both the structure and operation of the present application may best be understood with reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of an example system in accordance with the principles of the present invention;

[0016] Figure 2 shows a simplified architecture;

[0017] Figure 3 Schematically shows multi-layer video data for composing a single complete video frame;

[0018] Figure 4 Shows an architecture with multiple data buffers for rendering video frames;

[0019] Figure 5 Shows example logic consistent with the principles of the present invention in an example flowchart format;

[0020] Figure 6 Shows an example buffer capture script consistent with the principles of the present invention;

[0021] Figure 7 Shows a screenshot of a rendered frame from a traditional computer game;

[0022] Figure 8 Shows relative to Figure 7 A screenshot of a rendered frame that has been enhanced to use mid-render buffer capture to present foreground objects in three-dimensional (3D) form;

[0023] Figure 9 Shows relative to Figure 7 A screenshot of a rendered frame that has been enhanced to use mid-render buffer capture to present objects in more distinguishable colors; and

[0024] Figure 10 Shows relative to Figure 7 A screenshot of a rendered frame that has been enhanced to use mid-render buffer capture to highlight objects. Detailed Description

[0025] The present disclosure generally relates to computer ecosystems, which include aspects of a network of consumer electronics (CE) devices, such as but not limited to computer game networks. Systems herein may include server components and client components, which may be connected via a network such that data can be exchanged between the client components and the server components. The client components may include one or more computing devices, the one or more computing devices including game consoles (such as Sony or a game console made by Microsoft, Nintendo, or other manufacturers), extended reality (XR) headsets (such as virtual reality (VR) headsets, augmented reality (AR) headsets), portable televisions (e.g., smart TVs, Internet-enabled TVs), portable computers (such as laptops and tablets), and other mobile devices (including smartphones and additional examples discussed below). These client devices can operate in a variety of operating environments. For example, some client computers can employ, for example, the Linux operating system, an operating system from Microsoft, or the Unix operating system, or an operating system produced by Apple, Inc., Google, or the Berkeley Software Distribution or Berkeley Standard Distribution (BSD) OS (including descendants of BSD). These operating environments can be used to execute one or more browser programs, such as browsers made by Microsoft, Google, or Mozilla, or other browser programs that can access websites hosted by the Internet servers discussed below. Additionally, one or more computer game programs can be executed using an operating environment in accordance with the principles of the present invention.

[0026] Servers and / or gateways can be used, which can include one or more processors that execute instructions to configure the server to receive and transmit data over a network such as the Internet. Alternatively, the client and server can be connected via a local intranet or a virtual private network. The server or controller can be instantiated by a game console (such as Sony personal computers, etc.).

[0027] Information can be exchanged between the client and the server over the network. To this end, and for security reasons, the server and / or client can include a firewall, a load balancer, temporary storage devices, and a proxy, as well as other network infrastructure for reliability and security. One or more servers can form a device that implements a method for providing a secure community (such as an online social networking site or a gamer network) to network members.

[0028] The processor can be a single-chip or multi-chip processor that can perform logic by means of various lines (such as address lines, data lines, and control lines), as well as registers and shift registers. A processor that includes a digital signal processor (DSP) can be one implementation of the circuitry.

[0029] The components included in one embodiment can be used in other embodiments in any suitable combination. For example, any of the various components described herein and / or depicted in the figures can be combined, interchanged, or excluded from other embodiments. A "system having at least one of A, B, and C" (similarly, "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes the following systems: having only A; having only B; having only C; having both A and B; having both A and C; having both B and C; and / or having A, B, and C.

[0030] Now referring Figure 1 , an example system 10 is shown, and the example system can include one or more of the example devices mentioned above and further described below in accordance with the principles of the present invention. The first device among the example devices included in system 10 is a consumer electronics (CE) device, such as an audio-video device (AVD) 12, such as but not limited to a theater display system (which can be projector-based) or an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). Alternatively, the AVD 12 can also be a computerized Internet-enabled ("smart") phone, a tablet computer, a laptop computer, a head-mounted device (HMD), and / or a head-mounted headset (such as smart glasses or a VR headset), another wearable computerized device, a computerized Internet-enabled music player, a computerized Internet-enabled headset, a computerized Internet-enabled implantable device (such as an implantable skin device), etc. In any case, it should be understood that the AVD 12 is configured to implement the principles of the present invention (e.g., communicate with other CE devices to implement the principles of the present invention, execute the logic described herein, and perform any other functions and / or operations described herein).

[0031] Therefore, in order to implement these principles, the AVD 12 can be established by some or all of the components shown. For example, the AVD 12 can include one or more touch displays 14, which can be implemented by a high-definition or ultra-high-definition "4K" or higher-definition flat screen. The touch display 14 can include, for example, a capacitive or resistive touch-sensing layer having an electrode grid for touch sensing consistent with the principles of the present invention.

[0032] The AVD 12 may also include: one or more speakers 16 for outputting audio in accordance with the principles of the present invention; and at least one additional input device 18 (such as an audio receiver / microphone) for inputting audible commands to the AVD 12 to control the AVD 12. The exemplary AVD 12 may also include one or more network interfaces 20 to communicate via at least one network 22 (such as the Internet, WAN, LAN, etc.) under the control of one or more processors 24. Thus, the interface 20 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that the processor 24 controls the AVD 12 to implement the principles of the present invention, including controlling other elements of the AVD 12 described herein, such as controlling the display 14 to present an image on the display and receiving input from the display. Additionally, note that the network interface 20 may be a wired or wireless modem or router, or other suitable interface, such as a wireless telephone transceiver or a Wi-Fi transceiver as mentioned above, etc.

[0033] In addition to the foregoing, the AVD 12 may also include one or more input and / or output ports 26, such as a high-definition multimedia interface (HDMI) port or a universal serial bus (USB) port physically connected to another CE device and / or a headphone port for connecting headphones to the AVD 12 to present audio from the AVD 12 to the user through the headphones. For example, the input port 26 may be connected, either wired or wirelessly, to a wired or satellite source 26a of audio-video content. Thus, the source 26a may be a separate or integrated set-top box, or a satellite receiver. Alternatively, the source 26a may be a game console or a disk player containing content. When the source 26a is implemented as a game console, it may include some or all of the components described below with respect to the CE device 48.

[0034] The AVD 12 may also include one or more computer memories / computer-readable storage media 28 that are not transient signals, such as disk-based storage devices or solid-state storage devices, which in some cases are embodied as stand-alone devices within the housing of the AVD, or as a personal video recording device (PVR) or video disk player for playing back AV programs inside or outside the housing of the AVD, or as a removable memory medium or a server as described below. Additionally, in some embodiments, the AVD 12 may include a location or positioning receiver, such as, but not limited to, a cellular phone receiver, a GPS receiver, and / or an altimeter 30, which is configured to receive geographical location information from satellites or cellular phone base stations and provide the information to the processor 24 and / or to determine, in conjunction with the processor 24, the height at which the AVD 12 is set.

[0035] Continuing the description of the AVD 12, in some embodiments, the AVD 12 may include one or more cameras 32, and the one or more cameras may be thermal imaging cameras, digital cameras (such as webcams, IR sensors, event-based sensors), and / or cameras integrated into the AVD 12 and capable of being controlled by the processor 24 to acquire pictures / images and / or videos according to the principles of the present invention. The AVD 12 may also include a transceiver 34 and other near field communication (NFC) elements 36 to communicate with other devices using Bluetooth and / or NFC technologies, respectively. Example NFC elements may be radio frequency identification (RFID) elements.

[0036] In addition, the AVD 12 may include one or more auxiliary sensors 38 that provide input to the processor 24. For example, one or more of the auxiliary sensors 38 may include one or more pressure sensors that form a layer of the touch display screen 14 itself, and may be, but are not limited to, piezoelectric pressure sensors, capacitive pressure sensors, piezoresistive strain gauges, optical pressure sensors, electromagnetic pressure sensors, etc. Other sensor examples include pressure sensors, motion sensors (such as accelerometers, gyroscopes, odometers, or magnetic sensors), infrared (IR) sensors, optical sensors, speed and / or rhythm sensors, event-based sensors, gesture sensors (e.g., for sensing gesture commands). Thus, the sensor 38 may be implemented by one or more motion sensors, such as a separate accelerometer, gyroscope, and magnetometer and / or an inertial measurement unit (IMU), which typically includes a combination of an accelerometer, gyroscope, and magnetometer to determine the position and orientation of the AVD 12 in three dimensions, or by an event-based sensor (such as an event detection sensor (EDS)). The EDS consistent with the present disclosure provides an output indicating a change in the light intensity sensed by at least one pixel of the light sensing array. For example, if the light sensed by the pixel is decreasing, the output of the EDS may be -1; if the light sensed by the pixel is increasing, the output of the EDS may be +1. An output binary signal 0 may indicate no light intensity change below a specific threshold.

[0037] The AVD 12 may also include a wireless TV broadcast port 40 that provides input to the processor 24 for receiving OTA TV broadcasts. In addition to the foregoing, it should be noted that the AVD 12 may also include an infrared (IR) transmitter and / or IR receiver and / or IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, such as a kinetic energy harvester that can convert kinetic energy into electricity to charge the battery and / or power the AVD 12. A graphics processing unit (GPU) 44 and a field programmable gate array 46 may also be included. One or more haptic / vibration generators 47 may be provided to generate haptic signals that can be sensed by a person holding or touching the device. Thus, the haptic generator 47 may use an electric motor to vibrate all or part of the AVD 12, and the electric motor is connected to an eccentric and / or unbalanced counterweight via a rotatable shaft of the motor, such that the shaft can rotate under the control of the motor (and the electric motor can in turn be controlled by a processor, such as the processor 24) to generate vibrations of various frequencies and / or amplitudes and force simulations in various directions.

[0038] A light source, such as a projector, such as an infrared (IR) projector, may also be included.

[0039] In addition to the AVD 12, the system 10 may also include one or more other types of CE devices. In one example, the first CE device 48 may be a computer game console that can be used to send the audio and video of a computer game to the AVD 12 via commands sent directly to the AVD 12 and / or via a server as described below, while the second CE device 50 may include components similar to those of the first CE device 48. In the example shown, the second CE device 50 may be configured as a computer game controller manipulated by a player or a head-mounted display (HMD) worn by a player. The HMD may include a head-up transparent or opaque display for presenting AR / MR content or VR content (more generally, extended reality (XR) content) respectively. The HMD may be configured as a glasses-type display or as a larger YR-type display sold by a computer game device manufacturer.

[0040] In the example shown, only two CE devices are shown, and it should be understood that fewer or more devices may be used. The devices herein may implement some or all of the components shown for the AVD 12. Any of the components shown in the following figures may be combined with some or all of the components shown in the case of the AVD 12.

[0041] Now referring to the at least one server 52 mentioned previously, the at least one server includes at least one server processor 54, at least one tangible computer-readable storage medium 56 (such as a disk-based storage device or a solid-state storage device), and at least one network interface 58. The at least one network interface, under the control of the server processor 54, allows communication with other shown devices via the network 22 and can in fact facilitate communication between the server and client devices in accordance with the principles of the present invention. Note that the network interface 58 can be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface (such as, for example, a wireless telephone transceiver).

[0042] Thus, in some embodiments, the server 52 can be an Internet server or an entire server "farm", and in an exemplary embodiment for, e.g., a network gaming application, the server can include and execute "cloud" functions such that the devices of the system 10 can access the "cloud" environment via the server 52. Alternatively, the server 52 can be implemented by one or more game consoles or other computers in the same room as or near the other shown devices.

[0043] The components shown in the following figures can include some or all of the components shown herein. Any user interface (UI) described herein can be combined and / or extended, and UI elements can be mixed and matched between UIs.

[0044] The principles of the present invention can employ various machine learning models, including deep learning models. Machine learning models according to the principles of the present invention can use various algorithms trained in ways that include the following: supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that can be implemented by computer circuitry include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and a type of RNN called long short-term memory (LSTM) networks. Support vector machines (SVMs) and Bayesian networks can also be considered examples of machine learning models. In addition to the network types mentioned above, the models herein can also be implemented by classifiers.

[0045] As understood herein, performing machine learning can thus involve accessing training data and then training a model based on the training data so that the model can process additional data to make inferences. Thus, an artificial neural network / artificial intelligence model trained by machine learning can include an input layer, an output layer, and a plurality of hidden layers therebetween, the plurality of hidden layers being configured and weighted to make inferences about an appropriate output.

[0046] Now referring to Figure 2, a computer simulation engine (such as a traditional computer game engine 200) issues commands to one or more GPUs 202 to render the required simulation video frame by frame. The operating system 204 running the game engine can detect these commands when they are sent to the GPU, but the traditional game engine itself may not convey its rendering scheme (except for the commands sent to the GPU) to the O.S. 204. Therefore, the O.S. 204 can access an information program (such as a script built with an understanding of the rendering scheme) to inform the O.S. when and where to capture data during the rendering process of one or more frames. "Capture" means recording or copying the data representing the frame elements, which the author of the script may wish to use to enhance one or more objects in the simulation to make them more visible to some people, or to enhance their appearance in other ways described herein.

[0047] Figure 3 A single simulation video frame 300 rendered by multiple sequentially rendered frame layers 302 (only four frame layers are shown for illustration) is shown. One frame layer may represent depth information, one frame layer may represent color information, one or more layers may represent lighting information, one or more layers may represent transparency information, one or more layers may be used for opaque geometry information (such as smoke), and so on.

[0048] Therefore, in some cases, information may be scattered across multiple layers. For example, the final synthesized frame may consist of parts from different layers. The scripts discussed elsewhere in this document may specify this explicitly. Ultimately, the script will attempt to trace the origin of what is presented on the screen.

[0049] Also note that parts of each layer can move relative to each other, such that the script can specify that one particular layer appears on one part of the screen while another layer appears on another part of the screen. As an example, if heads-up display (HUD) elements are rendered separately in a small buffer, they may only appear in one corner of the screen.

[0050] As understood herein, in such layer-by-layer rendering, important information related to image generation may be lost when the complete image frame 300 is fully composed. To avoid this, as discussed herein, custom capture scripts or other tools are created for computer games to specify what to capture during the rendering of game video frames for future use, such as data in the frame buffer, data in the depth buffer, data in the stencil buffer, etc. The script also specifies when to record the data during frame rendering. By capturing data at intermediate frames, the object boundaries on the screen and the relative distances between objects can be inferred. This information can be used for 2D to 3D conversion, object shading to increase contrast, object highlighting, etc.

[0051] One problem addressed by the present disclosure is that when rendering a single frame with multiple layers, the same buffer can be used multiple times. For example, a color buffer can be used to render objects in color in one layer, but then the data is flushed and reused to render an illumination layer. If the color buffer is inspected after frame composition, in the absence of the principles of the present invention, it is not always clear what data is actually being inspected. The same is true for a buffer nominally called a "depth" buffer, which may not have any depth-related data at all when frame composition is complete.

[0052] Figure 3 Specific examples of forward renderers are essentially described. It should be understood that the principles of the present invention also apply to deferred rendering. In such an implementation, each layer may contain "buffers" with material information, depth, and other data.

[0053] Figure 4 More details are shown. When rendering video, the GPU 202 can access data buffers, including for example a color buffer 400, a depth buffer 402, and a stencil buffer 404. The data in the buffers is written to a rendering engine 406 for presentation on a display 408 (such as any display herein).

[0054] Figure 5 An example general logic is shown. Starting at block 500, it is determined how a particular conventional game engine composes its frames based on what layers it uses for each frame and what buffers it uses for each layer. This determination can be made by accessing the software of the game or by observing the game commands sent to the GPU in a laboratory environment and then identifying what those commands are.

[0055] Moving to block 502, it is determined what data in the frame render is of interest, such as depth data, color data, or other data useful for the designer's intended purpose. Once this information is compiled and reflected in, for example, a script or other format, it is provided to Figure 2 the O.S. 204 in to enable the O.S. to capture the desired data from the appropriate buffer(s) during the rendering of one or more frames by "buffer grabbing" (copying the data in the buffer or recording it before the data is flushed for use in rendering another layer of the frame with the buffer the next time).

[0056] Proceed to block 506, where the data captured in block 504 is used to enhance the image, e.g., by changing the appearance of at least one object in a computer simulation. This may be for accessibility reasons, such as changing the color of an object to a different color that is more distinguishable to some people, or by highlighting the object to draw attention. Alternatively, an object rendered in 2D in a traditional game can be rendered in 3D in a remastered version of that game. These changes may be made dynamically, i.e., for subsequent frames when the traditional game is presented, or offline in a remastered version of the game, and the changes are typically not made at the game level, but at the O.S. level.

[0057] In another embodiment, a highlighted version of a frame can be presented to the audience of the game, or certain scenes can be replayed in a newly introduced "replay mode" or highlight reel. For example, it can highlight which enemy shot the player in a gunfight, or highlight the football player about to sack the quarterback in a sports game.

[0058] Note that in addition to the above data, data can also be captured from memory as described by a script. Such additional data can be prepared by the CPU.

[0059] Figure 6 An example script 600 is shown, which can receive commands from the game engine 200 to the GPU 202 in Figure 2 as input (first, second, third draw calls, as a simple example). The script knows which layer is currently being drawn, whether it is an opaque layer, a depth layer, a color layer, a lighting layer, etc., and in which buffer the data for the layer is located, such that when the buffer where the data is located is valid for the desired data, the desired data (such as color data or depth data) can be captured.

[0060] Thus, the script 600 in the non - limiting example shown can include lines 602, each of which can correspond to a single layer out of multiple layers in a single frame. The name of the layer can also be included. Commands 604 can occur to cause at least one of the layers to capture data from a specific buffer (such as a color buffer or a depth buffer) before rendering the next layer. An indicator 606 can be provided after the last layer to indicate that the composition of the frame is complete once the last layer has been rendered.

[0061] During execution, by Figure 2The script 600 executed by O.S.204 in [description] can check the command codes passed from the game engine 200 to the GPU 202, determine whether the commands are object-related, and when these commands stop, whether the subsequent commands are related to the next layer being drawn. This is advantageous because, as understood herein, the number of layers used to compose a complete frame may change during the game, such as when simulated light is destroyed, and the associated lighting layer is thus no longer drawn in the rendered frame.

[0062] Furthermore, with respect to the script, frustum culling can be facilitated. Frustum culling refers to a situation where the depth values may be accurate with respect to object separation but may not necessarily correspond to the accurate depth for 3D reconstruction. The script can describe how to generate accurate depth values therefrom.

[0063] To this end, the projection matrix can be modified. More generally, in some techniques, such as frustum culling, the depth values stored in the depth buffer are changed, and the script provides a method to restore the depth values for 3D reconstruction.

[0064] Figure 7 A screenshot 700 of a rendered frame from a traditional computer game that can be presented on any display discussed herein, for example, is shown. All objects 702 in the non-limiting example shown are 2D-rendered by game engine 200 commands. Specifically, the foreground object 704 can be rendered in 2D and a first color and can overlap with another object, as shown.

[0065] Now consider Figure 8 , which shows a screenshot of the rendered frame enhanced relative to Figure 7 to present the foreground object 704 in three-dimensional (3D) form using buffer captures during the rendering process from the depth buffer, the data of which indicates that the foreground object 704 is in front of the other objects 702.

[0066] Further consider Figure 9 , which shows a screenshot of the rendered frame enhanced relative to Figure 7 to present an object (in the example shown, the foreground object 704) in a more distinguishable color using buffer captures during the rendering process, in this case, by capturing the color buffer while the color is still valid in the color buffer and then overlaying it with data for subsequent layers of the rendered frame.

[0067] In addition, Figure 10 shows a screenshot relative to Figure 7Screenshot of the enhanced rendered frame to highlight an object (foreground object 704 in the example shown) using buffer capture during rendering as indicated at 1000, in this case by capturing the color buffer while the colors are still valid in the color buffer and then overlaying with data from subsequent layers used to render the frame.

[0068] Although specific embodiments have been shown and described in detail herein, it should be understood that the subject matter covered by the present invention is limited only by the claims.

Claims

1. A device, which comprises: at least one computer storage device, which is not a transient signal and includes instructions that can be executed by at least one processor to: (a) Compose a computer-simulated first video frame at least in part by: Rendering a first video layer using a first data buffer; Rendering a second video layer using the first data buffer; and Capturing data in the first data buffer during the rendering process before completing the composition of the first video frame; and (b) Using the data captured from the first data buffer to change the appearance of at least one object associated with the computer simulation.

2. The device according to claim 1, wherein the at least one object is rendered in two dimensions (2D), the first data buffer includes a depth data buffer, and the instructions can be executed to change the appearance of the at least one object at least in part by: Rendering the at least one object in 3D in at least one frame after the first frame and / or in the first frame.

3. The device according to claim 1, wherein the at least one object is rendered in two dimensions (2D), the first data buffer includes a depth data buffer, and the instructions can be executed to change the appearance of the at least one object at least in part by: Rendering the at least one object in 3D in at least one frame of a reproduced version of the computer simulation.

4. The device according to claim 1, wherein the at least one object is rendered in a first color in the first frame, the first data buffer includes a color data buffer, and the instructions can be executed to change the appearance of the at least one object at least in part by: Presenting the object in a second color different from the first color in at least one frame after the first frame and / or in the first frame.

5. The device according to claim 1, wherein the at least one object is rendered in a first color in the first frame, the first data buffer includes a color data buffer, and the instructions can be executed to change the appearance of the at least one object at least in part by: Presenting the object in a second color different from the first color in at least one frame of a reproduced version of the computer simulation.

6. The device according to claim 1, wherein the instructions can be executed to change the appearance of the at least one object at least in part by: Highlighting the object in at least one frame after the first frame and / or in the first frame.

7. The device according to claim 1, wherein the instructions can be executed to change the appearance of the at least one object at least in part by: Highlighting the object in at least one frame of a reproduced version of the computer simulation.

8. The device according to claim 1, wherein the instructions can be executed to: Use a script to identify at least one of the first frame and / or the first data buffer for capturing data.

9. The apparatus according to claim 1, wherein the instructions are executable to: Use a script to identify both the first frame and / or the first data buffer for capturing data.

10. The apparatus according to claim 1, comprising the at least one processor.

11. An apparatus, which Comprises: At least one processor, the at least one processor being programmed with instructions to: Render at least one video frame of a computer simulation; During the rendering process, record first data from at least one buffer for the rendering of the frame such that the data is recorded before the composition of the frame is completed and before the first data is replaced by second data during the composition of the frame.

12. The apparatus according to claim 11, wherein the apparatus comprises at least one computer, the at least one computer being programmed with instructions to use the first data to change the appearance of at least one object associated with the computer simulation.

13. The apparatus according to claim 12, wherein the at least one object is rendered in two dimensions (2D), the first data buffer comprises a depth data buffer, and the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Rendering the at least one object in three dimensions (3D) in at least one frame after the first frame and / or in the first frame.

14. The apparatus according to claim 12, wherein the at least one object is rendered in two dimensions (2D), the first data buffer comprises a depth data buffer, and the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Rendering the at least one object in three dimensions (3D) in at least one frame of a reproduced version of the computer simulation.

15. The apparatus according to claim 12, wherein the at least one object is rendered in a first color in the first frame, the first data buffer comprises a depth data buffer, and the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Presenting the object in a second color different from the first color in at least one frame after the first frame and / or in the first frame.

16. The apparatus according to claim 12, wherein the at least one object is rendered in a first color in the first frame, the first data buffer comprises a depth data buffer, and the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Presenting the object in a second color different from the first color in at least one frame of a reproduced version of the computer simulation.

17. The apparatus according to claim 12, wherein the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Highlighting the object in at least one frame after the first frame and / or in the first frame.

18. The apparatus according to claim 12, wherein the instructions of the computer are executable to change the appearance of the at least one object at least in part by: Highlighting the object in at least one frame of the computer-simulated reproduction version.

19. A method, which comprises: Copying together video data representing a first video layer among a plurality of video layers to establish a single computer-simulated video frame, the single frame including at least one object; and Using the video data to change the appearance of the at least one object in the video simulation.

20. The method according to claim 19, wherein the video data is copied from a buffer.

Images