Controller Action Recognition from Video Frames Using Machine Learning

Abstract

A machine learning model (306) is used to receive (700) a recorded video game for presentation on a spectator computer and derive (702) from the video an identification of controller actions during game play that occurred in the recorded video game. Indications of the identified controller actions may be presented (704) along with the recorded video game to assist a viewer in learning how to play the game.

Description

[Technical field]

[0001] The present application relates to technically inventive and unconventional solutions which are necessarily due to computer technology and give rise to concrete technical improvements. [Background technology]

[0002] As understood herein, videos of previously played computer games may be shared over computer networks to instruct viewers on how to succeed in the game, such as by completing a level of the game. As further understood herein, such game videos may not include information about which controller buttons were pressed and timing during game play, because the game videos may be recorded without capturing controller actions as the game is played. Summary of the Invention

[0003] Also, as understood herein, information about which controller buttons are pressed and when is valuable to players learning to play computer games, making games more enjoyable for many types of gamers, from beginners to speed runners. Provided herein are machine learning techniques that generate controller action information by analyzing a sequence of video frames without additional controller data.

[0004] Thus, the device includes at least one computer memory, the at least one computer memory including instructions executable by at least one processor to receive a recorded computer simulation including a sequence of video frames, but not a temporal signal. The instructions are executable to process the sequence of video frames in a machine learning (ML) model, and to receive from the ML model an identification of a computer simulation controller (CSC) operation associated with generating the recorded computer simulation. Additionally, the instructions are executable to present the recorded computer simulation on at least one audio-video (AV) display together with at least one indication of the CSC operation received from the ML model.

[0005] In an example embodiment, the ML model includes at least one recurrent neural network (RNN), such as at least one long short-term memory (LSTM) network. Convolutional neural networks (CNN) can also be used.

[0006] The device may include a processor, which may be incorporated in the AV display or in a source of computer simulation, such as a computer simulation console, and / or in a server that communicates with the AV display over a wide area computer network.

[0007] In another aspect, the apparatus includes at least one display configured to present at least one recorded computer-simulated video generated under control of the at least one computer simulation controller. However, the recorded computer simulation does not include information regarding operation of the computer simulation controller during generation of the at least one recorded computer-simulated video. Thus, the apparatus includes at least one processor, the processor configured with instructions for identifying, from the video during generation of the at least one recorded computer-simulated video, information regarding operation of the computer simulation controller. The instructions are executable to provide, to the at least one display, information regarding operation of the computer simulation controller during generation of the at least one recorded computer-simulated video for presenting the information in conjunction with presenting the at least one recorded computer-simulated video.

[0008] In an example implementation, at least one machine learning (ML) model may be executable to identify, during generation of the at least one recorded video of the computer simulation, information related to the operation of the computer simulation controller from the video.

[0009] In another aspect, the method includes inputting at least a training set into at least one machine learning (ML) model. The training set includes sequences of video frames from a plurality of recorded computer simulations and information associated with the sequences of video frames regarding computer simulation controller (CSC) operations performed during generation of the sequences of video frames. The method then includes inputting at least a first recorded computer simulation into the ML model, the first recorded computer simulation not including information regarding CSC operations performed during generation of the first recorded computer simulation. The method includes presenting the first recorded computer simulation together with audible and / or visual information regarding CSC operations performed during generation of the first recorded computer simulation received from the ML model.

[0010] The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference characters refer to like parts, and in which: [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram of an exemplary system including an example according to the present principles; [Diagram 2] 1 shows an example of a game controller with exemplary key operations that can be learned from a video of a recorded computer simulation (such as a computer game) in accordance with the present principles. [Diagram 3] An example is provided for viewing a recorded computer simulation, such as a computer game, in accordance with the present principles. [Figure 4] 1 illustrates exemplary logic in an exemplary flowchart format for training a machine learning (ML) model to derive information about what controller operations were performed and when they were performed from recorded computer simulation video. [Diagram 5] 1 illustrates a schematic of a sequence of video frames with accompanying information regarding controller operations that can be used to train an ML model. [Figure 6] 1 illustrates a schematic of a sequence of video frames without accompanying information regarding controller actions and alternative video paths that may occur depending on the controller action, illustrating an identification of the controller action. [Figure 7] 1 illustrates exemplary logic in an exemplary flow chart format for identifying controller operations from a sequence of recorded video frames from a recorded computer simulation. [Figure 8] 1 illustrates an exemplary output representation from an ML model in accordance with present principles. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Thus, the present principles may use machine learning models, including deep learning models. Machine learning models use a variety of algorithms trained in ways that include supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that may be implemented by computer circuitry include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs) that may be suitable for learning information from a series of images, and a type of RNN known as a long short-term memory (LSTM) network. Support vector machines (SVMs) and Bayesian networks may also be considered examples of machine learning models.

[0013] As understood herein, performing machine learning involves accessing and then training a model with training data, allowing the model to process additional data and make predictions. A neural network may include an input layer, an output layer, and multiple hidden layers in between, which are configured and weighted to infer about a suitable output.

[0014] The present disclosure relates generally to computer ecosystems, including, but not limited to, aspects of consumer electronics (CE) device networks, such as computer gaming networks. The systems herein may include server and client components that may be connected through a network, whereby data may be exchanged between the client and server components. The client components may include one or more computing devices, including gaming consoles such as Sony PlayStation® or gaming consoles made by Microsoft® or Nintendo® or other manufacturers, virtual reality (VR) headsets, augmented reality (AR) headsets, portable televisions (e.g., smart televisions, Internet-enabled televisions), portable computers such as laptops and tablet computers, and smartphones and other mobile devices, including additional examples described below. These client devices may operate in a variety of operating environments. For example, some of the client computers may use, by way of example, a Linux® operating system, a Microsoft® operating system, or a Unix® operating system, or an operating system manufactured by Apple® or Google®. These operating environments may be used to run one or more browsing programs, such as browsers made by Microsoft® or Google® or Mozilla®, or other browser programs that can access web sites hosted by the Internet servers described below. Operating environments consistent with the present principles may also be used to run one or more computer game programs.

[0015] The server and / or gateway may include one or more processors that execute instructions to configure the server to receive and transmit data over a network such as the Internet. Alternatively, the clients and servers may be connected through a local intranet or a virtual private network. The server or controller may be instantiated by a gaming console such as a Sony PlayStation®, a personal computer, or the like.

[0016] Information may be exchanged between the clients and the servers over a network. For this purpose and for security, the servers and / or clients may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security. One or more servers may form an apparatus for implementing a method for providing a secure community, such as an online social website, for network members.

[0017] The processor may be a single-chip processor or a multi-chip processor capable of implementing logic through various lines, such as address lines, data lines, and control lines, as well as registers and shift registers.

[0018] Components included in one embodiment may be used in other embodiments in any suitable combination, for example, any of the various components described herein and / or illustrated in the figures may be combined, interchanged, or eliminated from other embodiments.

[0019] A "system having at least one of A, B, and C" (similarly, "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.

[0020] Referring now specifically to FIG. 1, an exemplary system 10 is shown, which may include one or more of the exemplary devices described above and in detail below in accordance with the present principles. A first exemplary device included in the system 10 is a consumer electronics (CE) device, such as an audio-video device (AVD) 12, such as, but not limited to, an Internet-enabled television having a television tuner (as well as a set-top box that controls the television). Alternatively, the AVD 12 may also be a computer-controlled Internet-enabled ("smart") phone, a tablet computer, a notebook computer, an HMD, a wearable computer-controlled device, a computer-controlled Internet-enabled music player, a computer-controlled Internet-enabled headphones, a computer-controlled Internet-enabled implantable device such as an implantable skin device, and the like. Regardless, it should be understood that the AVD 12 is configured to implement the present principles (e.g., to communicate with other CE devices to implement the present principles, to execute the logic described herein, and to perform any other functions and / or operations described herein).

[0021] Accordingly, to implement such principles, an AVD 12 may be established by some or all of the components shown in FIG. 1. For example, the AVD 12 may include one or more displays 14, which may be implemented by high-resolution or super-resolution "4K" or higher resolution flat screens and may be touch-enabled to receive user input signals by touching the display. The AVD 12 may include one or more speakers 16 for outputting audio in accordance with the present principles, and at least one additional input device 18, such as, for example, an audio receiver / microphone, for inputting audible commands to the AVD 12 and controlling the AVD 12. The exemplary AVD 12 may also include one or more network interfaces 20 for communicating over at least one network 22, such as the Internet, a WAN, a LAN, etc., under the control of one or more processors 24. It may also include a graphics processor. Thus, the interface 20 may be, but is not limited to, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, but not limited to, a mesh network transceiver. It should be understood that processor 24 controls AVD 12 to implement the present principles, including other elements of AVD 12 described herein, such as controlling display 14 to present images thereon and receiving input therefrom. It should further be noted that network interface 20 may be a wired or wireless modem or router, or other suitable interface, such as a wireless telephony transceiver or the Wi-Fi transceiver discussed above.

[0022] In addition to the above, AVD 12 may also include one or more input ports 26, such as a High-Definition Multimedia Interface (HDMI) port or a USB port for physically connecting to another CE device, and / or a headphone port for connecting headphones to AVD 12 to present audio to a user from AVD 12 via headphones. For example, input port 26 may be wired or wirelessly connected to a cable or satellite source 26a of audio-video content. Thus, source 26a may be a separate or integrated set-top box, or a satellite receiver. Alternatively, source 26a may be a game console or disc player containing content. When implemented as a game console, source 26a may include some or all of the components described below in connection with CE device 44.

[0023] AVD 12 may further include one or more computer memories 28, such as non-transitory, disk-based or solid-state storage, which in some cases are embodied within the AVD's chassis, either as a standalone device or as a personal video recording device (PVR) or video disk player, or as a removable memory medium, either internal or external to the AVD's chassis, for playing AV programs. In some embodiments, AVD 12 may also include a position or location receiver, such as, but not limited to, a cellular receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographic location information from a satellite or cellular base station, provide the information to processor 24, and / or determine the altitude at which AVD 12 is located in conjunction with processor 24. Component 30 may also be implemented with an inertial measurement unit (IMU), typically including a combination of accelerometers, gyroscopes, and magnetometers, to determine the position and orientation of AVD 12 in three dimensions.

[0024] Continuing with the description of the AVD 12, in some embodiments, the AVD 12 may include one or more cameras 32, which may be digital cameras such as thermal imaging cameras, webcams, and / or cameras integrated into the AVD 12 and controllable by the processor 24 to collect pictures / images and / or videos in accordance with the present principles. Also included in the AVD 12 may be a Bluetooth transceiver 34 and other NFC elements 36 for communicating with other devices using Bluetooth and / or Near Field Communication (NFC) technologies, respectively. An exemplary NFC element may be a Radio Frequency Identification (RFID) element.

[0025] Furthermore, the AVD 12 may include one or more auxiliary sensors 38 (e.g., motion sensors such as accelerometers, gyroscopes, cyclometers, or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., sensors for detecting gesture commands)) that provide input to the processor 24. The AVD 12 may include a wireless television broadcast port 40 for receiving over-the-air (OTA) TV broadcasts that provide input to the processor 24. In addition to the above, it is noted that the AVD 12 may also include an IR transmitter and / or an IR receiver and / or an IR transceiver 42, such as an infrared (IR) data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, and may be a kinetic energy harvester that may convert kinetic energy into electrical power to charge the battery and / or power the AVD 12. A graphics processing unit (GPU) 44 and a field programmable gate array 46 may also be included.

[0026] With further reference to FIG. 1, in addition to the AVD 12, the system 10 may include one or more other CE device types. In one example, the first CE device 48 may be a computer game console that can be used to transmit computer game audio and video to the AVD 12 via commands sent directly to the AVD 12 and / or via a server as described below, while the second CE device 50 may include similar components as the first CE device 48. In the example shown, the second CE device 50 may be configured as a computer game controller manipulated by a player or a head mounted display (HMD) worn by a player. It should be understood that in the example shown, only two CE devices are shown, and a fewer or greater number of devices may be used. The devices herein may implement some or all of the components shown for the AVD 12. Any of the components shown in the following figures may incorporate some or all of the components shown in the case of the AVD 12.

[0027] Referring now to the at least one server 52 mentioned above, the server 52 includes at least one server processor 54, at least one tangible computer-readable storage medium 56, such as disk-based or solid-state storage, and at least one network interface 58 that, under the control of the server processor 54, enables communication with other devices of FIG. 1 over the network 22, and may, in fact, facilitate communications between the server and client devices in accordance with the present principles. It should be noted that the network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as, for example, a wireless telephony transceiver.

[0028] Thus, in some embodiments, server 52 may be an entire Internet server or server "farm" and may include or perform "cloud" functions such that devices of system 10 may access a "cloud" environment via server 52, for example in an exemplary embodiment directed to a network gaming application, or server 52 may be implemented by one or more gaming consoles or other computers in the same room or nearby as the other devices shown in FIG.

[0029] The components illustrated in the following figures may include some or all of the components illustrated in FIG.

[0030] FIG. 2 shows a non-limiting example of a game controller that may be used in accordance with the present principles to control a computer simulation, such as a computer game, during game play, and game video (and audio / haptics, etc.) associated with the game may be shown on a display, such as display 14 (in the case of haptics, e.g., generated for haptic detection using a controller) and / or recorded for playback, e.g., computer memory 28 shown in FIG. 1 and / or server memory 56 shown in FIG. 1.

[0031] It should be understood that the game controller may incorporate one or more of the above-described components (such as CE device 48 embodied as a computer game machine and / or server 52) that communicate with a source of a computer simulation to control a computer game presented on display 14.

[0032] 2 shows a controller 200 that includes a lightweight handheld housing with circular, generally cylindrical, left and right handles 202, 204, each defining a top surface on which four operable keys are disposed. For example, four directional keys 206 are arranged in a cross pattern on the top of the left handle 202. The keys 206 can be used to move objects in their respective directions on the display.

[0033] Additional L1 and L2 keys 208 may be provided immediately forward of the left handle 202. A bridge connects the handles 202, 204, and a select key 210 may be located on the bridge along with a start key 212.

[0034] The four keys on the right handle 204 may include a triangle key 214, a square key 216, an "O" key 218, and an "X" key 220, each of which may assume a respective function according to the desires of the game designer. Additional R1 and R2 keys 222 may be provided immediately forward of the right handle 204.

[0035] Also, a left joystick 224 may be provided between handles 202 and 204, just inboard of left handle 202. The left joystick 224 may include a depressible top portion 226. Similarly, a right joystick 228 may be provided just inboard of right handle 204. The right joystick 228 may include a depressible top portion 230.

[0036] 3 illustrates a device 300 that records video of a computer simulation (such as a computer game) and, optionally, records audio and haptics associated with the video. Device 300 may be implemented, for example, by a device on which the game is played to generate the video, a computer game machine, a computer game server remote from the player, or a combination thereof.

[0037] A recorded computer simulation (such as a computer game) is provided to a recorded computer game source 302. Source 302 provides recorded video (and, optionally, recorded audio and other sensory output of the game, referred to herein as audio-video) to spectator / learner computing devices 304.

[0038] Since computer game AV may be recorded without recording the computer game controller operations input to control the game during recording, a machine learning (ML) engine 306 is provided for execution according to the principles herein to reconstruct controller operations from the computer game audio-video to present indications to the spectator / computer 304 along with the recorded game AV. The ML model 306 may include at least one recurrent neural network (RNN), such as at least one long short-term memory (LSTM) network. Convolutional neural networks (CNN) may also be used. The ML model 306 may be executed by any or a combination of processors disclosed herein, such as a processor in the spectator / student computer 304, the recorded game source 302 (such as a remote server or a local game machine). In some embodiments, elements 300, 302, and 304 may be implemented by the same device. For example, a user may attempt to learn from a game recording of a different user who was playing earlier on the same console.

[0039] Figure 4 shows logic for training the ML model 306 of Figure 3. At block 400, a training set of data is input to the ML model. The training set may include video and / or audio typically associated with multiple computer simulations, such as computer games.

[0040] With greater specificity, the training set may include sequences of video frames from multiple recorded computer simulations, and / or audio associated with the video, if desired. Additionally, the training set may include ground truth information associated with the sequences of video frames regarding computer simulation controller (CSC) operations that were performed during the occurrence of the sequences of video frames. The CSC operations may result, for example, from any one or more manipulations of the controls shown in the example of FIG. 2. From the training set, and using an appropriate learning technique (e.g., supervised, unsupervised, semi-supervised, etc.), in block 402, the ML model 306 learns the CSC operations from the video frame sequences.

[0041] FIG. 5 shows a sequence 500 of video frames 502 in a training set used to train an ML model. Metadata 504 is optionally associated with the sequence 500 for each frame, indicating which controller operations occurred during the recording of the training sequence and when they occurred. Note that in some embodiments, the metadata does not need to be per frame. The metadata may be updated only if the CSC operations have changed compared to the previous frame. Also, the CSC operations may be stored in an entirely separate file along with a timestamp that could be used to synchronize the video data and the CSC data. The metadata 504 may include an indication of what specific control surfaces were manipulated in a controller, such as the controller shown in FIG. 2, and when those control surfaces were manipulated during the generation of the recorded sequence 500.

[0042] The training set may be created in a variety of ways. Button manipulation ground truth may be collected during gameplay and associated with time-aligned generated video for use as part of the training set. The training set may also include pre-recorded game videos that have controller overlays on them that are generated when the original game is played and presented with the video. The controller manipulation data in the overlay is already time-aligned with the video since it is typically presented over the video. The pixel values ​​of the video for each frame may be checked to see which button was pressed as indicated in the overlay to generate the labeling data. This also provides timing data since each frame may be associated with a timestamp. The overlay feature may be turned on during initial playback when the training set videos are generated.

[0043] The ground truth controller actions can be streamed from the controller to a device that records game video under the control of the controller, thereby correlating the ground truth controller actions with the video to establish elements of a training set.

[0044] Following the training, the ML model may be used to receive a first recorded computer simulation with a sequence 600 of video frames (for clarity, optional audio is not shown in FIG. 6 ) that does not include information about CSC operations performed during the generation of the first recorded computer simulation. Depending on how the sequence of video frames progresses, the ML model may identify, based on the training of FIG. 5 , which CSC operations were performed during the generation of the first recorded simulation and when the CSC operations were performed. For example, between sequence 1 shown in FIG. 6 and potential sequence 2A, no CSC controller operation may have occurred. On the other hand, between sequence 1 and potential sequence 2B (a different sequence from potential sequence 2A), a CSC operation may be performed to shift the entire sequence from sequence 1-sequence 2A to sequence 1-sequence 2B, and such CSC operation identification 602 is performed by the ML model.

[0045] Figure 7 illustrates the principles discussed above in an exemplary flow chart format. The logic of Figure 7 may be executed by any one or more of the processors embedded in the spectator / learner AV display 304, and / or the source of generated computer simulation 300, and / or the source of recorded computer simulation 302, either or both of which may be instantiated as a computer simulation console or a server in communication with the AV display over a wide area computer network such as the Internet.

[0046] In addition to the following, it will be appreciated that blocks may be provided for pre-processing and downscaling to shorten the time required for the ML model to identify CSC operations, as well as blocks for post-processing and synchronization to filter unsupported CSC operations and adjust latency.

[0047] Beginning at step 700, a recorded computer simulation, such as a recorded computer game, is received, the recorded computer simulation including a sequence of video frames and / or an accompanying audio soundtrack. Typically, the recorded simulation does not include information regarding the operation of a computer simulation controller during the generation of the recorded computer simulation. The sequence of frames may be, for example, a snippet of a game video from an internet platform.

[0048] Proceeding to block 702, the recorded computer simulation, e.g., a sequence of video frames in the recorded simulation, is processed by the ML model 306 in FIG. 3, which has been trained as disclosed with reference to FIG. 4. The ML model identifies and outputs indications of computer simulation controller (CSC) operations, and the CSC operations are associated with the generation of the recorded computer simulation according to the training received and associated with the recorded computer simulation. The CSC operations identified by the ML model may include, for example, in the example of FIG. 2, which control elements were manipulated and when they were manipulated.

[0049] Proceeding to block 704, the recorded computer simulation is presented on at least one audio-video (AV) display, such as spectator / learner computer 304 of FIG. 3, along with at least one indication of the CSC operation received from the ML model.

[0050] Illustrated in Figure 8 is a recorded video game 800 that may be presented on a display, such as display 14 shown in Figure 1. Game 800 includes a character 802 firing a weapon 804 at a flying object 806. A line 808 indicates that object 806 was hit.

[0051] It will be appreciated that an indication 810 is presented on the display indicating which CSC operation occurred ("red key pressed") and when the CSC operation occurred during the game, shown in the example as "now" to indicate past CSC operations that led to the explosion and the time of operation, and may also indicate future CSC operations for exploration. The indication 810 may be presented visually as shown and / or audibly with a speaker associated with the display. In this manner, the recorded computer simulation is presented along with information received from the ML model regarding the CSC operations performed during generation of the recorded computer simulation.

[0052] The above logic may be provided as a plug-in with a computer game controller or other functionality to enable a gamer to download game video and obtain information about the sequence of controller actions that generated the video.

[0053] While the present principles have been described with reference to certain illustrative embodiments, it will be recognized that these are not intended to be limiting and that various alternative arrangements may be used to practice the subject matter claimed herein.

Claims

1. Not a temporary signal, receiving a recorded computer simulation including a sequence of video frames; inputting the sequence of video frames into a machine learning (ML) model; receiving, output from the ML model, an identification of at least some computer simulation controller (CSC) operations associated with generating the recorded computer simulation; presenting the recorded computer simulation on at least one audio-video (AV) display along with at least one indication of at least one of the CSC operations received from the ML model; at least one computer memory containing instructions executable by at least one processor for A device comprising:

2. The device of claim 1 , wherein the ML model includes at least one recurrent neural network (RNN).

3. The device of claim 2 , wherein the RNN includes at least one long short-term memory (LSTM) network.

4. The device of claim 1 , wherein the ML model includes at least one convolutional neural network (CNN).

5. the at least one processor; The device of claim 1 , wherein the at least one processor is integrated into the AV display.

6. the at least one processor; The device of claim 1 , wherein the at least one processor is integrated into the source of the computer simulation.

7. The device of claim 6 , wherein the source comprises at least one computer simulation console.

8. The device of claim 6 , wherein the source comprises at least one server in communication with the AV display over a wide area computer network.

9. at least one display configured to present at least a video of at least one recorded computer simulation generated under control of at least one computer simulation controller, the at least one recorded computer simulation not including information regarding operation of the computer simulation controller during generation of the video of the at least one recorded computer simulation; At least one processor; The at least one processor: receiving information regarding operation of the computer simulation controller output from at least one machine learning (ML) model by inputting the at least one recorded computer simulation into the at least one ML model during generation of the video of the at least one recorded computer simulation; presenting the information regarding operation of the computer simulation controller during generation of the video of the at least one recorded computer simulation on the at least one display to present the information in conjunction with presenting the video of the at least one recorded computer simulation; An apparatus for performing the above.

10. The apparatus of claim 9 , wherein the processor is integrated into the display.

11. The apparatus of claim 9 , wherein the processor is incorporated in a computer simulation console.

12. The apparatus of claim 9 , wherein the processor is incorporated in a server that communicates with the display over a wide area network.

13. The apparatus of claim 9 , wherein the ML model includes at least one recurrent neural network (RNN).

14. The apparatus of claim 13 , wherein the RNN includes at least one long short-term memory (LSTM) network.

15. inputting at least a training set into at least one machine learning (ML) model, the training set including sequences of video frames from a plurality of recorded computer simulations and information associated with the sequences of video frames regarding computer simulation controller (CSC) operations performed during generation of the sequences of video frames; inputting into the ML model at least the first recorded computer simulation that does not include information regarding CSC operations performed during generation of the first recorded computer simulation; receiving information regarding CSC operations performed during generation of the first recorded computer simulation, the CSC operations being output from the ML model; presenting the first recorded computer simulation along with information regarding the CSC operation; A method comprising:

16. 16. The method of claim 15, further comprising audibly presenting information regarding the CSC operations performed during generation of the first recorded computer simulation received from the ML model in conjunction with visually presenting the first recorded computer simulation.

17. 16. The method of claim 15, comprising visually presenting information regarding the CSC operations performed during generation of the first recorded computer simulation received from the ML model along with visually presenting the first recorded computer simulation.

18. 16. The method of claim 15, comprising associating the first recorded computer simulation with information regarding CSC operations performed during generation of the first recorded computer simulation at a server that provides the first recorded computer simulation on a display that presents the first recorded computer simulation.

19. 16. The method of claim 15, comprising associating the first recorded computer simulation with information regarding CSC operations performed during generation of the first recorded computer simulation at a local source that provides the first recorded computer simulation on a display that presents the first recorded computer simulation.

Images