System and method for emulating graphic and audio parameters during the play of legacy games - Patents.com
The system emulates graphical and audio parameters of legacy games by caching and compiling code into basic blocks, enabling execution on updated machines with improved graphics and audio, addressing compatibility issues and security concerns.
Patent Information
- Application Number
- JP2023573449
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-06-07
- Filing Date
- 2022-03-24
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-03-24
AI Technical Summary
Legacy games are often incompatible with modern gaming systems due to their complex graphical and audio parameters, which cannot be executed by updated machines without authority and security issues, leading to a desire to play simpler, older games.
A system and method for emulating graphical and audio parameters by caching and compiling legacy game code into basic blocks, allowing execution on updated machines using a cache and basic block compiler to convert emulated processing unit (PU) code into intermediate game code, facilitating the use of updated handheld controllers and high-resolution graphics and audio.
Enables the execution of legacy games on updated machines with improved graphics and audio, reducing security issues and execution time, while minimizing processing power and latency, and allowing seamless play with modern input devices.
Smart Images

Figure 0007587065000001 
Figure 0007587065000002 
Figure 0007587065000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to systems and methods for emulating graphic and audio parameters during the play of legacy games. [Background technology]
[0002] As electronic gaming and networking technologies become more advanced, the complexity of the games increases accordingly. As a result, there may be more complex storylines, gameplay objectives, missions and tasks, abilities associated with gameplay avatars, and scoring. Scoring may be generated and weighted in a variety of ways, as well as determined in a variety of categories or on an individual or team basis.
[0003] The severity of the aforementioned problems only increases as the complexity of electronic games increases, which may result in some players wishing to play older, less complex games.
[0004] It is in this context that embodiments of the present invention arise. Summary of the Invention
[0005] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure provide systems and methods for emulating graphic and audio parameters during the play of legacy games.
[0006] In one embodiment, a method for facilitating playing of a legacy game is described. The method includes receiving a user input during play of the legacy game, determining whether one or more blocks of code for processing the user input are cached, and accessing one or more instructions of the legacy game code upon determining that the one or more blocks of code are not cached. The method further includes compiling one or more code blocks of code from the one or more instructions of the legacy game code, caching the one or more blocks of code, and executing the one or more blocks of code to display a virtual environment.
[0007] In one embodiment, a computing device for facilitating playing of a legacy game is described. The computing device includes a processor configured to receive user input during play of the legacy game. The computing device further includes a cache coupled to the processor and a memory device coupled to the processor. The processor determines whether one or more blocks of code that process the user input are stored in the cache. If the processor determines that the one or more blocks of code are not stored in the cache, the processor accesses one or more instructions of the legacy game code from the memory device. The processor also compiles the one or more blocks of code from the one or more instructions of the legacy game code. The processor stores the one or more blocks of code in the cache and executes the one or more blocks of code to display a virtual environment.
[0008] In one embodiment, a method is described. The method includes generating a first verification result from one or more instructions of a legacy game code. The one or more instructions of the legacy game code are associated with one or more blocks of code. The method further includes inspecting one or more memory addresses associated with the one or more instructions to determine whether the one or more blocks of code should be marked as invalid. The method includes determining whether the one or more blocks of code should be executed, and determining whether the one or more blocks of code should be marked as invalid upon determining that the one or more blocks of code should be executed. The method includes inspecting the one or more memory addresses to generate a second verification result from the one or more instructions, comparing the first verification result and the second verification result to determine whether the one or more blocks of code are invalid, and upon determining that the one or more blocks of code are invalid, recompiling one or more additional blocks of code associated with the one or more instructions. The method includes executing the one or more additional blocks of code to display the virtual environment.
[0009] In one embodiment, a method for emulating user input during play of a legacy game is described. The method includes receiving user input from an updated handheld controller during play of a legacy game, analyzing the user input to identify an updated input device of the updated handheld controller, and determining a capability of the updated input device. The method further includes determining an identity of a legacy input device of the legacy handheld controller based on an identity of the updated input device. The method includes identifying a capability of the legacy input device that corresponds to a capability of the updated input device, determining whether one or more blocks of code for handling the capability of the legacy input device of the legacy handheld controller are cached, and accessing one or more instructions of a legacy game code of the legacy game upon determining that the one or more blocks of code are not cached. The method includes compiling the one or more blocks of code from the one or more instructions of the legacy game code.
[0010] In one embodiment, a system for playing a legacy game using an updated handheld controller is described. The system includes one or more processors configured to receive user input from the updated handheld controller during play of the legacy game. The one or more processors are further configured to analyze the user input to identify an updated input device of the updated handheld controller and determine a capability of the updated input device. The one or more processors are configured to determine an identity of a legacy input device of the legacy handheld controller based on an identity of the updated input device. The one or more processors are configured to identify a capability of the legacy input device corresponding to the capability of the updated input device and determine whether one or more blocks of code for processing the capability of the legacy input device of the legacy handheld controller are cached. The one or more processors are configured to access one or more instructions of legacy game code of the legacy game in response to determining that the one or more blocks of code are not cached. The one or more processors are configured to compile the one or more blocks of code from the one or more instructions of the legacy game code. A memory device is coupled to the one or more processors.
[0011] In one embodiment, a system for playing a legacy game using an updated handheld controller is described. The updated handheld controller has an updated input device. The updated handheld controller is configured to generate user input during play of the legacy game. A computing device is coupled to the updated handheld controller. The computing device is configured to receive user input from the updated handheld controller, analyze the user input to identify an updated input device of the updated handheld controller, and determine a capability of the updated input device. The computing device is configured to determine an identity of a legacy input device of the legacy handheld controller based on an identity of the updated input device. The computing device is further configured to identify a capability of the legacy input device corresponding to the capability of the updated input device, determine whether one or more blocks of code for processing the capability of the legacy input device of the legacy handheld controller are cached, and in response to a determination that the one or more blocks of code are not cached, access one or more instructions of legacy game code of the legacy game. The computing device is configured to compile the one or more blocks of code from the one or more instructions of the legacy game code.
[0012] In one embodiment, a method for emulating graphics parameters during play of a legacy game is described. The method includes receiving a user input from a handheld controller and determining whether one or more basic blocks of code that process the user input are stored in a cache. The method further includes compiling the one or more basic blocks of code from one or more emulated processing unit (PU) code instructions upon determining that the one or more basic blocks are not stored in the cache. The method includes executing the one or more basic blocks of code to generate one or more legacy graphics parameters. The method includes emulating the one or more legacy graphics parameters to generate one or more image frames having one or more updated graphics parameters for displaying one or more images of the legacy game on a display device.
[0013] In one embodiment, a method for emulating audio parameters of a legacy game is described. The method includes receiving a user input from a handheld controller during play of the legacy game and determining whether one or more basic blocks of code for processing the user input are stored in a cache. The method further includes compiling one or more basic blocks of code from one or more emulated PU code instructions upon determining that the one or more basic blocks are not stored in the cache. The method includes executing the one or more basic blocks of code to generate one or more legacy audio parameters. The method further includes emulating the one or more legacy audio parameters to generate one or more audio frames having one or more updated audio parameters. The one or more audio frames are output as sound along with one or more images of the legacy game displayed on a display device.
[0014] Some advantages of the systems and methods described herein for converting legacy code to updated code include allowing the functionality of the legacy code to be executed by the updated machine. Without the conversion, the functionality of the legacy code cannot be executed by the updated machine due to security concerns. For example, there is no authority to execute the legacy code from the updated machine and write data generated during execution of the legacy code to a register of the updated machine. Thus, providing the conversion facilitates execution of the functionality of the legacy code by the updated machine.
[0015] Further advantages of the systems and methods described herein include saving execution time. As an example, two or more instructions, such as a routine and a subroutine or two similar instructions of the legacy code, are combined into one basic block of the updated code. Therefore, the execution of the updated code is faster compared to the execution of the legacy code.
[0016] A further advantage of the systems and methods described herein for converting legacy code to updated code includes recompiling one or more additional basic blocks of the updated code upon determining that one or more basic blocks of the updated code are invalid. For example, when one or more basic blocks are marked as invalid, it is determined whether the one or more basic blocks are in fact invalid. Upon such determination, instead of executing the one or more basic blocks, one or more additional basic blocks are compiled and executed. The one or more additional basic blocks correspond to the same game to which the one or more basic blocks correspond.
[0017] Another additional advantage of the systems and methods described herein for converting legacy code to updated code includes not having to check every basic block for invalidity. For example, only basic blocks marked as invalid after compilation of the basic blocks are checked for validity. This reduces the latency in displaying a virtual environment from one or more basic blocks. Also, the processing power used to check every basic block for invalidity is not required.
[0018] Advantages of the systems and methods described herein for converting legacy code to updated code also include saving processing time and processing power when the updated code of a game has already been compiled. Once the updated code is generated on a server or game console, the updated code does not need to be recompiled. Rather, the updated code can be transferred from the server or game console to another game console. Thus, saving processing time and processing power in regenerating the updated code on the other game console.
[0019] Additional advantages of the systems and methods described herein for emulating user input during play of a legacy game include using an updated handheld controller to play the legacy game. An input emulator is used to convert user input received from the updated handheld controller into user input of the legacy handheld controller. The conversion of user input received from the updated handheld controller facilitates playing the legacy game using the updated handheld controller.
[0020] Further advantages of the systems and methods described herein for emulating legacy graphics parameters while playing legacy games include converting the legacy graphics parameters to updated graphics parameters for playing the legacy games. A graphics processing unit (GPU) emulator is used to convert the legacy graphics parameters to the updated graphics parameters. The conversion of the legacy graphics parameters facilitates playing legacy games with high resolution graphics.
[0021] Advantages of the system and method described herein for emulating legacy audio parameters while playing a legacy game also include converting the legacy audio parameters to updated audio parameters for playing the legacy game. An audio processing unit (APU) emulator is used to convert the legacy audio parameters to the updated audio parameters. The conversion of the legacy audio parameters facilitates the legacy games with high quality sound.
[0022] Other aspects of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the embodiments described in the present disclosure.
[0023] The various embodiments of the disclosure can be best understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief description of the drawings]
[0024] [Figure 1] 1 is a block diagram illustrating an embodiment of a system for generating basic blocks of updated code. [Diagram 2] 1 is a flow chart illustrating an embodiment of a method for compiling and executing basic blocks of game code. [Diagram 3]1 is a block diagram illustrating an embodiment of a system for compiling and executing basic blocks. [Figure 4A] FIG. 1 illustrates an embodiment of a system for illustrating compilation of basic blocks within a game console. [Figure 4B] FIG. 1 illustrates an embodiment of a system for illustrating compilation of basic blocks in a server system. [Figure 5A] FIG. 2 is a diagram for explaining an embodiment of a basic block. [Figure 5B] FIG. 2 is a diagram for illustrating an embodiment of a compilation operation performed by a basic block compiler. [Figure 6A] FIG. 1 is a diagram of an embodiment of a system showing components of an emulation processor system. [Figure 6B] 1 is a flow chart illustrating an embodiment of a method for compiling and executing basic blocks. [Figure 6C] FIG. 2 is a diagram of an embodiment of a system illustrating that different basic blocks are dynamically compiled for different user inputs. [Figure 7A] FIG. 1 illustrates an embodiment of a system illustrating removal of emulated processing unit (PU) code from a memory device. [Figure 7B] 1 is a flow chart illustrating an embodiment of a method for removing emulated PU code from a memory device. [Figure 8A] FIG. 1 is a diagram of an embodiment of an emulation processor system illustrating verification of basic blocks. [Figure 8B] 8B is a flow chart of an embodiment of a method illustrating verification operations performed by the emulation processor system of FIG. 8A. [Figure 8C] 8C is a continuation of the flow chart of the method of FIG. 8B. [Figure 9A] FIG. 2 illustrates an embodiment of a legacy machine. [Figure 9B] FIG. 1 illustrates an embodiment of an updated machine. [Figure 10A]FIG. 2 is a diagram of an embodiment of a system illustrating the combining of multiple basic blocks into one basic block by a basic block compiler. [Figure 10B] FIG. 1 illustrates an embodiment of a system illustrating modification of one or more basic blocks. [Figure 10C] FIG. 1 is a diagram of an embodiment of a system illustrating combining basic blocks created based on subroutines with basic blocks generated based on emulated PU code instructions that call the subroutines. [Figure 10D] FIG. 2 is a diagram of an embodiment of a system illustrating the insertion of a basic block between two basic blocks. [Figure 10E] FIG. 2 is a diagram of an embodiment of a system showing a switch in order of execution of basic blocks. [Figure 11A] 1 is a flow chart of an embodiment of a method for illustrating the use of a count of the number of cycles stored in a basic block n. [Figure 11B] 11B is a continuation of the flow chart of the method of FIG. 11A. [Figure 12] FIG. 1 illustrates an embodiment of a system for illustrating a transfer of a basic block from a first client device to a second client device. [Figure 13] FIG. 1 illustrates an embodiment of a system to illustrate emulation of user input received from an updated handheld controller during play of a legacy game. [Figure 14A-1] FIG. 1 illustrates an embodiment of a system for illustrating a change from a default mapping to a current mapping. [Figure 14A-2] FIG. 2 is a diagram of an embodiment of a memory device to illustrate default and current mappings. [Figure 14B] FIG. 1 illustrates an embodiment of a system to illustrate the use of a current mapping instead of a default mapping during play of a legacy game. [Figure 15]FIG. 1 illustrates an embodiment of a system to illustrate identification of the right movement button of a legacy handheld controller based on movement of the right thumbstick of an updated handheld controller. [Figure 16A] 1 is a diagram of an embodiment of a system to illustrate the generation of a virtual image of a crosshair when a touchpad is selected by a user. [Figure 16B] 1 is a diagram of an embodiment of a system for illustrating the correspondence between the location of a user's selection on a touchpad and the location of a crosshair on an image frame. [Figure 17] FIG. 1 illustrates an embodiment of a system to illustrate the use of different mappings with different types of handheld controllers. [Figure 18] FIG. 1 illustrates an embodiment of a system for illustrating communication of a current mapping from a game console to another game console. [Figure 19A] 13 is a screenshot of an image illustrating receiving an updated handheld controller type and subtype. [Figure 19B] 13 is a screenshot of an image illustrating a menu for receiving an updated handheld controller type. [Figure 20] 11 is an embodiment of screenshots of images to illustrate mapping between input devices of a legacy handheld controller and an updated handheld controller. [Figure 21A] FIG. 1 illustrates an embodiment of a system for illustrating receiving updated graphics parameters from a user. [Figure 21B-1] FIG. 1 illustrates an embodiment of a system for illustrating emulation of one or more legacy graphics parameters to one or more updated graphics parameters. [Figure 21B-2] FIG. 1 is a diagram of an embodiment of a system to illustrate the generation of an updated graphical frame from a legacy graphical frame output from a block dispatcher. [Figure 21C-1]1 is an embodiment of a list of legacy and updated graphic parameters, and a correspondence between the legacy and updated graphic parameters. [Figure 21C-2] FIG. 1 illustrates an embodiment of a method performed by a graphics processing unit (GPU) emulator to change the resolution of a virtual object during a rendering operation. [Figure 21D] 1A-1C are diagrams illustrating an embodiment of a method for generating an image frame based on updated graphic parameters. [Figure 22A] FIG. 1 is a diagram of an embodiment of a system to illustrate receiving updated audio parameters from a user. [Figure 22B-1] FIG. 1 illustrates an embodiment of a system for illustrating emulation of one or more legacy audio parameters to one or more updated audio parameters. [Figure 22B-2] FIG. 1 is a diagram of an embodiment of a system to illustrate generation of updated audio frames from legacy audio frames output from a block dispatcher. [Figure 22C-1] FIG. 2 is a diagram illustrating an embodiment of a method for generating an updated audio frame based on legacy audio parameters. [Figure 22C-2] FIG. 1 illustrates an embodiment of a system for illustrating processing of audio frames of a legacy game by a client device. [Figure 22C-3] FIG. 1 illustrates an embodiment of a system that illustrates synchronization of displaying images based on multiple image frames and outputting sounds based on multiple audio frames for multiple virtual scenes of a legacy game. [Figure 22D] FIG. 2 is a diagram of an embodiment of a display device for displaying legacy and normal modes. [Diagram 23] FIG. 1 is a diagram of an embodiment of a display device for displaying a virtual environment in which a menu is displayed. [Figure 24A]FIG. 1 is a diagram of an embodiment of a display device for displaying a virtual environment in which a menu is displayed. [Figure 24B] FIG. 1 illustrates an embodiment of a display device for displaying a virtual environment having multiple scan lines. [Figure 25A] FIG. 1 is a diagram of an embodiment of a display device for displaying a virtual environment in which a menu is displayed. [Figure 25B] FIG. 1 is a diagram of an embodiment of a display device for displaying a virtual environment in which a menu is displayed. [Figure 25C] FIG. 1 illustrates an embodiment of a display device for displaying a virtual environment in which multiple menus are displayed. [Figure 25D] FIG. 2 is a diagram of an embodiment of a display device for displaying a virtual environment with a crosshair displayed. [Figure 26] A flow diagram conceptually illustrating various operations performed to stream cloud video games to client devices in accordance with embodiments of the present disclosure. [Figure 27] FIG. 1 is a block diagram of an embodiment of a game console compatible with interfacing with a display device of a client device and capable of communicating with a game hosting system over a computer network. [Figure 28] FIG. 1 is a diagram showing components of a head mounted display (HMD). [Figure 29] 1 illustrates an embodiment of an Information Service Provider (INSP) architecture. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0025] A system and method for emulating graphic and audio parameters during the play of a legacy game is described. It should be noted that various embodiments of the present disclosure may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order to not unnecessarily obscure various embodiments of the present disclosure.
[0026] FIG. 1 is a block diagram illustrating an embodiment of a system 100 for generating basic blocks 1 through n of updated code, where n is a positive integer. The system 100 includes a cache 102, a basic block compiler 104, and emulated processing unit (PU) code 106 for a legacy game N having a game title GN, where N is a positive integer. As an example, a cache, as used herein, is a hardware or software component that stores data so that future requests for the data can be served faster. A cache hit occurs when the requested data can be found in the cache, while a cache miss occurs when it cannot be found. A cache hit is served by reading the data from the cache, which is faster than recalculating the result or reading from a slower data store such as a memory device, so the more requests that can be served from the cache, the faster the system will run. To explain, a cache is a group of registers that can be accessed faster, such as 10 to 100 times faster, compared to a main memory device.
[0027] As an example, a cache has a smaller number of memory addresses than the main memory device, and in this example, the processor first determines whether the data used in an operation is stored at a memory address in the cache, and if not, the processor accesses the memory address in the main memory device to locate the data.
[0028] As an example, a basic block compiler as used herein is a computer program that converts the emulated PU code 106 code into game code GCN, which is an example of updated code. The game code GCN represents the functionality of the legacy game N. The computer program is executed by one or more processors of the emulation processor system. The game code GCN is referred to as intermediate code herein. As an example, the intermediate code is neither source code nor machine code. To explain, the intermediate code includes basic blocks that are not specific to the architecture of the central processing unit (CPU), or the architecture of the graphic processing unit (GPU), or the architecture of the audio processing unit (APU) of the updated machine, examples of which are provided below. In the example, the intermediate code includes basic blocks that can be executed on the CPU or GPU of the updated machine, examples of which include a game console, a desktop computer, a laptop computer, a tablet, a smartphone, and a smart TV. As an example, the source code is written using a human-readable programming language, which may be plain text. As an example, the basic block compiler as used herein is implemented using hardware or software, or a combination thereof. To illustrate, the functionality of the basic block compiler is implemented using a controller or a programmable logic device (PLD) or an application specific integrated circuit (ASIC).
[0029] Examples of controllers used herein include a processor and a memory device. The processor is coupled to the memory device. As used herein, by way of example, a processor is a microprocessor, or a CPU, or a GPU, or a microcontroller, or an ASIC, or a PLD. Examples of memory devices used herein include random access memory (RAM) and read-only memory (ROM). Explained, the memory device is a flash memory device, or a hard disk, or a solid-state storage device, or a redundant array of independent disks (RAID), or a combination thereof.
[0030] An example of the emulated PU code 106 is machine code, which instructs a processor, such as a CPU or GPU or APU, of the legacy machine to perform an operation. For example, the emulated PU code 106 includes a sequence of instructions that instructs the CPU of the legacy machine to perform a particular operation, such as a load, store, jump, or arithmetic logic unit (ALU) operation, on data stored in a register of the CPU. As another example, the emulated PU code 106 is a binary code that includes a series of 1s and 0s. As another example, the emulated PU code 106 includes a sequence of instructions that instructs the GPU of the legacy machine to perform a particular operation, such as a load, store, jump, or ALU operation, on data stored in a register of the GPU. The GPU of the legacy machine performs operations on virtual objects and assigns parameters, such as color, intensity, shading, texture, or a combination thereof, to the virtual objects. Examples of virtual objects include virtual vehicles, virtual animals, virtual game characters, virtual stationary objects, virtual buildings, etc. It should be noted that color, intensity, shading, texture, or combinations thereof are examples of legacy graphics parameters.
[0031] As yet another example, the emulated PU code 106 includes a set of instructions that instruct the APU of the legacy machine to generate one or more legacy audio parameters, such as a reverb, which is an amount of reverberation that is output in sync with the display of the virtual scene of the legacy game N. Another example of the one or more legacy audio parameters includes a sampling rate based on which sounds may be output in sync with the virtual scene of the legacy game N. Yet another example of the one or more legacy audio parameters includes sounds being output as stereo only, in sync with the virtual objects of the legacy game N.
[0032] The emulated PU code 106 is specific or proprietary to the architecture of the CPU or GPU of the legacy machine. For example, the emulated PU code 106 cannot be executed by the CPU or GPU of the updated machine. As another example, the emulated PU code 106 can be executed on the legacy machine, but not on the updated machine.
[0033] As an example, the emulated PU code 106 is executed by a processor of a legacy machine to perform operations on a legacy game N. Explaining, the emulated PU code 106 is executed to play the legacy game on the legacy machine. Examples of legacy games include video games such as fighter games, dance lesson games, adventure games, action games, etc. Explaining, the legacy game has not been updated by the game developer. To further explain, there is no current version of the legacy game. To further explain, the legacy game was created many years prior to the creation of the current game.
[0034] The basic block compiler 104 accesses the emulated PU code 106 of the legacy game N and converts the emulated PU code 106 into one or more basic blocks, such as basic blocks 1-n. As an example, each basic block 1-n has a beginning identifier and an ending identifier to distinguish one basic block from another basic block. The basic block compiler 104 stores the basic blocks 1-n in the cache 102. When the basic blocks 1-n are executed, the legacy game N is emulated.
[0035] 2 is a flow chart illustrating an embodiment of a method 200 for compiling and dispatching basic blocks of game code GCN. Method 200 is executed by one or more processors of an updated machine. Method 200 includes an operation 202 that determines whether there is a cache hit, e.g., whether a basic block is stored in cache 102. For example, operation 202 is executed or triggered when a user input is received during play of a legacy game. Illustratively, the user input is received to change a position or orientation, or a combination thereof, of a virtual object of legacy game N. Illustratively, upon determining that a user input is received, it is determined whether a basic block, such as one of basic blocks 1-n, is stored in cache 102 (FIG. 1). Illustratively, the basic block is executed to change a position or orientation, or a combination thereof, of a virtual object.
[0036] As another illustration, operation 202 is performed when a user input is received to change a parameter, such as the look and feel, of a virtual object of a legacy game. Illustratively, upon determining that the user input is received, it is determined whether a basic block, such as one of basic blocks 1-n, is stored in cache 102. Illustratively, the basic block is executed to generate or change a parameter of the virtual object. Note that look and feel is one example of a legacy graphics parameter.
[0037] As yet another illustration, operation 202 is performed upon user input to modify legacy audio parameters based on which sounds are to be output. Illustratively, when user input is received, a determination is made whether a basic block, such as one of basic blocks 1-n, is stored in cache 102. Illustratively, the basic block is one that is to be executed to generate or modify legacy audio parameters to be output as sounds accompanying the display of a virtual scene of legacy game N.
[0038] In response to determining that the basic block is cached, in operation 204 of method 200, the basic block is dispatched. For example, the basic block is executed or operated in operation 204. Explained, the basic block is executed by a CPU of the updated machine to move a virtual object from one position to another position, or from one direction to another direction, or a combination thereof. As another explanation, the basic block is executed by a GPU of the updated machine to generate parameters and assign the parameters to a portion of the virtual object. An example of a portion of a virtual object includes a pixel of the virtual object, or a triangular portion of the virtual object, or a portion of a predefined shape of the virtual object. Explained, the virtual object is divided into a predetermined number of pixels, and each pixel is assigned a value of the parameter. As another example, the basic block is executed by an APU of the updated machine to generate legacy audio parameters and output the legacy audio parameters as a sound accompanying a virtual scene of the legacy game N.
[0039] In response to determining that the basic block is not cached, the method 200 performs an operation 206 of compiling the basic block. The operation 206 is performed by the basic block compiler 104 (FIG. 1). For example, the basic block compiler 104 analyzes the emulated PU code 106 (FIG. 1) of the legacy game N to identify, prior to operation 202, emulated PU code instructions that include a function for processing a user input received during play of the legacy game. The user input received prior to operation 202 triggers operation 202. To illustrate, the basic block compiler 104 traverses each emulated code PU instruction of the emulated PU code 106 to determine whether the emulated code PU code instruction includes an operation (e.g., a function) that satisfies (e.g., generates a response) the user input received during play of the legacy game. As illustrated, once a function is identified, the basic block compiler 104 converts the PU code instructions of the emulated code to generate a basic block. As shown, in response to a received user input, prior to operation 202, other emulated code PU code instructions of the legacy game that are not required to process the user input are not compiled into basic blocks by the basic block compiler 104.
[0040] In operation 208 of method 200, the basic blocks generated in operation 206 are stored in cache 102 by basic block compiler 104. The cached basic blocks are then executed in operation 204 to process user input received prior to operation 202.
[0041] In one embodiment, the method 200 is executed by one or more processors of a server system. As an example, the server system includes an updated machine as a server. Illustratively, each server blade is an updated machine such as a game console.
[0042] In one embodiment, method 200 is not executed until user input is received, e.g., no determination is made as to whether a basic block, such as one of basic blocks 1-n, is stored in cache 102, and no compilation or execution of the basic block occurs until user input is received.
[0043] In one embodiment, the basic block compiler 104 compiles one or more of basic blocks 1-n in response to a first user input during play of the legacy game N, and compiles one or more remaining of basic blocks 1-n in response to a second user input during play of the legacy game N. Illustratively, the basic block compiler 104 generates basic blocks 1 and 2 to process the first user input, and generates basic blocks 3-7 to process the second user input, which is received after the first user input.
[0044] 3 is a block diagram illustrating an embodiment of a system 304 for compiling and dispatching basic blocks, such as basic blocks 1-n. The system 300 includes a basic block compiler 104, a cache 102, and a block dispatcher 302. As an example, the block dispatcher 302 is hardware or software, or a combination thereof, and performs operations on one or more of the basic blocks 1-n to process user input. Illustratively, the block dispatcher 302 is a PLD or an ASIC or a controller. As another example, the block dispatcher 302 is computer software. As an example, the block dispatcher 302 is a GPU or a CPU of an updated machine.
[0045] The basic block compiler 104 decodes a portion of the emulated PU code 106 (FIG. 1), such as an operation code (opcode), and converts the portion into an intermediate representation for a processing unit, such as a CPU or GPU or APU, of the updated machine. For example, the basic block compiler 104 analyzes a portion of the CPU code of the emulated PU code 106, such as one or more instructions, to determine whether the portion of the CPU code includes functionality for processing user input. Upon such determination, the basic block compiler 104 converts the portion of the CPU code into one or more basic blocks, such as basic blocks 1-n. As another example, the basic block compiler 104 analyzes a portion of the GPU code of the emulated PU code 106, such as one or more instructions, to determine whether the portion of the GPU code includes functionality for processing user input. Upon such determination, the basic block compiler 104 converts the portion of the GPU code into one or more basic blocks, such as basic blocks 1-n. As yet another example, basic block compiler 104 analyzes a portion of the APU code of emulated PU code 106, such as one or more instructions, to determine whether the portion of the APU code includes a function that processes user input. Upon such determination, basic block compiler 104 converts the portion of the APU code into one or more basic blocks, such as basic blocks 1 through n.
[0046] The basic block compiler 104 also generates an estimated count by estimating the number of cycles of execution of each basic block generated from the portion of the emulated PU code 106. For example, the basic block compiler 104 determines that basic block 1 includes a jump operation, and the jump operation takes a predetermined time. The basic block compiler 104 estimates that the jump operation in basic block 1 takes a predetermined time. The block compiler 104 stores the estimated count in the cache 102. For example, the block compiler 104 stores the estimated count for basic block n, in which the number of cycles is estimated.
[0047] As basic blocks are compiled, they are stored in cache 102 for fast lookup. For example, when a user input is received after another user input is received in response to which basic block is being compiled, the basic block can be quickly accessed from cache 102 and does not need to be regenerated, and the same basic block can be used to process the other user input.
[0048] Additionally, one or more basic blocks stored in cache 102 may be marked as invalid after compilation. The one or more basic blocks marked as invalid are then validated or invalidated during execution of the basic blocks. When the one or more basic blocks are invalidated, one or more additional basic blocks are compiled. Compiling the one or more additional basic blocks is sometimes referred to herein as recompiling the one or more basic blocks.
[0049] Each of the one or more additional basic blocks has a structure that is the same as the structure of the one or more basic blocks 1-n. For example, each of the one or more additional basic blocks has a source register address, a destination register address, and an operation. As another example, each of the one or more additional basic blocks has a source register address, a destination register address, an operation, and a number of cycles of execution of the operation of the additional basic block. In this example, some of the one or more additional basic blocks include an invalid mark. As yet another example, each of the one or more additional basic blocks has a source register address, a destination register address, an operation, and a number of cycles of execution of the operation of the additional basic block. Note that each of the additional blocks of code is executed in the same manner as each of the basic blocks 1-n.
[0050] The block dispatcher 302 executes or operates one or more of the basic blocks 1-n based on the user input. For example, the block dispatcher 302 executes basic blocks 1 and 2 to process a first user input and executes basic blocks 3-7 in response to a second user input. As an example, the block dispatcher 302 includes a clock source, such as a digital clock oscillator or clock generator, that counts the number of cycles used to execute one or more of the basic blocks 1-n based on the user input to generate an actual count. The block dispatcher 302 sends the actual count to the block compiler 104 to update the estimated count with the actual count. For example, the actual count is stored in the basic block n for which the actual count is calculated. To explain, the actual count is stored in one or more memory registers of the cache 102 that are allocated to the basic block n.
[0051] In one embodiment, the basic block compiler 104 does not estimate the number of cycles for execution of any basic block. In this embodiment, there is no replacement of the estimated count with an actual count. Rather, in this embodiment, the actual count is stored by the block compiler 104 in the basic block n for which the count is actually determined.
[0052] FIG. 4A is a diagram of an embodiment of a system 400 for illustrating the compilation of basic blocks 1-n in a game console 402. The system 400 includes a game console 402, a server system 404, a computer network 408, and a display device 410. The server system 404 includes one or more servers. As an example, the server system 404 is disposed in a data center housing. The server system 404 includes a memory device 412 that stores emulated PU code, such as the emulated PU code 104. For example, the memory device 412 stores game code 1 (gc1), game code 2 (gc2), and so on up to game code N (gcN). The game code gcN is an example of the emulated PU code 106 (FIG. 1). Each of the game codes 1-N is a legacy code of a legacy game. To explain, the game code gc1 is a machine code for playing a first legacy game, and the game code gc2 is a machine code for playing a second legacy game. The second legacy game is different from the first legacy game. Note that, as an example, the memory device 412 is a memory device of a legacy machine.
[0053] As an example, none of the game codes gc1-gcN can be executed on an updated machine and cannot be executed on a legacy machine. To explain, the CPU or operating system of the updated machine cannot support the execution of the game codes gc1-gcN. Meanwhile, the CPU or operating system of the legacy machine supports the execution of the game codes gc1-gcN. Examples of computer networks used herein include a wide area and at work (WAN), such as the Internet, or a local area network (LAN), such as an intranet, or a combination thereof.
[0054] The game console 402 is an example of an updated machine. Examples of the display device 410 include a television, a smart television, and a computer monitor. Illustratively, the display device 410 is a liquid crystal display (LCD) device, or a light emitting diode (LED) display device, or an organic light emitting diode (OLED) display device.
[0055] The system 400 further includes a handheld controller (HHC) 414 that is held in one or both hands of the user 1. Examples of handheld controllers as used herein include controllers with buttons, a Move controller from Sony® Corporation, and a gun-style controller. Examples of buttons on a handheld controller include a joystick, buttons for moving a virtual object up, down, left, or right on the display device 410, and other buttons for selecting various functions of a legacy game N having a game title GN.
[0056] The game console 402 includes a memory device 406, an input processor system 407, and an emulation processor system 409. As an example, a processor system as used herein includes one or more processors coupled to each other. The emulation processor system 409 is coupled to the memory device 406, and the input processor system 407 is coupled to the emulation processor system 409. The emulation processor system 409 includes a basic block compiler 104 and a cache 102. The basic block compiler 104 is coupled to the cache 102.
[0057] The game console 402 is coupled to the display device 410 via a wired communication medium, such as a High-Definition Media Interface (HDMI®) cable or a wireless connection. Examples of wireless connections used herein include a Wi-Fi® connection or a Bluetooth® connection. Also, the handheld controller 414 is coupled to the game console 402 via a wired or wireless connection. Examples of wired connections used herein include a serial transfer cable, a parallel transfer cable, and a Universal Serial Bus (USB) cable.
[0058] One example of a client device includes a combination of a handheld controller, a game console, and a display device. Another example of a client device includes a combination of a handheld controller and a display device. Yet another example of a client device includes a display device such as a smart television or a head mounted display. Another example of a client device includes a computer or a smartphone.
[0059] User 1 logs into his / her user account 1 once his / her user identification (ID) and password are authenticated by server system 404. User account 1 is assigned user ID 1 by server system 1. Once user 1 logs into user account 1, user 1 can access multiple game titles, such as game title G1, game title Ga, game title G2, and so on through game title GN. Game titles G1, G2, and so on through game title GN are examples of legacy game titles. Game title Ga is a game title that is not a legacy game. Rather, game title Ga is a current game, such as Fortnite®, that cannot be played on legacy machines.
[0060] After logging in to user account 1, user 1 selects one or more buttons on handheld controller 414 to select game title GN for playing legacy game N. When user 1 selects game title GN, a user input 418 indicating the selection is transmitted from handheld controller 414 to server system 404 via game console 402 and computer network 408. As an example, the user input is an input signal. Upon receiving user input 418 indicating the selection of game title GN, server system 404 identifies game code gcN based on user input 418. For example, server system 404 identifies that game code gcN has the same game title as game title GN and the selection is indicated in user input 418.
[0061] The server system 404 transmits the game code gcN to the game console 402 via a computer network 408. Upon receiving the game code gcN, the emulation processor system 409 stores the game code gcN in the memory device 406 of the game console 402.
[0062] When a user input 419 is received from the handheld controller 414 via a wireless connection during play of the legacy game N having the game code gcN, the input processor system 407 converts the user input 419 into a user input 420 and provides the user input 420 to the emulation processor system 409. As an example, the user input 419 includes a request to move a fighter jet from a position P1 to a position P2 or from one direction to another direction during play of the legacy game N. The fighter jet is an example of a virtual object. Upon receiving the user input 420, the emulation processor system 409 executes the basic block compiler 104 to generate a portion of the game code GCN from the portion of the game code gcN stored in the memory device 406. The portion of the game code GCN is generated based on the user input 420. For example, when the user input 420 includes a request to move a fighter from position P1 to position P2 during the play of the legacy game N, the basic block compiler 104 parses the game code gcN to identify instructions to calculate from position P1 to position P2. The basic block compiler 104 converts the instructions into basic blocks of the game code GCN, which are then executed to change the position of the fighter from P1 to P2. Furthermore, in this example, when the position changes from P1 to P2, the virtual environment in which the fighter is at position P1 is changed to a virtual environment in which the fighter is at position P2. In this example, the basic blocks of the game code GCN are executed by the GPU of the emulation processor system 409 to generate an image frame 422. Explainingly, the image frame 422 is displayed on the display device 410 to display the virtual environment with the fighter at position P2. In this manner, most or all of the game code GCN is compiled by the basic block compiler 104 and stored in the cache 102 for execution. As an example, a virtual environment, such as a virtual scene, may include one or more virtual reality (VR) images or one or more augmented reality (AR) images.
[0063] In one embodiment, communication of data between the server system 404 and the game console 402 occurs via a network communication protocol, such as the Transmission Control Protocol over Internet Protocol (TCP / IP). For example, the server system 404 includes a network interface controller (NIC) for converting data into packets. The network interface controller of the server system 404 is coupled to the memory device 412 to receive data from the memory device 412. Upon receiving the data from the memory device 412, the network interface controller of the server system 404 embeds the data into one or more packets by applying a network communication protocol to the data. The one or more packets are transferred from the network interface controller of the server system 404 to the game console 402 via the computer network 408. The network interface controller of the game console 402 extracts the data from the one or more packets by applying the network communication protocol. The network interface controller of the game console 402 is coupled to the emulation processor system 409. The network interface controller of the game console 402 provides the data received from the computer network 408 to the emulation processor system 409. Additionally, the network interface controller of the game console 402 receives data from the emulation processor system 409, embeds the data within one or more packets by applying a network communication protocol, and transmits the one or more packets to the server system 404 via the computer network 408. The network interface controller of the server system 404 applies the network communication protocol to the one or more packets received from the computer network 408 to extract data from the one or more packets, and transmits the data to the memory device 412 for storage.Examples of network interface controllers include a network interface card, an Ethernet card, and a network adapter. Illustratively, the network interface controller of the game console 402 enables the input processor system 407 to communicate with a computer network 408.
[0064] In one embodiment, in addition to or instead of the computer network 408, a cellular network is used to communicate data between the server system 404 and the game console 402. For example, the communication between the server system 404 and the game console 402 is facilitated using wireless technology. Wireless technology includes, for example, 4G or 5G wireless communication technology. As used herein, 5G is the fifth generation of cellular network technology. Also, a 5G network is a digital cellular network in which the service area covered by a provider is divided into small geographic areas called cells. In 5G wireless communication technology, analog signals representing sound and images are digitized by the phone, converted by an analog-to-digital converter, and transmitted as a stream of bits. All 5G wireless devices within a cell communicate by radio waves of a local antenna array and low-power automatic transceivers (transmitters and receivers) within the cell over frequency channels assigned by the transceivers from a pool of frequencies reused in other cells. The local antennas are connected in the cellular network by high-bandwidth optical fiber or wireless backhaul connections. As with other cell networks, a mobile device moving from one cell to another is automatically transferred to the new cell. It should be understood that a 5G network is merely one exemplary type of communication network, and embodiments of the present disclosure may utilize previous generations of wireless or wired communications, such as 3G or 4G, as well as subsequent generations of wired or wireless technologies following 5G.
[0065] In one embodiment, either the game console 402 or the server system 404 is referred to herein as a computing device. Other examples of computing devices include tablets, smartphones, laptop computers, desktop computers, and smart televisions.
[0066] In one embodiment, each of the game codes gc1 to gcN is stored in a separate memory device of the server system 404 or the legacy machine. For example, the game code gc1 is stored in a memory device of a first legacy machine, and the game code gc2 is stored in a memory device of a second legacy machine. As another example, the game code gc1 is stored in a first memory device of the server system 404, and the game code gc2 is stored in a second memory device of the server system 404.
[0067] In one embodiment, memory device 412 or memory device 406 is not a cache. Rather, each of memory device 412 or memory device 406 is a main memory, such as a RAM.
[0068] In one embodiment, the memory device 412 is coupled to a memory controller. The memory controller reads data from and writes data to the memory device 412. The memory controller is coupled to a network interface controller of the server system 404. The memory controller sends data received from the network interface controller of the server system 404 to the memory device 412 for storage. The memory controller also sends data received from the memory device 412 to the network interface controller of the server system 404 for transmission over the computer network 408 to the game console 402.
[0069] 4B is a diagram of an embodiment of a system 450 showing that the emulation processor system 409 and the input processor system 407 are located in a server system 404 and that image frames 422 are transmitted from the server system 404 over a computer network 408 to a display device 410 for displaying the virtual environment or scene. The system 450 includes the server system 404, the display device 410, and a handheld controller 414.
[0070] The server system 404 includes a memory device 412, an input processor system 407, and an emulation processor system 409. The memory device 412 is coupled to the emulation processor system 409, and the input processor system 407 is coupled to the emulation processor system 409. The display device 410 is coupled to the computer network 408 via a network interface controller of the display device 410. The display device 410 includes a processor coupled to the network interface controller of the display device 410. The processor of the display device 410 receives user input 419 during play of a legacy game N having a game title GN and a game code gcN, and transmits the user input 419 to the network interface controller of the display device 410. The network interface controller of the display device 410 transmits the user input 419 to the input processor system 407 of the server system 404 via the computer network 408.
[0071] Upon receiving the user input 419, the input processor system 407 converts the user input 419 into a user input 420 and provides the user input 420 to the emulation processor system 409. Upon receiving the user input 420, the emulation processor system 409 performs the same functions described above with reference to FIG. 4A with respect to the game code gcN to compile the basic blocks 1-n for the generation of the image frame 422. The server system 404 transmits the image frame 422 to the display device 410 via the computer network 408 to display a virtual environment, such as the virtual environment 452, on the display screen of the display device 410. For example, the virtual environment 452 includes a virtual object 454, which is an example of a fighter jet. In this example, the virtual environment 452 includes a virtual background that includes one or more virtual objects, such as a virtual pyramid 455 and a virtual structure 456. In this example, the virtual object 454 can fire a virtual missile at the virtual pyramid 455 and the virtual structure 456 during the play of the legacy game N having the game code gcN.
[0072] In one embodiment, communication of data between the server system 404 and the display device 410 occurs via a network communication protocol. For example, the server system 404 includes a network interface controller that converts data into packets. The network interface controller of the server system 404 is coupled to the emulation processor system 409 to receive data from the emulation processor system and embeds the data into one or more packets by applying the network communication protocol. The packets are transferred from the network interface controller of the server system 404 to the display device 410 via the computer network 408. The network interface controller of the display device 410 extracts the data from the one or more packets by applying the network communication protocol. The network interface controller of the display device is coupled to the processor of the display device 410. The network interface controller of the display device provides the data received from the computer network 408 to the processor of the display device 410. The processor of the display device 410 renders the data, such as an image frame 422, on a display screen of the display device 410. Additionally, the network interface controller of the display device 410 receives data from the processor of the display device 410, embeds the data within one or more packets by applying a network communication protocol, and transmits the one or more packets via the computer network 408 to the server system 404. The network interface controller of the server system 404 applies the network communication protocol to the one or more packets received from the computer network 408 to extract data from the one or more packets, and transmits the data to the emulation processor system 409.
[0073] In one embodiment, in addition to or instead of the computer network 408, a cellular network is used to communicate data between the server system 404 and the display device 410. For example, communication between the server system 404 and the display device is facilitated using wireless technology.
[0074] In one embodiment, a head mounted display (HMD) is used instead of the display device 410. The head mounted display is worn on the head of the user 1 and includes a display screen such as an LED screen, an OLED screen, an LCD screen, etc. The HMD performs the same functions as those performed by the display device 410.
[0075] FIG. 5A is a diagram illustrating an embodiment of a basic block. Each basic block includes a source register address, a destination register address, and an operation. For example, basic block 1 includes source register address 1, destination register address 1, and operation 1. Basic block 2 includes source register address 2, destination register address 2, and operation 2, and basic block n includes source register address n, destination register address n, and operation n. As an example, the source register address is an address of one or more source registers in the cache 102, and the destination register address is an address of one or more destination registers in the cache 102. Examples of operations of a basic block include a jump operation, a branch operation, a read operation, a write operation, a compare operation, and a return operation. Further examples of operations of a basic block include arithmetic operations such as an add operation, a subtract operation, a multiply operation, and a divide operation.
[0076] As an example, when operation n is a read operation, data is read from source register address n and basic block n is executed. As another example, when operation n is a write operation, data is written to destination register address n and basic block n is executed. As another example, when operation n is a move operation, data is read from source register address n and operation n is performed on the data and data is written to destination register address n and basic block n is executed. As yet another example, when operation n is a compare operation, a first value of data stored in a first source register address listed in basic block n is compared to a second value of data stored in a second source register address listed in basic block n to generate a comparison result and the comparison result is stored in destination register address n and basic block n is executed. As another example, when operation n is an add operation, a first value of data stored in a first source address listed in basic block n is added to a second value of data stored in a second source address listed in basic block n to generate an addition result and the addition result is stored in destination register address n and basic block n is executed. As yet another example, when a virtual object described herein is to move from a position P1 to a position P2, and the operation n is a write operation in which the position of the virtual object is updated from P1 to P2, the position P1 at the destination register address n is overwritten with the position P2 to execute the basic block n. In this example, the execution of the basic block n indicates to the emulation processor system 409 that the virtual object is to move from a position P1 to a position P2. Also in this example, the user input 420 (FIG. 4A) instructs the emulation processor system 409 to move the virtual object from a position P1 to P2. Similarly, as another example, when a virtual object described herein is to move from a direction O1 to a direction O2, and the operation n is a write operation in which the direction of the virtual object is updated from O1 to O2, the direction O1 at the destination register address n is overwritten with the direction O2 to execute the basic block n.In this example, execution of basic block n indicates to the emulation processor system 409 that the virtual object is to move from direction O1 to direction O2. Further, in this example, user input 420 instructs the emulation processor system 409 to move the virtual object from direction O1 to direction O2.
[0077] As yet another example, when a portion of a virtual object described herein is to change color from red to green and operation n is a write operation in which the color of the virtual object is updated from red to green, data representing red at destination register address n is overwritten with data representing green and basic block n is executed. In this example, execution of basic block n indicates to emulation processor system 409 that the color of a portion of the virtual object is to change from red to green. Also in this example, user input 420 instructs emulation processor system 409 to change the color of the portion of the virtual object from red to green. In a similar manner, other parameters such as intensity and texture can be modified based on user input 420. Intensity and texture are examples of legacy graphics parameters.
[0078] Each basic block includes a cycle count for the basic block execution. For example, basic block 1 includes a cycle count for the basic block execution of 1. As another example, basic block 2 includes a cycle count for the basic block execution of 2, and basic block n includes a cycle count for the basic block execution of n. As an example, an estimated cycle count for the basic block execution is estimated by the basic block compiler 104 (FIG. 1) when compiling the basic block. In this example, the estimated cycle count is stored in the basic block. Also in this example, after the basic block is executed by the block dispatcher 302 (FIG. 3), the block dispatcher 302 updates the estimated execution cycle count with a real number in the manner described above and provides the real number to the block compiler 104. The estimated cycle count is replaced by the actual count for the basic block by the block compiler 104. As another example, an actual count of the cycle count for the execution of operation n is generated by the block dispatcher 302 and stored in basic block n. In this example, there is no estimation of the cycle count for the execution of operation n.
[0079] Additionally, as another example, one or more of basic blocks 1-n may include an invalid mark indicating that one or more of basic blocks 1-n is to be checked for validity, e.g., basic block n includes an invalid mark n.
[0080] It should be noted that by converting the emulated PU code 106 into basic blocks 1-n of the game code GCN, a hook such as a hook block can be inserted between any two of the basic blocks 1-n. For example, a hook block n can be inserted between basic blocks (n-1) and n. The hook block n has the same structure as the structure of the basic block n. For example, the hook block includes a source register address, a destination register address, an operation, and the number of execution cycles of the operation of the hook block. As an example, due to security issues associated with legacy machines, the hooks described herein cannot be inserted between instructions of the emulated PU code 106 (FIG. 1) stored in a legacy machine for execution on the legacy machine.
[0081] Further note that basic blocks 1-n are keyed into cache 102, such as being pinned therein (FIG. 1). For example, basic block 1 has a starting memory address 1 that indicates where basic block 1 begins in cache 102. Basic block 1 also has an ending memory address 1 that indicates where basic block 1 ends in cache 102. As another example, the ending address 1 of basic block 1 is indicated by an offset in cache 106 from starting memory address 1. As yet another example, basic block 2 has a starting memory address 2 that indicates where basic block 2 begins in cache 102. Basic block 2 also has an ending memory address 2 that indicates where basic block 2 ends in cache 102. As another example, the ending address 2 of basic block 2 is indicated by an offset in cache 106 from starting memory address 2. Similarly, as another example, basic block n has a starting memory address n that indicates where basic block n begins in cache 102. Basic block n also has an ending memory address n that indicates where basic block n ends in cache 102. As another example, the end address n of basic block n is indicated by an offset in cache 106 from the starting memory address n. From the starting and ending memory addresses of basic blocks 1-n stored in cache 102, an emulation processor system 409 (FIG. 4A), such as basic block compiler 102, can determine the location of basic blocks 1-n in cache 102.
[0082] Also note that if user input indicates that block dispatcher 302 of the updated machine should execute basic block n immediately after executing basic block 1, block dispatcher 302 skips execution of basic blocks 2 through (n-1) in cache 102 and jumps from basic block 1 to basic block n. In this case, execution of basic block 1 is closed by block dispatcher 302 when block dispatcher 302 jumps to basic block n. Also, in this case, the start address of the next basic block is contiguous with the end address of the previous basic block. For example, start address 2 is contiguous with end address 1, and start address n is contiguous with end address (n-1) of basic block (n-1).
[0083] In one embodiment, source register addresses 1-n are memory addresses of registers in cache 102 and destination register addresses 1-n are memory addresses of registers within cache 102.
[0084] In one embodiment, a basic block includes multiple operations. For example, basic block n includes a first operation, a first source register address, and a first destination register address. Basic block n further includes a second operation, a second source register address, and a second destination register address.
[0085] In one embodiment, a basic block includes an operation, a number of source addresses, and a destination address.
[0086] In one embodiment, a basic block includes an operation, multiple destination addresses, and a source address.
[0087] In one embodiment, a basic block includes multiple operations, multiple source addresses, and multiple destination addresses.
[0088] In one embodiment, a basic block includes one or more operations, one or more source addresses, and one or more destination addresses.
[0089] In one embodiment, a basic block contains either a source register address or a destination register address, but not both.
[0090] In one embodiment, the block compiler 102 does not estimate the number of cycles for the execution of basic block n. Rather, the block dispatcher 302 generates an actual count of the number of cycles for the execution of basic block n and stores the actual count in basic block n.
[0091] FIG. 5B is a diagram illustrating an embodiment of a compilation operation performed by the basic block compiler 104 (FIG. 1). An example of an emulated PU code instruction M is shown as instruction 550, and an example of a basic block n is shown as basic block 552, where M is a positive integer. The instruction 550 includes a source address M having a length of a bits, a destination address M having a length of b bits, and an operation M represented by bits of length c, where a, b, and c are positive integers. As an example, a is 4, b is 4, and c is 32. As an example, the a bits, b bits, and c bits are stored in one or more memory devices of the legacy machine. For example, the a bit is stored in memory device 406 or 412 (FIGS. 4A and 4B). The operation 552 includes a source register address n having a length of d bits, a destination address n having a length of e bits, and an operation n represented by bits of length f, where d, e, and f are positive integers. As an example, d is 8, e is 8, and f is 64. As an example, the d, e, and f bits are stored in one or more registers of the updated machine. As an example, d is greater than a, e is greater than b, and f is greater than c. To explain, when a is 4, b is 4, and c is 32, d is 8, e is 8, and f is 64. As another example, when a is 4, b is 4, and c is 32, d is 16, e is 16, and f is 128.
[0092] To perform a compilation operation, the basic block compiler 104 (FIG. 1) converts the source address M to a source register address n, converts the destination address M to a destination register address n, and converts the operation M to an operation n to generate a basic block n from the emulated PU code instruction M. For example, the basic block compiler 104 shifts four bits of the source address M to the right to occupy four memory addresses of the eight source registers in the cache 102 and masks any bits of the remaining four memory addresses of the eight source registers. The four memory addresses occupied by the four bits of the source address M are in the lowest positions of the eight source registers in the cache 102, and the remaining four memory addresses whose bits are masked are in the eight most significant positions of the source registers in the cache 102.
[0093] As another example, the basic block compiler 104 shifts four bits of the destination address M to the right to occupy four memory addresses of the eight destination registers in the cache 102 and masks any bits of the remaining four memory addresses of the eight destination registers. The four memory addresses occupied by the four bits of the destination address m are in the lowest positions of the eight destination registers in the cache 102, and the remaining four memory addresses whose bits are masked are in the highest positions of the eight destination registers in the cache 102.
[0094] Similarly, as another example, the basic block compiler 104 shifts 32 bits of the operation M to the right to occupy 32 memory addresses of the 64 operation registers in the cache 102 and masks any bits of the remaining 32 memory addresses of the 64 operation registers. The 32 memory addresses occupied by the 32 bits of the operation M are in the lowest positions of the 32 operation registers in the cache 102, and the remaining 32 memory addresses whose bits are masked are in the highest positions of the 64 operation registers in the cache 102. The operation n is stored in the operation register of the cache 102.
[0095] 6A is a diagram of an embodiment of a system 600 to illustrate components of the emulation processor system 409. The system 600 includes a memory device 412 and an emulation processor system 409. The basic block compiler 104 of the emulation processor system 409 includes a parser or decoder 602. The basic block compiler 104 further includes a block creator 604, a block cacher 606A, and a block reader 608. The emulation processor system 409 includes a block dispatcher 302.
[0096] As an example, each of the parser 609, the block creator 604, the block cacher 606, the block reader 608, and the block dispatcher 302 is implemented using software, hardware, or a combination thereof. For example, each of the parser 609, the block creator 604, the block cacher 606, the block reader 608, and the block dispatcher 302 is a separate integrated circuit, such as a PLD or an ASIC or a controller or a processor or a portion of a computer program. As another example, each of the parser 609, the block creator 604, the block cacher 606, the block reader 608, and the block dispatcher 302 is a separate computer software program.
[0097] The game code gcN includes multiple instructions: emulated PU code instruction 1, emulated PU code instruction 2, and so on up to emulated PU code instruction M. For example, each instruction of the game code gcN is a series of bits that can be executed by a processor of a legacy machine to implement a function such as moving a virtual object from position P1 to position P2, or changing the orientation of a virtual object from O1 to O2, or modifying a parameter of a portion of a virtual object.
[0098] The parser 602 is coupled to a block creator 604, which is coupled to a block cacher 606 and a block reader 608. The block cacher 606 and the block reader 608 are coupled to the cache 102. The block reader 608 is coupled to the block dispatcher 302.
[0099] FIG. 6B is a flow chart illustrating an embodiment of a method 650 for compiling and executing basic blocks 1-n. Method 650 is illustrated using system 600 of FIG. 6A. In operation 652 of method 650, block creator 604 determines whether any user input, such as user input 1, is received. As an example, user input 1 is a signal generated based on one or more identifications of one or more buttons of handheld controller 414 (FIG. 4A) selected by user 1. As another example, user input 1 is a signal generated based on a measure of movement, such as pitch, yaw, and roll, relative to the origin of an xyz coordinate system centered on handheld controller 414. As yet another example, user input 1 is a signal generated based on a measure of movement, such as pitch, yaw, and roll, relative to the origin of an xyz coordinate system of the HMD. An example of user input 1 is user input 420 (FIG. 4A).
[0100] If the block creator 604 determines that user input has not been received, it continues by checking whether user input has been received in operation 652. For example, the basic block compiler 104 does not execute any of basic blocks (n+1) through p stored in the cache 102 (FIG. 1), and the basic block compiler 104 (FIG. 1) does not identify any of the emulated PU code instructions 1 through M (FIG. 6A), where p is a positive integer, when no user input has been received in operation 652. It is assumed that basic blocks 1 through n have not been generated and stored in the cache 102 at the time that user input is received in operation 652.
[0101] Meanwhile, in response to determining that user input has been received in operation 652, in operation 654 of method 650, block creator 604 determines whether one or more of basic blocks (n+1)-p stored in cache 102 satisfy the user input. For example, block creator 604 sends a request to block reader 608 to access basic block (n+1)-p from cache 102. In this example, upon receiving the request, block reader 608 reads basic block (n+1)-p from cache 102 and sends basic block (n+1)-p to block creator 604. Further, in this example, block creator 604 determines whether one or more functions of basic blocks (n+1)-p satisfy user input 1, such as processing. Specifically, when user input 1 indicates that the position of a virtual object is to be changed from position P1 to position P2, block creator 604 determines whether any of basic blocks (n+1) to p includes an operation of overwriting position P1 with position P2. If block creator 604 determines that one or more basic blocks (n+1) to p include an operation of overwriting position P1 with P2, block creator 604 determines that one or more functions of one or more of blocks (n+1) to p satisfy user input 1. On the other hand, if block creator 604 determines that none of basic blocks (n+1) to p includes an operation of overwriting position P1 with P2, block creator 604 determines that the functions of basic blocks (n+1) to p do not satisfy user input 1.
[0102] As another explanation, when the user input 1 is to change the direction of the virtual object from the direction O1 to the direction O2, the block creator 604 determines whether any of the basic blocks (n+1) to p includes an operation of overwriting the direction O1 with the direction O2. If the block creator 604 determines that one or more of the basic blocks (n+1) to p include an operation of overwriting the direction O1 with O2, the block creator 604 determines that one or more functions of one or more of the basic blocks (n+1) to p satisfy the user input 1. On the other hand, if the block creator 604 determines that none of the basic blocks (n+1) to p includes an operation of overwriting the direction O1 with O2, the block creator 604 determines that the functions of the basic blocks (n+1) to p do not satisfy the user input 1. As yet another explanation, when the user input 1 is to change the value of a parameter of a part of the virtual object from a first value to a second value, the block creator determines whether any of the basic blocks (n+1) to p includes an operation of overwriting the first value with the second value. When the block creator 604 determines that one or more of the basic blocks (n+1) to p include an operation of overwriting a first value with a second value, the block creator 604 determines that one or more functions of the one or more of the basic blocks (n+1) to p satisfy the user input 1. On the other hand, when the block creator 604 determines that none of the basic blocks (n+1) to p includes an operation of overwriting a first value with a second value, the block creator 604 determines that the functions of the basic blocks (n+1) to p do not satisfy the user input 1.
[0103] Upon determining that one or more functions of basic blocks (n+1)-p satisfy user input 1, in operation 656 of method 600, block dispatcher 302 executes one or more of basic blocks (n+1)-p. For example, upon determining that one or more functions of basic blocks (n+1)-p satisfy user input 1, block creator 604 sends instructions to block dispatcher 302 to execute one or more of basic blocks (n+1)-p. In this example, in response to receiving the instructions, block dispatcher 302 sends a command to block reader 608 to read one or more of basic blocks (n+1)-p from cache 102 to satisfy the user input received in operation 652. Also in this example, in response to receiving one or more of basic blocks from cache 102 in response to the command, block dispatcher 302 executes one or more of basic blocks (n+1)-p.
[0104] On the other hand, if it is determined that the functionality of blocks (n+1) through p does not satisfy the user input 1, then in operation 658 of method 600, the block compiler 102 identifies one or more of emulated PU code instructions 1 through M for processing the user input received in operation 652. For example, if it is determined that the functionality of blocks (n+1) through p does not satisfy the user input 1, the block creator 604 sends a request to the parser 602 to parse the game code gcN to identify and obtain one or more of emulated PU code instructions 1 through M of the game code gcN that satisfy the functionality identified in the user input 1. In this example, in response to receiving the request, the parser 602 accesses the memory device 412 to parse the game code gcN and determines whether the functionality of one or more of the emulated PU code instructions 1-M satisfies the user input 1, and if it determines that the functionality of one or more of the emulated PU code instructions 1-M satisfies the user input 1, the parser 602 provides one or more of the emulated PU code instructions 1-M to the block creator 604.
[0105] To explain, when the user input 1 is for changing the position of the virtual object from position P1 to position P2, the parser 602 determines whether any of the emulated PU code instructions 1 to M includes an operation of overwriting the position P1 with the position P2. If the parser 602 determines that one or more of the emulated PU code instructions 1 to M include an operation of overwriting the position P1 with P2, the parser 602 determines that one or more functions of one or more of the emulated PU code instructions 1 to M satisfy the user input 1. On the other hand, if the parser 602 determines that none of the emulated PU code instructions 1 to M includes an operation of overwriting the position P1 with P2, the parser 602 determines that the functions of the emulated PU code instructions 1 to M do not satisfy the user input 1.
[0106] As another explanation, when the user input 1 is to change the direction of the virtual object from the direction O1 to the direction O2, the parser 602 determines whether any of the emulated PU code instructions 1 to M includes an operation of overwriting the direction O1 with the direction O2. If the parser 602 determines that one or more of the emulated PU code instructions 1 to M includes an operation of overwriting the direction O1 with O2, the parser 602 determines that one or more functions of one or more of the emulated PU code instructions 1 to M satisfy the user input 1. On the other hand, if the parser 602 determines that none of the emulated PU code instructions 1 to M includes an operation of overwriting the position O1 with O2, the parser 602 determines that the functions of the emulated PU code instructions 1 to M do not satisfy the user input 1. As yet another example, when the user input 1 is to change the value of a parameter of a part of the virtual object from a first value to a second value, the parser 602 determines whether any of the emulated PU code instructions 1 to M includes an operation of overwriting the first value with the second value. When determining that one or more of the emulated PU code instructions 1-M include an operation of overwriting a first value with a second value, the parser 602 determines that one or more functions of one or more of the emulated PU code instructions 1-M satisfy the user input 1. On the other hand, when determining that none of the emulated PU code instructions 1-M includes an operation of overwriting a first value with a second value, the parser 602 determines that the functions of the emulated PU code instructions 1-M do not satisfy the user input 1.
[0107] In operation 660 of method 600, when one or more of emulated PU code instructions 1-M that satisfy user input 1 are received from parser 602, block creator 604 applies the compilation operations described above to generate one or more basic blocks 1-n from one or more of emulated PU code instructions 1-M. For example, block creator 604 compiles one or more basic blocks 1-n, sends one or more basic blocks 1-n to block cacher 606, and sends instructions to block dispatcher 302 to execute one or more of basic blocks 1-n. Upon receiving one or more basic blocks 1-n, block cacher 606 stores one or more basic blocks 1-n in cache 102.
[0108] Upon receiving an instruction from the block creator 604 to execute one or more of the basic blocks 1-n in operation 662 of the method 600, the block dispatcher 302 executes the one or more of the basic blocks 1-n to process the user input received in operation 652. For example, the block dispatcher 302 sends a request to the block reader 608 to read one or more of the basic blocks 1-n from the cache 102. Upon receiving the command, the block reader 608 reads one or more of the basic blocks 1-n from the cache 102 and provides the one or more of the basic blocks 1-n to the block dispatcher 302. Upon receiving one or more of the basic blocks 1-n, the block dispatcher 302 executes one or more of the basic blocks 1-n to generate the virtual environment 452 (FIG. 4B). To illustrate, the block dispatcher 302 executes one or more of the basic blocks 1-n to generate an image frame, which includes virtual environment data, such as a position and orientation of the virtual object 454, and parameters, such as legacy graphic parameters, of the virtual object 454, and positions and orientations of other virtual objects in the virtual environment 452, and parameters, such as legacy graphic parameters, of other virtual objects in the virtual environment 452. Further, in this example, the virtual environment changes when the position or orientation of the virtual object 454, or a combination thereof, changes. The block dispatcher 302 provides the image frame to a GPU of the emulation processor system 409 to render and display the virtual environment 452 on a display screen of the display device 410 (FIG. 4A). As another example, the block dispatcher 302 executes one or more of the basic blocks 1-n to generate one or more audio frames including legacy audio parameters, and provides the one or more audio frames to a client device, which outputs sound based on the one or more audio frames. Illustratively, each audio frame is an audio file that includes one or more legacy audio parameters. After user input 1 is received in operation 652, method 600 is repeated when another user input, such as user input 2, is received.
[0109] In one embodiment, the image frames are provided from the block dispatcher 302 to a GPU of the display device 410 to display the virtual environment 452 on the display screen of the display device 410 .
[0110] FIG. 6C is a diagram of an embodiment of a system 670 for illustrating that different basic blocks or different sets of basic blocks are dynamically compiled for different user inputs. The system 670 includes a memory device 412, a basic block compiler 104, a block dispatcher 302, and a cache 102. When the basic block compiler 104 receives a user input 1, the basic block compiler 104 determines whether a basic block 1 corresponding to the user input 1 is stored in the cache 102. For example, when the basic block 1 includes a function for processing the user input 1, the basic block 1 corresponds to the user input 1. To explain, when the user input 1 is to move a virtual object from a position P1 to a position P2, and the basic block 1 includes an operation of updating the position P1 stored in the destination register address 1 with the position P2, the basic block 1 can process the user input 1. In this figure, the user input 1 is a signal indicating the selection of a right button, a left button, an up button, or a down button on the handheld controller 414 (FIG. 4A). Similarly, when basic block 2 does not include a function for processing user input 1, basic block 2 does not correspond to user input 1. To explain, when user input 1 is for moving a virtual object from position P1 to position P2 without changing the orientation of the virtual object, and basic block 2 includes an operation for updating the orientation O1 stored in destination register 2 with the orientation O2, basic block 2 cannot process user input 1. Upon determining that basic block 1 capable of processing user input 1 is stored in cache 102, basic block compiler 104 provides basic block 1 to block dispatcher 302 for execution.
[0111] On the other hand, when it is determined that the basic block 1 is not stored in the cache 102, the basic block compiler 104 analyzes the emulated PU code 106 stored in the memory device 412 to identify the emulated PU code instruction 1 corresponding to the user input 1. For example, the basic block compiler 104 analyzes the emulated PU code 106 to identify that the emulated PU code instruction 1 satisfies (e.g., processes) the user input 1. To explain, when the user input 1 is for moving a virtual object from a position P1 to a position P2, and the emulated PU code instruction 1 includes a function for updating the position P1 stored in the destination address 1 with the position P2, the emulated PU code instruction 1 can process the user input 1. Similarly, when the emulated PU code instruction 2 does not include a function for processing the user input 1, the emulated PU code instruction 2 does not correspond to the user input 1. To explain, when user input 1 is for moving a virtual object from position P1 to position P2 without changing the orientation of the virtual object, and emulated PU code instruction 2 includes an operation of updating direction O1 stored in destination address 2 with direction O2, the emulated PU code instruction 2 cannot process user input 1.
[0112] Upon determining that emulated PU code instruction 1 can process user input 1 and emulated PU code instruction 2 cannot process user input 1, basic block compiler 104 accesses (e.g., reads) emulated PU code instruction 1 from memory device 412 and compiles basic block 1 from emulated PU code instruction 1. Basic block compiler 102 does not compile basic block 2 from emulated PU code instruction 2 in response to receiving user input 1. Basic block compiler 104 stores basic block 1 in cache 102 and sends instructions to block dispatcher 302 to access and execute basic block 1. Upon receiving the instructions, block dispatcher 302 reads basic block 1 from cache 102 and executes basic block 1.
[0113] Similarly, when the basic block compiler 104 receives the user input 2, the basic block compiler 104 determines whether the basic block 2 corresponding to the user input 2 is stored in the cache 102. For example, when the basic block 2 includes a function for processing the user input 2, the basic block 2 corresponds to the user input 2. To explain, when the user input 2 is to move a virtual object from a direction O1 to a direction O2, and the basic block 2 includes an operation for updating the direction O1 stored in the destination register address 2 with the direction O2, the basic block 2 can process the user input 2. In this figure, the user input 2 is a signal indicating the selection of a button for rotating clockwise or a button for rotating counterclockwise on the handheld controller 414. Similarly, when the basic block 1 does not include a function for processing the user input 2, the basic block 1 does not correspond to the user input 2. To explain, when user input 2 is for moving a virtual object from direction O1 to direction O2 without changing the position of the virtual object, and basic block 1 includes an operation of updating position P1 stored in destination register 1 with position P2, basic block 1 cannot process user input 2. Upon determining that basic block 2 capable of processing user input 2 is stored in cache 102, basic block compiler 104 provides basic block 2 to block dispatcher 302 for execution.
[0114] On the other hand, when it is determined that the basic block 2 is not stored in the cache 102, the basic block compiler 104 analyzes the emulated PU code 106 stored in the memory device 412 to identify an emulated PU code instruction 2 corresponding to the user input 2. For example, the basic block compiler 104 analyzes the emulated PU code 106 to identify that the emulated PU code instruction 2 satisfies (e.g., processes) the user input 2. To explain, when the user input 2 is for moving a virtual object from a direction O1 to a direction O2, and the emulated PU code instruction 2 includes a function for updating the direction O1 stored in the destination address 2 with the direction O2, the emulated PU code instruction 2 can process the user input 2. Similarly, when the emulated PU code instruction 1 does not include a function for processing the user input 2, the emulated PU code instruction 1 does not correspond to the user input 2. To explain, when the user input 2 is for moving a virtual object from direction O1 to direction O2 without changing the orientation of the virtual object, and the emulated PU code instruction 1 includes an operation of updating the position P1 stored in the destination address 1 with the position P2, the emulated PU code instruction 1 cannot process the user input 2.
[0115] Upon determining that emulated PU code instruction 2 can process user input 2 and emulated PU code instruction 1 cannot process user input 2, basic block compiler 104 accesses (e.g., reads) emulated PU code instruction 2 from memory device 412 and compiles emulated PU code instruction 2 to generate basic block 2. Basic block compiler 102 does not compile emulated PU code instruction 1 in response to receiving user input 2. Basic block compiler 104 stores basic block 2 in cache 102 and sends instructions to block dispatcher 302 to access and execute basic block 2. Upon receiving the instructions, block dispatcher 302 reads basic block 2 from cache 102 and executes basic block 2.
[0116] 7A is a diagram of an embodiment of a system 700 to illustrate removal of emulated PU code 106 from a memory device 412. The system 700 includes a block creator 604, an instruction remover 702, and a memory device 412. Examples of the instruction remover 702 include a processor, an ASIC, a PLD, a computer program, a portion of a computer program, and a microcontroller. The instruction remover 702 is coupled to the block creator 604 and coupled to the memory device 412.
[0117] FIG. 7B is a flow chart illustrating an embodiment of a method 720 for deleting the emulated PU code 106 (FIG. 6C) from the memory device 412 (FIG. 6C). The method 720 includes an operation 702 for determining whether all instructions of the emulated PU code 106 have been compiled. For example, the block creator 604 (FIG. 7A) determines whether all of the emulated PU code instructions 1-M of the game code gcN have been compiled. To illustrate, before compiling any of the emulated PU code instructions 1-M of the game code gcN, the block creator 604 sends a request to the processor of the legacy machine to obtain all identities of the emulated PU code instructions 1-M of the game code gcN. In this figure, the block creator 604 sends an identity of the game code gcN, such as one or more bits, to the processor of the legacy machine to obtain all identities of the emulated PU code instructions 1-M of the game code gcN. An example of the identity of the emulated PU code instructions 1-M includes one or more bits. Illustratively, the identity of the emulated PU code instruction M is represented by a first sequence of bits, and the identity of the emulated PU code instruction M-1 is represented by a second sequence of bits, which is different from the first sequence. Upon receiving the identities of the emulated PU code instructions 1-M from the processor of the legacy machine, the block creator 604 stores the identities in a table in the cache 102. As each of the emulated PU code instructions 1-M is compiled, the block creator 604 updates the table to include an indication identifying which of the emulated PU code instructions 1-M have been compiled. The block creator 604 determines whether all of the emulated PU code instructions 1-M have been compiled from the indication identifying which of the emulated PU code instructions 1-M have been compiled.
[0118] Upon determining that all of the emulated PU code instructions 1-M of the game code gcN have been compiled, the block creator 604 sends a command to the instruction remover 702 (FIG. 7A) to delete the game code gcN from the memory device 412. Upon receiving the command, at operation 724 of the method 720, the instruction remover 702 erases the emulated PU code instructions 1-M from the memory device 412.
[0119] On the other hand, if it is determined that one or more of the emulated PU code instructions 1-M of the game code gcN have not been compiled based on the table, then in operation 726 of the method 720, the block creator 604 does not send a command to the instruction remover 702 to remove the game code gcN from the memory device 412. The instruction remover 702 does not remove the emulated PU code instructions 1-M from the memory device 412 until it receives a command to remove the game code gcN.
[0120] In one embodiment, the block creator 604 determines whether the game code gcN stored in the cache 102 is accessed from the memory device 412 (FIG. 4A) within a predetermined period from the latest time that the game code gcN is accessed from the memory device 412. As an example, the block creator 604 can access an Internet clock via the computer network 408 to determine the latest time and the predetermined period. As another example, the block creator 604 includes a clock source, such as a clock oscillator, to count the latest time and the predetermined period. Upon determining that the game code gcN is not accessed from the memory device 412 within the predetermined period, the block creator 604 sends a command to the instruction remover 702 to remove the game code gcN from the memory device 412. As an example, when the game code gcN is not accessed within the predetermined period, a majority of the emulated PU code instructions 1-M are accessed by the basic block compiler 104 to compile one or more of the basic blocks 1-n. As another example, when the game code gcN is not accessed within a predetermined period of time, some of the emulated PU code instructions 1-M that are frequently used during the play of the legacy game N having the game title GN are accessed by the basic block compiler 104 to compile one or more of the basic blocks 1-n. On the other hand, upon determining that the game code gcN has been accessed within a predetermined period of time, the block creator 604 does not send a command to the instruction remover 702, and the game code gcN is not deleted from the memory device 412.
[0121] 8A is a diagram of an embodiment of an emulation processor system 800 to illustrate validation of basic blocks. The emulation processor system 800 is an example of the emulation processor system 409 (FIG. 4B). The emulation processor system 800 includes a basic block compiler 104, a cache 102, a block validator 802, and a block flagger 804. The block validator 802 is coupled to the block creator 604, the block dispatcher 302, the parser 602, and the block flagger 804. Each of the clock validator 802 and the block flagger 804 is coupled to the cache 102.
[0122] As an example, the block validator 802 is implemented as an ASIC, or a PLD, or a microcontroller, or a processor, or a computer program, or a portion of a computer program, and as an example, the block flagger 804 is implemented as an ASIC, or a PLD, or a microcontroller, or a processor, or a computer program, or a portion of a computer program.
[0123] In one embodiment, the terms ASIC, PLD, microcontroller, microprocessor, controller, and processor are used interchangeably herein.
[0124] FIG. 8B is a flow chart of one embodiment of a method 850 to illustrate a validation operation performed by emulation processor system 800 (FIG. 8A). In operation 852 of method 800, block validator 802 (FIG. 8A) determines whether basic block n has been compiled. For example, block validator 802 accesses (e.g., reads) cache 102 to determine whether a new basic block, such as basic block n, has been stored in cache 102 by comparing with basic blocks 1 through n-1 previously stored in cache 102. Block validator 802 previously identified basic blocks 1 through n-1 stored in cache 102. Upon determining that basic block n has not been cached, block validator 802 continues to access cache 102 to determine whether a new basic block has been cached. For example, block validator 802 periodically analyzes basic blocks 1 through n-1 stored in cache 102 to determine whether basic block n has been stored in cache 102.
[0125] Upon determining that basic block n is cached, in operation 854 of method 800, block validator 802 generates a first hash value from one or more of emulated PU code instructions 1-M from which basic block n is compiled. For example, block validator 802 sends a request to parser 602 to retrieve one or more of emulated PU code instructions 1-M from memory device 412. The request includes an identity of a memory address of basic block n in cache 102. The identity of the memory address of basic block n is received by block validator 602 from block creator 604, such as retrieved by a request. Upon receiving the request from block validator 802, parser 602 reads one or more of emulated PU code instructions 1-M from memory device 412 and provides one or more of emulated PU code instructions 1-M to block validator 802. To illustrate, when one or more of the emulated PU code instructions 1-M are provided to the basic block compiler 104 for compiling a basic block n, the parser 602 stores a one-to-one correspondence between one or more identities of one or more memory addresses occupied by the basic block n in the cache 102 and one or more identities of one or more memory addresses occupied by one or more of the emulated PU code instructions 1-M in the memory device 412. The parser 602 receives the identities of the memory addresses of the basic block n in the cache 102 from the block validator 802 and identifies one or more memory addresses of one or more of the emulated PU code instructions 1-M in the memory device 412 from the one-to-one correspondence. The parser 602 reads one or more of the emulated PU code instructions 1-M from the one or more memory addresses in the memory device 412 and provides one or more of the emulated PU code instructions 1-M to the block validator 802.
[0126] Continuing with this example, upon receiving one or more of the emulated PU code instructions 1-M from the parser 602, the block validator 802 generates a first hash value from the one or more of the emulated PU code instructions 1-M and stores the first hash value in the cache 102. Illustratively, the block validator 802 generates a digest or checksum from the one or more of the emulated PU code instructions 1-M that correspond to basic block n. In this example, the first hash value is stored in one or more registers of the cache 102 that are keyed to include basic block n.
[0127] In operation 856 of method 800, block validator 802 sends a command to parser 602 to determine whether one or more memory addresses have been overwritten in memory device 412 where one or more emulated PU code instructions 1-M are stored. One or more memory addresses in memory device 412 can be overwritten with data from a compact disc read only memory (CD-ROM) of the legacy machine. The data can be updates to one or more of emulated PU code instructions 1-M. The updates to one or more of emulated PU code instructions 1-M are referred to as updated instructions. Alternatively, the data can be corrupted, but is not an updated instruction. Upon receiving the command from block validator 802, parser 602 sends a request to a processor of the legacy machine to determine whether one or more memory addresses have been overwritten in memory device 412 where one or more of emulated PU code instructions 1-M are stored.
[0128] Upon receiving the request from parser 602, the processor of the legacy machine provides a response to the request, the response indicating whether one or more memory addresses have been overwritten in memory device 412 in which one or more of emulated PU code instructions 1-M are stored. Upon receiving a response that one or more memory addresses in memory device 412 in which one or more of emulated PU code instructions 1-M are stored have not been overwritten, block validator 802 sends a command to block flagger 804 (FIG. 8A) to not mark basic block n as invalid. Upon receiving the command, in operation 858 of method 800, block flagger 804 does not mark basic block n as invalid.
[0129] On the other hand, upon receiving a response that one or more memory addresses in memory device 412 where one or more of emulated PU code instructions 1-M are stored have been overwritten, in operation 860 of method 850, block validator 802 sends a command to block flagger 804 (FIG. 8A) to mark basic block n as invalid. Upon receiving the command to mark basic block n as invalid, block flagger 804 marks basic block n as invalid. For example, block flagger 804 accesses basic block n in cache 102 and includes an identifier, such as invalid mark n (FIG. 5A), in the memory address in cache 102 that has basic block n to indicate that basic block n is invalid. One example of an invalid mark n is a sequence of bits.
[0130] In one embodiment, the hash value is an example of a verification result.
[0131]
[0046] Figure 8C is a continuation of the flowchart of method 800 of Figure 8B. In operation 862 of method 800, block validator 802 (Figure 8A) determines whether basic block n should be executed. For example, block dispatcher 302 sends an indication to block validator 802 that basic block n should be executed. Upon receiving an indication from block dispatcher 302 that basic block n should be executed, block validator 802 determines that basic block n should be executed. Until an indication is received, block validator 802 does not determine that basic block n should be verified.
[0132] Upon determining that basic block n should be executed, the block validator 802 determines whether basic block n is marked as invalid in operation 864 of the method 800. For example, the block validator 802 sends a command to the block flagger 804 to determine whether basic block n is flagged as invalid. Upon receiving the command, the block flagger 804 sends an indication of whether basic block n is flagged as invalid to the block validator 802. In an illustrative manner, the block flagger 804 accesses the basic block n to determine whether basic block n includes an invalid mark n, and generates and sends an indication of invalidity to the block validator 802. In an alternative illustrative manner, the block flagger 804 accesses the basic block n to determine whether basic block n does not include an invalid mark n, and generates and sends an indication of lack of invalidity to the block validator 802. Upon receiving an indication from the block flagger 804 that basic block n is marked as invalid, the block validator 802 determines that basic block n is invalid. On the other hand, upon receiving an indication from block flagger 804 that basic block n does not have invalid mark n, block validator 802 determines that basic block n is not marked as invalid.
[0133] Upon determining that basic block n is not marked as invalid, in operation 866 of method 800, basic block n is executed. For example, operation 866 is similar to operation 662 (FIG. 6B). To illustrate, block validator 802 sends a command to block dispatcher 302 to execute basic block n. Upon receiving the command to execute basic block n, block dispatcher 302 executes basic block n. In this illustration, block dispatcher 302 does not execute basic block n until a command to execute basic block n is received from block validator 802.
[0134] Meanwhile, in response to determining that basic block n is marked as invalid, in operation 866 of method 800, block validator 802 determines whether basic block n is in fact valid. For example, upon determining that basic block n is marked as invalid, block validator 802 generates a second hash value from either the updated instructions or the corrupted data stored at the same memory address in memory device 412 where one or more of emulated PU code instructions 1-M at which basic block n is compiled are stored. In this example, the second hash value is stored in one or more registers of cache 102 keyed to include basic block n. Illustratively, block validator 802 sends a request to parser 602 to retrieve the updated instructions or the corrupted data stored at one or more memory addresses in memory device 412. Upon receiving the request, parser 602 reads the updated instructions or the corrupted data from the one or more memory addresses from memory device 412 and provides the updated instructions or the corrupted data to block validator 802. In this diagram, the block validator 802 generates a second hash value from the updated instructions or corrupted data and stores the second hash value in the cache 102. For example, the block validator 802 generates a digest or checksum from the updated instructions or corrupted data. In this diagram, the block validator 802 compares the second hash value to the first hash value to determine whether the basic block n is valid. If the block validator 802 determines based on the comparison that there is a match between the first hash value and the second hash value, the block validator 802 determines that the basic block n is valid. This match occurs when the second hash value is generated from the updated instructions. On the other hand, if the block validator 802 determines based on the comparison that there is no match between the first hash value and the second hash value, the block validator 802 determines that the basic block n is invalid. This lack of match occurs when the second hash value is generated from corrupted data.
[0135] In response to determining that the basic block n is valid, in operation 870 of the method 800, the invalid mark n is removed. For example, the block validator 802 sends a command to the block flagger 804 to remove the invalid mark n from the basic block n stored in the cache 102. Upon receiving the command from the block validator 802, the block flagger 804 accesses the basic block n from the cache 102 and clears the invalid mark n from the basic block n. In particular, the block flagger 804 clears the invalid mark n from one or more memory addresses of the cache 102 where the invalid mark n is stored. Upon removing the invalid mark n, the block flagger 804 sends a command to the block dispatcher 302 to execute the basic block n, and upon receiving the command, in operation 866, the block dispatcher 302 executes the basic block n. As an example, the block dispatcher 302 does not execute the basic block n until a command to execute the basic block n is received from the block flagger 804.
[0136] On the other hand, if it is determined that basic block n is not valid, then in operation 872 of method 800, an additional basic block having the same function or operation as basic block n is compiled. For example, block validator 802 sends a command to block dispatcher 302 not to execute basic block n and sends an indication to block creator 604 that basic block n is invalid. Upon receiving the indication, block creator 604 recompiles basic block n. To illustrate, block creator 604 compiles the additional basic block in the same manner as it compiles basic block n from one or more of emulated PU code instructions 1-M stored in memory device 412, except that the additional basic block is compiled from an additional set of emulated PU code instructions, such as one or more of emulated PU code instructions 1-M, stored in a memory device of the additional legacy machine. The additional legacy machine is different from the legacy machine including memory device 412. Also, the additional set of emulated PU code instructions is of the same game title GN as the additional set of game code gcN. Alternatively, block creator 604 compiles the additional basic block in the same manner as it compiles basic block n from one or more of emulated PU code instructions 1-M stored in memory device 412, except that the additional basic block is compiled from an additional set of emulated PU code instructions in memory device 412, such as one or more of emulated PU code instructions 1-M, stored in a memory address of memory device 412 that is different from the memory address at which emulated PU code instructions 1-M are stored.As yet another explanation, the block creator 604 compiles the additional basic blocks in the same manner as it compiles basic block n from one or more of the emulated PU code instructions 1-M stored in memory device 412, except that the additional basic blocks are compiled from additional sets of emulated PU code instructions, such as one or more of the emulated PU code instructions 1-M, stored in a memory device different from memory device 412.
[0137] The additional basic blocks are sent from the block creator 604 to the block dispatcher 302 for execution. For example, the block dispatcher 302 executes the additional basic blocks to generate a portion of an additional virtual environment, such as virtual environment 452 (FIG. 4B). Explained, the block dispatcher 302 executes the additional basic blocks to generate a portion of an image frame, where the image frame includes additional virtual environment data, such as the position and orientation of virtual object 454, parameters of virtual object 454, positions and orientations of other virtual objects in virtual environment 452, and parameters of other virtual objects in the additional virtual environment. The block dispatcher 302 provides the image frame to a GPU of the emulation processor system 409 to render, e.g., display, the additional virtual environment on a display screen of the display device 410.
[0138] It should be noted that although method 850 is illustrated with reference to basic block n and an additional basic block, method 850 is equally applicable to other basic blocks 1 through (n-1) and more additional basic blocks that are generated in the same manner that the additional basic blocks are generated.
[0139] In one embodiment, the set of emulated PU code instructions 1-M are stored in a first set of one or more memory devices located external to the legacy machine, and an additional set of emulated PU code instructions is stored in a second set of one or more memory devices located external to the additional legacy machine.
[0140] FIG. 9A is a diagram illustrating an embodiment of a legacy machine 900. The legacy machine 900 includes a legacy CPU 902, a legacy GPU 904, a legacy APU 910, a memory device 906, and a CD-ROM drive 908. The memory device 906 is an example of the memory device 412 (FIG. 4A). An example of the legacy CPU 902 is a 32-bit CPU capable of processing up to 32 bits during one clock cycle. An example of the legacy GPU 904 is a 32-bit GPU capable of processing up to 32 bits during one clock cycle. An example of the legacy APU 910 is a 16-bit APU capable of processing up to 16 bits during one clock cycle to generate one or more legacy audio parameters. An example of the memory device 906 is a 2 megabyte (MB) RAM.
[0141] The legacy CPU 902, the legacy APU 910, and the legacy GPU 904 are coupled to a memory device 906, which is coupled to a CD-ROM drive 908. The emulated PU code 106 is stored within the memory device 906.
[0142] The legacy CPU 902, the legacy APU 910, or the legacy GPU 904 accesses the emulated PU code 106 from the memory device 906 and processes the emulated PU code 106. The CD-ROM drive 908 accepts a CD-ROM containing updated instructions or corrupted code. The updated instructions or corrupted code can be transferred from the CD-ROM to the memory device 906 by the legacy CPU 902.
[0143] In one embodiment, legacy machine 900 does not include a cache. In one embodiment, legacy machine 900 includes a limited capacity cache, such as a 4 kilobyte (KB) cache.
[0144] In one embodiment, legacy machine 900 is not currently manufactured by an entity, such as a company or corporation, but was manufactured by an entity in the past.
[0145] FIG. 9B is a diagram illustrating an embodiment of an updated machine 920. The updated machine 920 includes a CPU 922, a GPU 924, an APU 931, a memory system 926, and a cache 928, which is an example of the cache 102 (FIG. 1). The game console 402 (FIG. 4B) is an example of the updated machine 920. As an example, the CPU 922 includes two quad-core modules, each capable of processing 64 bits during each clock cycle. Each core has a 32 kilobyte (KB) cache. Another example of the CPU 922 is a 64-bit CPU that can process up to 64 bits during one clock cycle. As an example, the GPU 924 has 1152 cores, each capable of processing 64 bits during one clock cycle. As another example, the legacy GPU 924 is a 64-bit GPU that can process up to 64 bits during one clock cycle. As yet another example, APU 931 may be a 16-bit APU and may process up to 16 bits during one clock cycle. As yet another example, APU 931 may be a 24-bit APU and may process up to 24 bits during one clock cycle. As yet another example, APU 931 may be a 32-bit APU and may process up to 32 bits during one clock cycle.
[0146] The CPU 922, GPU 924, and APU 931 are coupled to a memory system 906. As an example, the emulated PU code 106 is stored in a legacy memory system 906. An example of what the memory system 926 includes is a hard drive providing 500 gigabytes (GB) or 2 terabytes (TB) of storage. The CPU 922, GPU 924, cache 928, and memory system 926 are coupled to each other via a bus 930.
[0147] The CPU 922 or GPU 924 first accesses the cache 928 before accessing the memory system 926. If the cache 928 determines that it does not contain the data requested by the CPU 922 or GPU 924, the CPU 922 or GPU 924 or APU 931 accesses the memory system 926.
[0148] Note that in one embodiment, the game code gcN cannot be executed by the CPU 922 or the GPU 924 or the APU 931, but can be executed by the legacy CPU 902 or the legacy GPU 904. Also, the basic blocks 1-n can be executed by the CPU 922 or the GPU 924 or the APU 930, but cannot be executed by the legacy CPU 902 or the legacy GPU 904 or the legacy APU 910.
[0149] In one embodiment, the cache 928 is located within the CPU 922 .
[0150] In one embodiment, the cache 928 is located within the GPU 924 .
[0151] In one embodiment, the cache 928 is located in the APU 931 .
[0152] In one embodiment, cache 928 is located in CPU 922 and another cache, such as cache 102, is located in GPU 924.
[0153] In one embodiment, updated machine 920 is currently manufactured by the entity or was recently manufactured by the entity, for example, updated machine 920 was manufactured by the entity several years after the manufacture of legacy machine 900.
[0154] FIG. 10A is a diagram of an embodiment of a system 1000 for illustrating the combination of multiple basic blocks into one basic block by the basic block compiler 104 (FIG. 1). The block creator 604 (FIG. 6A) receives a user input 1, such as a signal indicating that the virtual object 454 (FIG. 4B) is to start at a position P1 and a direction O1 in a level 1 of a legacy game N having a game title GN. Upon receiving the user input 1, a basic block 1 is generated by the basic block compiler 104 based on the emulated PU code instructions 1 to process the user input 1. Similarly, the block creator 604 receives a user input 2, such as a signal indicating that the virtual object 454 is to start at a position P1 and a direction O1 in a level 2 of the legacy game N. Upon receiving the user input 2, a basic block 2 is generated by the basic block compiler 104 based on the emulated PU code instructions 2 to process the user input 2.
[0155] When user input 3 is received, such as a signal indicating that virtual object 454 is to start at position P1 and direction O1 in level 3 of legacy game N, block compiler 104 identifies basic blocks 1 and 2 as processing of user input 3. Upon such identification, block creator 604 consolidates basic blocks 1 and 2, such as by combining them into a single basic block, such as basic block 1 or basic block 2. This consolidation conserves memory space in cache 102 and also increases efficiency in accessing the single basic block instead of accessing basic block 1 and basic block 2.
[0156] The block creator 604 also generates a value for a pointer in the cache 102. An example of a pointer used herein is a register. The pointer indicates to the block dispatcher 302 to execute a single basic block when a user input is received to process basic block 1 or basic block 2. When the block dispatcher 302 executes basic block 1 or 2, the block dispatcher 302 accesses the pointer and executes the single basic block in place of basic block 1 or 2.
[0157] FIG. 10B is a diagram of an embodiment of a system 1020 for illustrating modification of one or more of basic blocks 1-n. The system 1020 includes a block interface 1022, a cache 102, and a block dispatcher 302. As an example, basic block n includes an operation n of dead reckoning. As another example, basic block n includes an operation n of bouncing a crosshair off an edge of a display screen of a display device 410 (FIG. 4B). An example of the block interface 1022 is an ASIC, or a PLD, or a microprocessor, or a microcontroller, or a computer program, or a portion of a computer program. The block interface 1022 is coupled to the cache 102.
[0158] The block interface 1022 provides a user with access to one or more basic blocks 1-n stored in the cache 102. For example, a user selects one or more buttons on an input device, such as a keyboard, mouse, or keypad, to generate a modification input 1024. As an example, the modification input 1024 includes one or more user instructions in the form of source code for modifying basic block n. To illustrate, the modification input 1024 includes user instructions for removing a dead-reckoning operation n from basic block n. As another example, the modification input 1024 includes instructions for modifying operation n to include causing a crosshair to slide off an edge of the display screen of the display device 410 and be displayed on an opposite edge of the display screen of the display device 410. The opposite edge is diagonal to the edge from which the crosshair slides off.
[0159] The input device is coupled to the block interface 1022. The modification input 1024 is sent from the input device to the block interface 1022 to modify the basic block n and output the modified basic block n. As an example, the modified basic block n does not include a dead reckoning operation. As another example, the modified basic block n includes an operation n of sliding a crosshair from one edge to the opposite edge. As yet another example, the modified basic block n includes an operation of calculating the number of clock cycles for the execution of the operation n and storing the number in the basic block n. Explaining, the number of cycles is stored in the memory address of the cache 102 where the basic block 1 is stored. The operation of calculating the number of clock cycles is performed by the block dispatcher 302 (FIG. 3) in addition to executing the operation n. The block dispatcher 302 calculates the number of clock cycles when executing the operation n and stores the number in the basic block n.
[0160] The block creator 604 further generates a value for a pointer in the cache 102 that indicates to the block dispatcher 302 to execute the modified basic block n when user input to process basic block n is received. When the block dispatcher 302 executes basic block n, the block dispatcher 302 accesses the pointer and executes the modified basic block n instead of basic block n.
[0161] FIG. 10C is a diagram of an embodiment of a system 1030 illustrating combining a basic block created based on a subroutine with a basic block generated based on an emulated PU code instruction that calls the subroutine. The block creator 604 (FIG. 6A) receives a user input 1, such as a signal indicating a change in position and / or orientation of a virtual object 454 (FIG. 4B) resulting in the destruction of the virtual object 454. Upon receiving the user input 1, a basic block 1 is generated by the basic block compiler 104 based on the emulated PU code instruction 1 to process the user input 1. The emulated PU code instruction 1 includes a function call to the subroutine 1. Thus, upon receiving the user input 1, a basic block 2 is generated by the basic block compiler 104 based on the subroutine 1, which is an example of an emulated PU code instruction 2. As an example, the basic block 2 includes an operation 2 that recreates the virtual object 454 at a position P1 and an orientation O1.
[0162] When user input 2 is received, such as a signal indicating a change in position and / or orientation of virtual object 454 (FIG. 4B) that results in the destruction of virtual object 454, block compiler 104 identifies basic blocks 1 and 2 as processing user input 2. Upon such identification, block creator 604 consolidates, such as by combining, basic blocks 1 and 2 into a single basic block, such as basic block 1 or basic block 2. For example, receipt of user input 2 triggers the destruction of virtual object 454 and the regeneration of virtual object 454. When basic blocks 1 and 2 are combined, subroutine 1 is skipped.
[0163] Block creator 604 also generates a value and stores the value in a pointer in cache 102. This value indicates to block dispatcher 302 to execute the single basic block when user input is received to process basic blocks 1 and 2. When block dispatcher 302 is to execute basic blocks 1 and 2, block dispatcher 302 accesses the pointer and executes the single basic block instead of basic block 1.
[0164] 10D is a diagram of an embodiment of a system 1040 illustrating the insertion of a basic block between two basic blocks. The system 1040 includes a block interface 1022 and a cache 102. After basic blocks 1 and 2 are stored in the cache 102, a modification input 1042 is received from a user via an input device. As an example, the modification input 1042 is a signal including source code, which defines a basic block 1.1, which includes a source register address 1.1 in the cache 102, a destination register address 1.1 in the cache 102, and an operation 1.1 to be performed on data stored in the source register address 1.1, or stored on the destination register address 1.1, or stored at both the source register address and the destination register address 1.1. Examples of operations 1.1 include jumps, stores, loads, branches, and arithmetic operations.
[0165] The modification input 1042 further includes an indication of the location of basic block 1.1 and an association of basic block 1.1 with basic block 1 or 2. For example, the modification input 1042 includes a signal indicating that basic block 1.1 is to be inserted between basic blocks 1 and 2 in the cache 102 and a signal indicating that basic block 1.1 is to be linked to basic blocks 1 and / or 2. Explained, basic block 1.1 is inserted to include a location in the cache 102, such as one or more register addresses, for receiving the invalid mark 2 of basic block 2. As another explanation, basic block 1.1 is inserted to include a location in the cache 102, such as one or more register addresses, for receiving the cycle number of execution of basic block 2. In this explanation, the block interface 1022 receives the modification input 1042, identifies from the modification input 1042 that basic block 1.1 is to be inserted between basic blocks 1 and 2, and inserts basic block 1.1 between basic blocks 1 and 2. As another example, the block interface 1022 determines that basic block 1.1 includes an operation for unlocking a level of a legacy game N having a game title GN. The level is between a first level identified in an operation of basic block 1 and a second level identified in an operation of basic block 2. The first level and the second level are in the legacy game N having the game title GN. The level inserted between the first level and the second level is not part of the game code gcN, but is a new level for the legacy game N. An example of a level inserted between the first level and the second level is a level in which the virtual object 454 fires a laser gun instead of missiles.
[0166] Additionally, block interface 1022 identifies a pointer value from modified input 1042 and stores the value in cache 102. As an example, the pointer value indicates that execution of basic block 1.1 occurs immediately before execution of basic block 2. When block dispatcher 302 executes basic block 2, block dispatcher 302 identifies a value in the pointer that points to basic block 1.1 and executes basic block 1.1 immediately before executing basic block 2. As another example, the pointer value indicates that execution of basic block 1.1 occurs immediately after executing basic block 2. After block dispatcher 302 executes basic block 2, block dispatcher 302 identifies a value in the pointer that points to basic block 1.1 and executes basic block 1.1 immediately after executing basic block 2.
[0167] 10E is a diagram of an embodiment of a system 1050 illustrating a switch in the order of execution of basic blocks. System 1050 includes cache 102 and block interface 1022. Cache 102 includes a value in a pointer in cache 102 indicating that basic block 2 will execute after basic block 1 executes. A user uses an input device to provide a modification input 1052 including a signal indicating that the order of execution of basic blocks 1 and 2 is to be switched. Upon receiving the signal, block interface 1022 changes the value of the pointer in cache 102 to indicate that basic block 1 will execute after basic block 2 executes.
[0168] 11A is a flow chart of an embodiment of a method 1100 to illustrate the use of an actual count of the number of cycles stored in basic block n. Method 1100 includes operation 662 of executing basic block n. In operation 1102 of method 1100, block dispatcher 302 (FIG. 3) counts the number of cycles of execution of basic block n in operation 662 to generate a first count. Block dispatcher 302 stores the first count in basic block n in operation 1104 of method 1100. For example, block dispatcher 302 writes a real number to a register having one of the register addresses in cache 102 that is assigned to basic block n.
[0169] In operation 1106 of method 1100, block creator 604 (FIG. 6A) determines whether the same basic block n is to be executed again. For example, block creator 604 determines whether user input is received from a client device to process the same basic block n. Block creator 604 continues to determine whether user input is received to process the same basic block n until user input is received.
[0170] Figure 11B is a continuation of the flowchart of method 1100 of Figure 11A. Upon determining that user input to process basic block n has again been received, basic block n is executed again by block dispatcher 304. In operation 1108 of method 1100, block dispatcher 304 counts the number of cycles of execution of basic block n in operation 1106 to calculate a second count.
[0171] The block dispatcher 304 determines whether the second count is within a predetermined limit from the first count in operation 1108 of the method 1100. The predetermined limit is stored in the cache 102. In response to determining that the second count is not within the predetermined limit from the first count, the block dispatcher 304 generates a notification in operation 1110 of the method 1100. For example, when the block dispatcher 304 is located within the game console 402 (FIG. 4A), the GPU 924 (FIG. 9B) of the game console 402 displays a representation of the notification on the display device 410 (FIG. 4A) of the client device and transmits the notification to the server system 404 via the computer network 408 to notify the server system 404. As another example, when the block dispatcher 304 is located within the server system 404 (FIG. 4B), the block dispatcher 304 generates the notification and the GPU of the server system 404 displays a representation of the notification on a display device coupled to the server system 404. On the other hand, if the second count is determined to be within the predetermined limit from the first count, then in operation 1112 of the method 1100, the block dispatcher 304 does not trigger a notification.
[0172] FIG. 12 is a diagram of an embodiment of a system 1200 for illustrating the transfer of basic blocks 1-n from a first client device to a second client device. The system 1200 includes a game console 402, a computer network 408, a server system 404, a game console 1202, and a display device 1204. The game console 1202 is similar to the game console 402. For example, both game consoles 402 and 1202 are examples of updated machines. As another example, the game console 1202 is not a legacy machine. Also, the display device 1204 is similar to the display device 410 of FIG. 4A. For example, the display device 410 is an HMD or a television or a smart television or a computer monitor.
[0173] The game console 402 includes a network interface controller 1212. The game console 1202 includes an emulation processor system 1206, an input processor system 1207, a cache 1208, and a network interface controller 1210. The emulation processor system 1206 has the same structure and functions as the emulation processor system 409. The input processor system 1207 has the same structure and functions as the input processor system 407 (FIG. 4A). Also, the cache 1208 has the same structure and functions as the cache 102. The emulation processor system 1206, the input processor system 1207, the cache 1208, and the network interface controller 1210 are coupled to each other via a bus 1218.
[0174] Once basic blocks 1-n are stored in cache 102, basic block compiler 104 of emulation processor system 409 transmits basic blocks 1-n to network interface controller 1212. Network interface controller 1212 applies a network communication protocol to basic blocks 1-n to generate one or more packets that embed basic blocks 1-n, and transmits the one or more packets to server system 404 via computer network 408. Upon receiving the one or more packets, network interface controller of server system 404 applies a network communication protocol to the one or more packets to extract basic blocks 1-n of legacy game N having game title GN, and stores basic blocks 1-n within one or more memory devices of server system 404.
[0175] User 2 uses handheld controller 1212 to select one or more buttons on handheld controller 1212 to log into user account 2 assigned to user 2 by server system 404. User 2 is logged into user account 2 once user ID 2 and password are authenticated by server system 404. Once user 2 is logged into user account 2, user 2 can access multiple game titles, such as game title G1, game title Ga, game title G2, and so on up to game title GN.
[0176] Upon logging into user account 2, user 2 uses handheld controller 1212 to select one or more buttons on handheld controller 1212 to generate user input 1214. User input 1214 is generated upon selection of legacy game N having title GN displayed on display device 1204. When user input 1214 is generated, cache 1208 does not contain basic blocks 1-n. For example, upon receiving user input 1214, a basic block compiler of emulation processor system 1206 checks cache 102 to determine whether cache 102 contains one or more of basic blocks 1-n of game code GCN for processing user input 1214. Upon determining that cache 102 does not contain one or more of basic blocks 1-n for processing user input 1214, emulation processor system 1206 generates a request 1220 for one or more of basic blocks 1-n and sends request 1220 to network interface controller 1210.
[0177] Upon receiving request 1220, network interface controller 1210 generates one or more packets embedding request 1220 by applying a network communication protocol to request 1220, and transmits the one or more packets to server system 404 over computer network 408. The network interface controller of server system 404 receives the one or more packets and applies the network communication protocol to extract request 1220 from the one or more packets. A processor of server system 404 analyzes request 1220 to identify that basic blocks 1-n are being requested.
[0178] In response to determining that basic blocks 1-n have been requested, the server system 404 accesses basic blocks 1-n stored in one or more memory devices of the server system 404 and provides basic blocks 1-n to a network interface controller of the server system 404. The network interface controller of the server system 404 applies a network communications protocol to generate one or more packets embedding basic blocks 1-n and transmit the one or more packets to the game console 1202 over the computer network 408.
[0179] Network interface controller 1210 of game console 1202 receives one or more packets having basic blocks 1-n, applies a network communication protocol to extract basic blocks 1-n from the one or more packets, and sends basic blocks 1-n to emulation processor system 1206. A basic block compiler of emulation processor system 1206 stores basic blocks 1-n in cache 1208.
[0180] Upon receiving user input 1224 from handheld controller 1212 during play of a game having game title GN, input processor system 1207 converts user input 1224 to user input 1225 in the same manner that input processor system 407 converts user input 419 to user input 420. Upon receiving user input 1225, a basic block compiler of emulation processor system 1206 identifies one or more of basic blocks 1-n in cache 1208 to process user input 1225. A block dispatcher of emulation processor system 1206 executes one or more of basic blocks 1-n to process user input 1225. In this manner, once basic blocks 1-n are compiled by emulation processor system 409, basic blocks 1-n do not need to be compiled by emulation processor system 1206, but can be accessed by emulation processor system 1206 from server system 404.
[0181] 13 is a diagram of an embodiment of a system 1300 to illustrate emulation of user input 419 received from a handheld controller 414 during play of a legacy game N. System 1300 includes an input processor system 407, an emulation processor system 409, a handheld controller 414, and a handheld controller 1306.
[0182] The handheld controller 1306 is an example of a legacy handheld controller. As an example, the handheld controller 1306 can be used with the legacy machine 900 (FIG. 9A) but cannot be used with the updated machine 920 (FIG. 9B). To explain, the input processor system 407 includes an input processor 1314, which cannot interpret user input generated by and received from the handheld controller 1306. To further explain, the input processor 1314 can interpret user input 419 represented in 64 bits or 128 bits, but cannot interpret user input having 32 bits received from the handheld controller 1306. As another example, the handheld controller 1306 is an example of a legacy controller and the handheld controller 414 is an example of an updated controller. To explain, the handheld controller 1306 is not currently manufactured by the entity and the handheld controller 414 is currently manufactured by the entity. To further illustrate, handheld controller 414 is a current version and handheld controller 1306 is an older version. As another further illustration, handheld controller 1306 was manufactured by an entity several years before handheld controller 414 was manufactured by the entity. As yet another example, handheld controller 1306 does not include a thumbstick but does include a selection button and handheld controller 414 does not include a selection button. An example of input processor 1314 is a processor integrated within a serial input / output (I / O) chip.
[0183] The input processor system 407 further includes an input emulator 1318, a memory device 1316, and a communication device 1320. A default mapping 1308 is stored within the memory device 1316. Note that the default mapping 1308 is associated with user accounts 1 and 2. For example, when legacy game N is accessed via user account 1, the default mapping 1308 is applied by the input emulator 1318 unless there is a current mapping stored in the memory device 1316 and selected for use with user account 1.
[0184] As an example, the input emulator described herein is hardware or software, or a combination thereof. Explaining, the input emulator is a processor, or a PLD, or an ASIC, or a controller. To further explain, the input emulator 1318, when implemented in the updated machine 920, enables the updated machine 920 to receive and process user input received from the handheld controller 414 to facilitate playing the legacy game N. The input emulator 1318 facilitates the use of the handheld controller 414 to play the legacy game N. An example of the communication device 1320 is a communication circuit that applies a short-range communication protocol, such as a wireless transfer protocol or a wired transfer protocol. An example of a wired transfer protocol is the USB protocol, and an example of a wireless transfer protocol is the Bluetooth protocol. The communication device 1320 is coupled to the input processor 1314, which is coupled to the input emulator 1318. The input emulator 1318 is coupled to the basic block compiler 104. Both the input processor 1314 and the input emulator 1318 are coupled to the memory device 1316.
[0185] The handheld controller 414 is also coupled to the input processor 1314 via a connection, such as a wired connection or a wireless connection. An example of a wired connection includes a parallel transfer connection, in which information is communicated from the handheld controller 414 to the input processor 1314 in a simultaneous parallel manner.
[0186] User 1 operates one or more input devices on the handheld controller 414 to select the game title GN and generate a user input 418 (FIG. 4A). Examples of input devices on the handheld controller 414 include a left thumbstick 1302A, a right thumbstick 1302B, and one or more buttons. Explaining, to select the game title GN, a button on the handheld controller 414 is selected by user 1, such as pressed or depressed. Based on the user input 418, the server system 404 transmits the game code gcN to the game console 402 via the computer network 408 for storage in the memory device 406 (FIG. 4A) of the game console 402.
[0187] During play of legacy game N, one or more input devices of handheld controller 414 are manipulated by user 1 to generate one or more corresponding electrical signals, which ultimately form user input 419. For example, when user 1 moves a thumbstick of handheld controller 414 in a forward or backward or right or left direction, an electrical signal is generated. To illustrate, the forward and reverse directions are along the y-axis of an xyz coordinate system of handheld controller 414, and the right and left directions are along the x-axis of the xyz coordinate system. In this illustration, the forward direction is opposite to the reverse direction, and the right direction is opposite to the left direction.
[0188] The handheld controller 414 includes motion-sensitive hardware, such as one or more magnetometers, one or more accelerometers, one or more potentiometers, and one or more gyroscopes, coupled to one or more input devices of the handheld controller 414. The handheld controller 414 further includes an analog-to-digital converter coupled to the motion-sensitive hardware. The handheld controller 414 includes a processor, such as an encoder coupled to the analog-to-digital converter, and a communication device coupled to the processor. When one or more input devices of the handheld controller 414 are manipulated by the user 1, corresponding electrical signals, such as one or more analog motion-sensitive signals, are generated by the motion-sensitive hardware. The one or more analog motion-sensitive signals are transmitted from the motion-sensitive hardware to the analog-to-digital converter of the handheld controller 414. The analog-to-digital converter of the handheld controller 414 converts the analog motion-sensitive signals from analog format to digital format and outputs digital motion-sensitive signals. The digital motion-sensitive signal is transmitted from the analog-to-digital converter to a processor in the handheld controller 414 .
[0189] The processor of the handheld controller 414 applies an encoding protocol to encode the digital motion-sensitive signal and outputs the encoded signal. For example, upon determining that the first digital motion-sensitive signal represents a selection of an option button on the handheld controller 414, the processor of the handheld controller 414 encodes the first digital motion-sensitive signal using a first frequency. In this example, in response to determining that the second digital motion-sensitive signal represents movement of the left thumbstick 1302A of the handheld controller 414, the processor of the handheld controller 414 encodes the second digital motion-sensitive signal using a second frequency that is different from the first frequency, such as greater than or less than the first frequency. Also in this example, in response to determining that the third digital motion-sensitive signal represents movement of the right thumbstick 1302B of the handheld controller 414, the processor of the handheld controller 414 encodes the third digital motion-sensitive signal using a third frequency that is different from the first and second frequencies.
[0190] The processor of the handheld controller 414 transmits the encoded signal to a communication device of the handheld controller 414. The communication device of the handheld controller 414 applies a wireless transfer protocol, such as the Bluetooth™ protocol, to the encoded signal to generate one or more data packets that are examples of user input 419. The communication device of the handheld controller 414 transmits the user input 419 to the communication device 1320 of the input processor system 407 via a wireless connection.
[0191] Upon receiving the user input 419, the communication device 1320 applies a wireless transfer protocol to extract an encoded signal from one or more data packets and transmits the encoded signal to the input processor 1314. The input processor 1314 analyzes, such as by decoding and analyzing, the encoded signal to determine initial input device information associated with the operation of the handheld controller 414. For example, the input processor 1314 identifies which of the input devices of the handheld controller 414 is being operated by the user 1, the amount of movement of the input device, and the direction of the amount of movement based on the one or more digital motion sensitive signals. Illustratively, the input processor 1314 determines whether a particular button (e.g., an option button) on the handheld controller 414 is selected by the user 1 from one or more digital motion sensitive signals generated based on one or more analog motion sensitive signals received from an accelerometer of the handheld controller 414. To further explain, upon determining that the first digital motion-sensitive signal is encoded using a first frequency, the input processor 1314 determines that an option button is selected, and in response to determining that the second digital motion-sensitive signal is encoded using a second frequency, the input processor 1314 determines that the left thumbstick 1302A is being manipulated by user 1.
[0192] As another explanation, the input processor 1314 determines from one or more digital motion-sensitive signals generated based on one or more analog motion-sensitive signals received from the accelerometer, gyroscope, and magnetometer of the handheld controller 414 whether the thumbstick is moved in a particular direction, such as forward, backward, right, left, etc., and whether the amount of movement exceeds a pre-determined threshold in the particular direction. In this explanation, the movement in the particular direction and the amount of movement are examples of thumbstick functionality. To further explain, the input processor 1314 determines that the amount of movement of the left thumbstick 1302A exceeds a predetermined amount of movement to determine that the amount of movement exceeds a predetermined threshold. The predetermined amount of movement is stored in the memory device 1316. Another example of the functionality of a button of the handheld controller 414 is the selection (e.g., pressing or depressing) of a button by user 1. In this example, the functionality of the button is determined when the button is identified by the input processor 1314 as being operated by user 1. As another example of the function of a button of the handheld controller 414, the input processor 1314 determines the number of times a digital motion-sensitive signal is generated when User 1 operates the button. In this example, the number of times is equal to the number of times a digital motion-sensitive signal is determined to be received by the input processor 1314 from the handheld controller 414. Also, in this example, the number of times is the same as the number of times the button is selected by User 1.
[0193] The input processor 1314 sends the initial input device information to the input emulator 1318. Upon receiving the initial input device information during play of the legacy game N, the input emulator 1318 accesses the default mapping 1308 stored in the memory device 1316 to identify one or more legacy input devices of the handheld controller 1306 based on the identifiers 1312 of the one or more updated input devices of the handheld controller 414. For example, upon receiving the initial input device information, the input emulator 1318 determines from the default mapping 1308 one or more legacy input device identifiers 1310 that correspond to one or more of the updated input device identifiers 1312 of the one or more updated input devices of the handheld controller 414 operated by User 1. The default mapping 1408 includes a one-to-one correspondence, such as a link between each of the legacy input devices of the handheld controller 1306 and a corresponding one of the updated input devices of the handheld controller 414.
[0194] Examples of updated input device identifiers 1312 include identifying characters, such as alphanumeric characters and symbols or bits, that identify one or more of the updated input devices of handheld controller 414. Illustratively, updated input device identifiers 1312 include a symbol for a left thumbstick 1302A and a symbol for a right thumbstick 1302B. As another example, updated input device identifiers 1312 include a symbol for a touchpad on handheld controller 414.
[0195] Examples of the legacy input device identifier 1310 include identifying characters, such as alphanumeric characters and symbols or bits, that identify one or more of the legacy input devices of the handheld controller 1306. Illustratively, the updated input device identifier 1310 includes a symbol for a left movement button 1304A of the handheld controller 1306, a symbol for a forward movement button 1304B of the handheld controller 1306, a symbol for a right movement button 1304C of the handheld controller 1306, and a symbol for a backward movement button 1304D of the handheld controller 1306. As another illustration, the legacy input device identifier 1310 includes a symbol for a select button of the handheld controller 1306.
[0196] Further, the input emulator 1318 determines one or more functions of one or more legacy input devices of the handheld controller 1306 from one or more functions of the handheld controller 414 operated by user 1. In one embodiment, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a predefined threshold corresponds to three selections of the left button 1304A by user 1. In another embodiment, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a first predefined threshold corresponds to two selections of the left button 1304A by user 1. In this embodiment, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a second predefined threshold corresponds to three selections of the left button 1304A by user 1. The second predefined threshold is greater or higher than the first predefined threshold. As yet another explanation, one or more functions of one or more legacy input devices of the handheld controller 1306, as determined from one or more functions of the handheld controller 414, achieve the same result as achieved by one or more functions of the handheld controller 414. In this explanation, one or more functions of one or more legacy input devices of the handheld controller 1306 and one or more functions of the handheld controller 414 result in the same amount of movement of a virtual object in the virtual environment 452. An example of the same amount of movement is movement from one position to another, or movement from one orientation to another, or a combination thereof.
[0197] The input emulator 1318 generates a user input 420 having one or more legacy input device identifiers 1310 that correspond to the one or more updated input device identifiers 1312 operated by the user 1. The user input 420 also includes one or more functions of the one or more legacy input devices identified by the one or more legacy input device identifiers 1310. The input emulator 1318 sends the user input 420 to the basic block compiler 104.
[0198] Based on the user input 420, the basic block compiler 104 determines whether one or more of the basic blocks 1-n for processing the user input 420 are stored in the cache 102 (FIG. 1). For example, based on the user input 420, the basic block compiler 104 determines whether one or more of the basic blocks 1-n stored in the cache 102 can be executed to move a fighter from position P1 to position P2 during the play of the legacy game N. To explain, based on the selection of the left movement button 1304A, the basic block compiler 104 determines that the fighter is moved from position P1 to position P2. The basic block compiler 104 further determines whether the execution of one or more of the basic blocks 1-n stored in the cache 102 enables the movement of the fighter from position P1 to position P2 according to the selection of the left movement button 1304A.
[0199] Upon determining that one or more of the basic blocks 1-n for processing the user input 420 are stored in the cache 102, the basic block compiler 104 sends an instruction to the block dispatcher 302 (FIG. 3). In response to the instruction, the block dispatcher 302 executes one or more of the basic blocks 1-n from the cache 102 to process the user input 420. For example, one or more of the basic blocks 1-n are executed to move a fighter plane from position P1 to position P2. Furthermore, one or more of the basic blocks 1-n are executed to generate one or more image frames in which the fighter plane moves from position P1 to position P2.
[0200] Meanwhile, in response to determining that one or more of the basic blocks 1-n for processing the user input 420 are not stored in the cache 102, the basic block compiler 102 identifies one or more of the emulated PU code instructions 1-M for processing the user input 420. The basic block compiler 102 compiles the one or more of the emulated PU code instructions 1-M to generate one or more of the basic blocks 1-n to process the user input 420 and provide the one or more of the basic blocks 1-n to the cache 102 for storage. The basic block compiler 102 also sends instructions to the block dispatcher 302 to execute one or more of the basic blocks 1-n that were recently stored in the cache 102. The one or more of the basic blocks 1-n are then executed by the block dispatcher 302 to process the user input 420.
[0201] In one embodiment, the handheld controller 414 is not compatible with the legacy machine 900 (FIG. 9A). For example, the input processor system of the legacy machine 900 includes an input processor that can interpret user input generated by and received from the handheld controller 1306. The input processor system of the legacy system 900 cannot interpret user input 419 generated by and received from the handheld controller 414. To further explain, the input processor of the legacy system 900 can interpret user input received from the handheld controller 1306 and represented in 32 bits, but cannot interpret user input 419 represented in 64 bits or 128 bits.
[0202] In one embodiment, the input emulator 1318 is part of the input processor 1314. For example, the input emulator 1318 is a computer program stored in a cache of the input processor 1314 for execution by a control unit and an arithmetic logic unit (ALU) of the input processor 1314.
[0203] In one embodiment, the communication device of the handheld controller 414 is coupled to the motion-sensitive hardware of the handheld controller 414. The communication device applies a wired transport protocol to transmit digital motion-sensitive signals to the communication device of the input processor system 407 over different channels of the wired connection. For example, upon receiving a first digital motion-sensitive signal representing a movement of a button on the handheld controller 414, the communication device of the handheld controller 414 transports the first digital motion-sensitive signal over a first channel, such as a first connection link or a first conductor of the wired connection. In this example, in response to receiving a second digital motion-sensitive signal representing a movement of the left thumbstick 1302A of the handheld controller 414, the communication device of the handheld controller 414 transports the second digital motion-sensitive signal over a second channel, such as a second connection link or a second conductor of the wired connection. Also in this example, in response to receiving a third digital motion-sensitive signal representing movement of the right thumbstick 1302B of the handheld controller 414, the communication device of the handheld controller 414 transmits the third digital motion-sensitive signal via a third channel, such as a third connection link or a third wire of a wired connection.
[0204] Continuing with the embodiment, the communication device of the input processor system 407 receives the digital motion sensitive signal and provides the digital motion sensitive signal along with channel information to the input processor 1314. For example, the communication device of the input processor 407 indicates to the input processor 1314 that a first digital motion sensitive signal is received via a first channel, a second digital motion sensitive signal is received via a second channel, and a third digital motion sensitive signal is received via a third channel. The input processor 1314 identifies which of the input devices of the handheld controller 414 is operated by user 1 based on the channel.
[0205] In one embodiment, the handheld controller 414 is itself an input device: for example, the handheld controller 414 is moved by the user 1 in a forward, backward, right, or left direction to control a virtual object moving in that direction.
[0206] 14A-1 is a diagram of an embodiment of a system 1400 for illustrating a change by User 1 from a default mapping 1308 to a current mapping 1404. The system 1400 includes a handheld controller 414, an input processor system 407, a NIC 1212, a computer network 408, and a server system 404.
[0207] Memory device 412 of server system 404 stores information regarding user accounts 1 and 2. For example, memory device 412 stores user accounts 1 and 2 and verification information to enable login to user accounts 1 and 2. As another example, memory device 412 stores the number of points accumulated in user account 1 by user 1 during play of a current game having legacy game N and game title Ga. In this example, memory device 412 stores the number of points accumulated in user account 2 by user 2 during play of a current game having legacy game N and game title Ga.
[0208] User 1 manipulates one or more updated input devices of the handheld controller 414, and when the one or more updated input devices are manipulated, one or more digital motion-sensitive signals are generated in a manner similar to that shown above. For example, User 1 manipulates one or more of the updated input devices of the handheld controller 414 to pause legacy game N. In this example, when legacy game N is paused, User 1 manipulates one or more of the updated input devices of the handheld controller 414 to change the default mapping 1308 to the current mapping 1414. As another example, before playing legacy game N, User 1 manipulates one or more of the updated input devices of the handheld controller 414 to change the default mapping 1308 to the current mapping 1414.
[0209] As an example, the default mapping 1308 maps a first first identifier of an updated input device of the controller 414 to a first first first identifier of a legacy input device of the controller 1306 (FIG. 13). Also as an example, the default mapping 1308 maps a second identifier of an updated second input device of the controller 414 to a second identifier of a second legacy input device of the controller 1306.
[0210] The communication device of the handheld controller 414 generates one or more data packets embedding one or more digital motion sensitive signals indicating that the default mapping 1308 is to be changed to a current mapping 1414. The communication device of the handheld controller 414 applies a wireless transfer protocol to generate the one or more data packets. The one or more data packets for changing the default mapping 1308 to the current mapping 1414 are transmitted as user input 1402 from the communication device of the handheld controller 414 to the communication device 1320 over a wireless connection.
[0211] The communication device 1320 applies a wireless transfer protocol to extract one or more digital motion sensitive signals from the one or more packets to change the default mapping 1308 to a current mapping 1414. The communication device 1320 transmits the one or more digital motion sensitive signals to the input processor 1314 to change the default mapping 1308. Upon receiving the one or more digital motion sensitive signals, the input processor 1314 determines to generate a current mapping 1404 from the default mapping 1308. For example, in response to receiving an indication that a first identifier of a first one of the updated input devices of the controller 414 has been mapped to a second identifier of a second one of the legacy input devices of the controller 1306, the input processor 1314 links the first identifier to the second identifier to generate a current mapping 1404 and associates the current mapping 1404 with the input emulator 1318 and the user account 1. As another example, in response to receiving an indication that a second identifier of a second one of the updated input devices of the controller 414 has been mapped to a first identifier of a first one of the legacy input devices of the controller 1306, the input processor 1314 links the second identifier to the first identifier to generate a current mapping 1404 and associates the current mapping 1404 with the input emulator 1318.
[0212] Further, after the current mapping 1404 is generated via user account 1, the input processor 1314 stores the current mapping 1404 in the memory device 1316 and associates the current mapping 1404 with user account 1. For example, when accessing user account 1, user 1 may access the current mapping 1404 in addition to the default mapping 1308. To illustrate, when user 1 logs into user account 1, after receiving a digital motion-sensitive signal indicating a selection of game title GN, the input processor 1314 provides user 1 with a choice to select between the default mapping 1308 and the current mapping 1404 for playing legacy game N. As another example, in response to generating the current mapping 1404, the input processor 1314 instructs the input emulator 1318 to emulate future user inputs received from the handheld controller 414 during the play of legacy game N. In this illustration, after user 1 logs into user account 1, future user inputs are received by the input processor system 407. Also, in this illustration, future user inputs are emulated according to the current mapping 1404 instead of the default mapping 1308. To further illustrate, after user s1 manipulates one or more of the updated input devices of the handheld controller 414, user s1 logs out of user account 1, indicating that the default mapping 1308 is changed to the current mapping 1404. To further illustrate, user s1 logs out of user account 1, and then logs into user account 1. To further illustrate, after user s1 logs into user account 1, the input emulator 1318 applies the current mapping 1404 instead of the default mapping 1308 to future user inputs until a selection of the default mapping 1308 is received from the handheld controller 414.
[0213] 14A-2 is a diagram of an embodiment of the memory device 1316 to illustrate the default mapping 1308 and the current mapping 1404. The default mapping 1308 maps a legacy input device identifier 1310 of a legacy input device of the handheld controller 1306 (FIG. 13) with a function identifier of the legacy input device, an updated input device identifier 1312 of an updated input device of the handheld controller 414 (FIG. 13), and an identifier of the function of the updated input device. For example, the default mapping 1308 includes a one-to-one correspondence between the legacy input device identifier LID1 of the legacy input device of the handheld controller 1306, the legacy input device function LIF1, the updated input device identifier UID1 of the updated input device of the handheld controller 1306, and the updated input device function UIF1. As another example, default mapping 1308 includes a one-to-one correspondence between a legacy input device identifier LID2, a legacy input device capability LIF2, an updated input device identifier UID2, and an updated input device capability UIF2 of a legacy input device of handheld controller 1306. As yet another example, default mapping 1308 includes a one-to-one correspondence between a legacy input device identifier LID5, a legacy input device capability LIF5, an updated input device identifier UID5, and an updated input device capability UIF5 of an updated input device of handheld controller 1306.
[0214] Similarly, the current mapping 1404 maps the legacy input device identifier 1310 of the legacy input device of the handheld controller 1306 with an identifier of the function of the legacy input device, an updated input device identifier of the updated input device of the handheld controller 414, and an identifier of the function of the updated input device. For example, the current mapping 1404 includes a one-to-one correspondence between the legacy input device identifier LID1, the function LIF1, the updated input device identifier UID2, and the function UIF2 of the updated input device. As another example, the current mapping 1404 includes a one-to-one correspondence between the legacy input device identifier LID2, the function LIF2, the updated input device identifier UID1, and the function UIF1. It should be noted that in the current mapping 1404, the mapping between the identifiers LID1 and UID1 of the default mapping 1308 is switched to generate a mapping between the identifiers LID1 and UID2. Also, in the current mapping 1404, the mapping between the identifiers LID2 and UID2 in the default mapping 1308 is switched to generate a mapping between the identifiers LID2 and UID1.
[0215] In one embodiment, multiple legacy input device identifiers are mapped to one or more updated input device identifiers. For example, multiple legacy input device identifiers identifying left movement button 1304A and right movement button 1304C are mapped to a single updated input identifier for a thumbstick (such as left thumbstick 1302A or right thumbstick 1302B (FIG. 13)) of handheld controller 414. As another example, multiple legacy input device identifiers identifying forward movement button 1304B and backward movement button 1304D are mapped to a single updated input identifier for a thumbstick (such as left thumbstick 1302A or right thumbstick 1302B) of handheld controller 414.
[0216] In one embodiment, the default mapping 1308 does not include an identifier for the capabilities of the legacy input device of the handheld controller 1306 and an identifier for the capabilities of the updated input device of the handheld controller 414 .
[0217] In one embodiment, the mapping includes any number of identifiers for a corresponding number of updated input devices for the updated handheld controllers as described herein, and the mapping includes any number of identifiers for a corresponding number of legacy input devices for the legacy handheld controllers.
[0218] 14B is a diagram of an embodiment of a system 1450 to illustrate the use of a current mapping 1404 in place of the default mapping 1308 (FIG. 13) during play of a legacy game N. User 1 logs into user account 1 and selects the current mapping 1404 via one or more of the updated input devices of the handheld controller 414. Upon receiving user input from the handheld controller 414 indicating a selection of the current mapping 1404, the input processor 1314 instructs the input emulator 1318 to apply the current mapping 1404 in place of the default mapping 1308.
[0219] During play of legacy game N, one or more input devices of handheld controller 414 are manipulated by user 1 to generate one or more corresponding electrical signals that ultimately form user input 1452. For example, when user 1 moves a thumbstick of handheld controller 414 forward or backward or right or left, an electrical signal is generated.
[0220] When one or more input devices of the handheld controller 414 are manipulated by the user 1, corresponding electrical signals, such as one or more additional analog motion-sensitive signals, are generated by the motion-sensitive hardware. The one or more additional analog motion-sensitive signals are transmitted from the motion-sensitive hardware to an analog-to-digital converter of the handheld controller 414. The additional analog-to-digital converter of the handheld controller 414 converts the analog motion-sensitive signals from analog format to digital format to output additional digital motion-sensitive signals. The additional digital motion-sensitive signals are transmitted from the analog-to-digital converter to a processor of the handheld controller 414.
[0221] The processor of the handheld controller 414 applies an encoding protocol to encode the additional digital motion sensitive signal and output the additional encoded signal. For example, upon determining that the first additional digital motion sensitive signal represents the actuation of an option button on the handheld controller 414, the processor of the handheld controller 414 encodes the first additional digital motion sensitive signal using a first frequency. In this example, in response to determining that the second additional digital motion sensitive signal represents the movement of the left thumbstick 1302A of the handheld controller 414, the processor of the handheld controller 414 encodes the second additional digital motion sensitive signal using a second frequency. Also in this example, in response to determining that the third additional digital motion sensitive signal represents the movement of the right thumbstick 1302B of the handheld controller 414, the processor of the handheld controller 414 encodes the third additional digital motion sensitive signal using a third frequency.
[0222] The processor of the handheld controller 414 transmits the additional encoded signal to a communication device of the handheld controller 414. The communication device of the handheld controller 414 applies a wireless transport protocol to the additional encoded signal to generate one or more additional data packets that are examples of user input 1452. The communication device of the handheld controller 414 transmits the user input 1452 to the communication device 1320 of the input processor system 407 via a wireless connection.
[0223] Upon receiving the user input 1452, the communication device 1320 applies a near field communication protocol to extract an additional encoded signal from the one or more data packets and transmits the additional encoded signal to the input processor 1314. The input processor 1314 analyzes, such as decodes and analyzes, the additional encoded signal to determine additional input device information associated with the operation of the handheld controller 414. For example, the input processor 1314 identifies which of the input devices of the handheld controller 414 is being operated by the user 1, the amount of movement of the input device, and the direction of the amount of movement based on the one or more additional digital motion-sensitive signals. Explained, the input processor 1314 determines whether a particular button (e.g., an option button) on the handheld controller 414 is selected by the user 1 from one or more additional digital motion-sensitive signals generated from an additional analog motion-sensitive signal output from an accelerometer of the handheld controller 414. To further explain, upon determining that the first additional digital motion-sensitive signal is encoded using a first frequency, the input processor 1314 determines that an option button is selected, and in response to determining that the second additional digital motion-sensitive signal is encoded using a second frequency, the input processor 1314 determines that the left thumbstick 1302A is being manipulated by user 1.
[0224] As another explanation, the input processor 1314 determines from one or more additional digital motion-sensitive signals generated from additional analog motion-sensitive signals output from the accelerometer, gyroscope, and magnetometer of the handheld controller 414 whether the thumbstick is moved in a particular direction, such as forward, backward, right, left, etc., and whether the amount of movement exceeds a pre-determined threshold in the particular direction. To further explain, the input processor 1314 determines that the amount of movement of the left thumbstick 1302A exceeds a predetermined amount to determine that the amount of movement exceeds a predetermined threshold. In this explanation, the movement in a particular direction and the amount of movement are examples of thumbstick functionality. Another example of a function of a button of the handheld controller 414 is selection of a button by user 1. In this example, the function of the button is determined when the button is identified by the input processor 1314 as being selected by user 1. As another example of a function of a button of the handheld controller 414, the input processor 1314 determines the number of times that an additional digital motion-sensitive signal is generated when user 1 operates the button. The number of times is equal to the number of times that an additional digital motion-sensitive signal is determined to be received by the input processor 1314 from the handheld controller 414. This number is also the same as the number of times that button was selected by User 1.
[0225] Upon receiving additional input device information during play of legacy game N, the input emulator 1318 accesses the current mapping 1404 stored in the memory device 1316 to identify one or more legacy input devices of the handheld controller 1306 based on the identifiers 1312 of the one or more updated input devices of the handheld controller 414. For example, upon receiving the additional input device information, the input emulator 1318 determines from the current mapping 1404 one or more legacy input device identifiers 1310 that correspond to one or more of the updated input device identifiers 1312 of the one or more updated input devices of the handheld controller 414 operated by User 1.
[0226] Further, the input emulator 1318 determines one or more functions of one or more legacy input devices of the handheld controller 1306 from one or more functions of the handheld controller 414 operated by the user 1 to generate additional input device information. To explain, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a predetermined threshold corresponds to three selections of the left button 1304A by the user 1. To explain another, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a first predetermined threshold corresponds to two selections of the left button 1304A by the user 1. In this explanation, the input emulator 1318 determines that a leftward movement of the left thumbstick 1302A that exceeds a second predetermined threshold corresponds to three selections of the left button 1304A by the user 1.
[0227] The input emulator 1318 generates a user input 1454 having one or more legacy input device identifiers 1310 that correspond to one or more updated input device identifiers 1312 of the one or more updated input devices of the handheld controller 414 operated by the user 1. The user input 1454 also includes one or more functions of the one or more legacy input devices identified by the one or more legacy input device identifiers 1310. The input emulator 1318 sends the user input 1454 to the basic block compiler 104.
[0228] Based on the user input 1454, the basic block compiler 104 determines whether one or more of the basic blocks 1-n for processing the user input 1454 are stored in the cache 102 (FIG. 1). For example, based on the user input 1454, the basic block compiler 104 determines that one or more of the basic blocks 1-n for moving a fighter from position P1 to position P2 during playing of the legacy game N are stored in the cache 102. To explain, based on the selection of the left movement button 1304A, the basic block compiler 104 determines that the fighter is moved from a first position to a second position, or from a first direction to a second direction, or a combination thereof. The basic block compiler 104 further determines whether the execution of one or more of the basic blocks 1-n stored in the cache 102 enables the movement of the fighter from a first position to a second position, or from a first direction to a second direction, or a combination thereof, according to the selection of the left movement button 1304A.
[0229] Upon determining that one or more of the basic blocks 1-n for processing the user input 1454 are stored in the cache 102, the basic block compiler 104 sends instructions to the block dispatcher 302 (FIG. 3). In response to the instructions, the block dispatcher 302 executes one or more of the basic blocks 1-n from the cache 102 to process the user input 1454. For example, one or more of the basic blocks 1-n are executed for moving the fighter plane from a first position to a second position, or for moving from a first direction to a second direction, or a combination thereof. Furthermore, one or more of the basic blocks 1-n are executed to generate one or more image frames in which the fighter plane moves from a first position to a second position, or for moving from a first direction to a second direction, or a combination thereof.
[0230] On the other hand, in response to determining that one or more of the basic blocks 1-n for processing the user input 1454 are not stored in the cache 102, the basic block compiler 102 identifies one or more of the emulated PU code instructions 1-M for processing the user input 1454. The basic block compiler 102 compiles the one or more of the emulated PU code instructions 1-M to generate one or more of the basic blocks 1-n for processing the user input 1454 and provides the one or more of the basic blocks to the cache 102 for storage. The basic block compiler 102 also sends instructions to the block dispatcher 302 to execute one or more of the basic blocks 1-n that were recently stored in the cache 102.
[0231] In one embodiment, the communication device of the handheld controller 414 is coupled to the motion-sensitive hardware of the handheld controller 414. The communication device applies a wired transport protocol to transmit the additional digital motion-sensitive signal to the communication device of the input processor system 407 via a different channel of the wired transport medium. For example, upon receiving a first additional digital motion-sensitive signal representing a movement of a button on the handheld controller 414, the communication device of the handheld controller 414 transfers the first additional digital motion-sensitive signal via a first channel of the wired connection. In this example, in response to receiving a second additional digital motion-sensitive signal representing a movement of the left thumbstick 1302A of the handheld controller 414, the communication device of the handheld controller 414 transfers the second additional digital motion-sensitive signal via a second channel of the wired connection. Also in this example, in response to receiving a third additional digital motion-sensitive signal representing movement of the right thumbstick 1302B of the handheld controller 414, the communication device of the handheld controller 414 transmits the third additional digital motion-sensitive signal via a third channel of the wired connection.
[0232] Continuing with the embodiment, the communication device of the input processor system 407 receives the additional digital motion sensitive signal and provides the additional digital motion sensitive signal along with the channel information to the input processor 1314. For example, the communication device of the input processor 407 indicates to the input processor 1314 that a first additional digital motion sensitive signal is received via a first channel, a second additional digital motion sensitive signal is received via a second channel, and a third additional digital motion sensitive signal is received via a third channel. The input processor 1314 identifies which of the input devices of the handheld controller 414 is operated by user 1 based on the channel.
[0233] 15 is a diagram of an embodiment of a system 1500 to illustrate identification of the right movement button 1302C based on movement of the right thumbstick 1302B. The system 1500 includes a handheld controller 414, a handheld controller 1306, an input processor system 407, and an emulation processor system 409.
[0234] The handheld controller 1306 includes a directional pad (Dpad) 1510 and a pad 1512. Buttons 1302A to 1302B are arranged on the directional pad 1510. Furthermore, a square button, a triangle button, a circle button, and an X button are arranged on the pad 1512. Furthermore, the handheld controller 1306 includes a left shoulder button 1506 and a right shoulder button 1508.
[0235] User 1 moves the right thumbstick 1302 in a right direction 1502. As shown in the close-up of the right thumbstick 1302B, as the user moves the right thumbstick in the right direction 1502, the moment exceeds a predetermined threshold 1504. The predetermined threshold 1504 is measured from a central axis 1516, which is a vertical axis that passes through the center of the right thumbstick 1302B. For example, the central axis 1516 passes through the center of gravity of the right thumbstick 1302B. To illustrate, there is a predetermined horizontal distance between the central axis 1516 and the predetermined threshold 1504. As an example, the predetermined threshold 1504 is a vertical axis.
[0236] The handheld controller 414 generates a user input 1514 including one or more digital motion sensitive signals when the right thumbstick 1302 is moved in the right direction 1502 and exceeds a predetermined threshold 1504. The one or more digital motion sensitive signals indicate that the right thumbstick 1302B is moved in the right direction 1502 and exceeds the predetermined threshold 1504. The user input 1514 is an example of a user input 419 (FIG. 13). The user input 1514 is sent from the handheld controller 414 to the input processor system 407.
[0237] The input processor 1314 receives one or more digital motion sensitive signals embedded within the user input 1514 and determines, based on a frequency channel, that the one or more digital motion sensitive signals identify that the right thumbstick 1302B has been moved by the user 1. The frequency channel is used to wirelessly transfer the one or more digital motion sensitive signals embedded within the user input 1514 from the handheld controller 414 to the input processor system 407. Based on an identifier of the frequency channel, such as a channel having a first frequency or a channel having a second frequency, the one or more digital motion sensitive signals identify that there is movement of the right thumbstick 1302B. For example, when the one or more digital motion sensitive signals are transferred over a channel at a first frequency, the input processor 1314 determines that the right thumbstick 1302B has been moved, and when the one or more digital motion sensitive signals are transferred over a channel at a second frequency, the input processor 1314 determines that the left thumbstick 1302A has been moved. In this example, the second frequency is greater than or less than the first frequency.
[0238] Additionally, the input processor 1314 further determines that the right thumbstick 1302B has been moved in the right direction 1502 based on the directionality indicated by the one or more digital motion-sensitive signals. For example, the one or more digital motion-sensitive signals are output based on one or more analog motion-sensitive signals generated by a gyroscope of the handheld controller 414, which measures the directionality of the movement of the right thumbstick 1302B.
[0239] The input processor 1314 also determines the amount of movement of the right thumbstick 1302B in the rightward direction 1502 based on one or more digital motion-sensitive signals output based on one or more analog motion-sensitive signals generated by a potentiometer of the handheld controller 414. The input processor 1314 further determines that the amount of movement of the right thumbstick 1302B exceeds a predetermined threshold 1504 stored in the memory device 1316.
[0240] The movement of the right thumbstick 1302B, the movement in the right direction 1502, and the movement in the right direction 1502 exceeding the predetermined threshold 1504 are examples of input device information, such as initial input device information or additional input device information. The input processor 1314 provides the input device information to the input emulator 1318. The input emulator 1318 receives the input device information and accesses a mapping 1530, such as a default mapping 1308 (FIG. 14A-2) or a current mapping 1404 (FIG. 14A-2), based on the input device information. For example, upon receiving an indication that the right thumbstick 1302B has been moved in the right direction 1502, the input emulator 1318 identifies from the default mapping 1308 that the movement of the right thumbstick 1302B corresponds to the selection of the right movement button 1302C. Additionally, the input emulator 1318 determines from the default mapping 1308 that the amount of movement of the right thumbstick 1302B corresponds to a predetermined number of selections of the right button 1302C (eg, two, three, or four).
[0241] The input emulator 1318 generates a user input 1520 that includes information indicating operation of the right button 1302C. This information indicates a selection of the right button 1302C and a predetermined number of times that the right button 1302C is selected. Note that the right button 1302C is not actually selected by the user 1. Rather, movement of the right thumbstick 1302B is emulated as a selection of the right button 1302C. The user input 1520 is an example of a user input 420 (FIG. 13). The input emulator 1318 provides the user input 1520 to the basic block compiler 104, which operates in the manner described above with reference to FIG. 13.
[0242] 15 is for moving the right thumbstick 1302B in a right direction 1502. In one embodiment, instead of the right thumbstick 1302B, the left thumbstick 1302A is moved in a right direction, a left direction, a forward direction, or a backward direction, and based on that movement, a user input is generated by the handheld controller 414. In one embodiment, instead of movement in the right direction 1502, movement in any other direction occurs, such as right, forward, backward, or a combination thereof.
[0243] In one embodiment, a wheel controller is used instead of the handheld controller 414. The wheel controller is in the form of a driving wheel. As an example, the wheel controller does not include a left thumbstick 1302A and a right thumbstick 1302B. Instead, the driving wheel is operated by the user 1 to move a virtual object of the legacy game N in a left direction, a right direction, a forward direction, and a backward direction. The driving wheel can be rotated clockwise or counterclockwise. When the driving wheel is rotated clockwise by the user 1, the input processor 1314 determines that the driving wheel has been rotated and whether the driving wheel has been rotated clockwise over a counterclockwise direction based on one or more digital motion-sensitive signals. The input processor 1314 further determines whether the amount of rotation exceeds a preset threshold. The input processor 1314 provides information to the input emulator 1318, such as an identifier of the driving wheel, a direction of rotation, and an indication of whether the amount of rotation exceeds a preset threshold. The input emulator 1318 accesses a mapping, such as a default mapping or a current mapping, to identify a corresponding button on the handheld controller 1306 and the operation of the corresponding button to generate a user input. For example, the input emulator 1318 identifies, based on the information and the mapping, that a corresponding button on the handheld controller 1306 has been selected and the number of times the corresponding button has been selected to generate a user input. The input emulator 1318 provides the user input to the basic block compiler 104, which manipulates the user input in the manner described above with reference to FIG. 13 or 14B.
[0244] 16A is a diagram of an embodiment of a system 1600 to illustrate the generation of a virtual image of a crosshair when a touchpad 1602 is selected by a user 1. The touchpad 1602 is an example of a button. The system 1600 includes an input processor system 407, a handheld controller 414, and an emulation processor system 409. The handheld controller 414 includes a touchpad 1602, a directional pad 1604, a pad 1606, a left thumbstick 1302A, and a right thumbstick 1302B.
[0245] During play of legacy game N, user 1 uses his finger to select touchpad 1602. Touchpad 1602 is an example of one of the updated input devices of handheld controller 414. Motion-sensitive hardware of handheld controller 414 generates one or more analog motion-sensitive signals in response to a selection of touchpad 1602. For example, the one or more analog motion-sensitive signals indicate a selection of touchpad 1602 and a location of the selection of touchpad 1602 relative to a reference coordinate system of touchpad 1602. Explained, based on the location of the selection along an x-axis, a first potentiometer of handheld controller 414 generates a first electrical signal, and based on the location of the selection along a y-axis, a second potentiometer of handheld controller 414 generates a second electrical signal. The x-axis and y-axis associated with touchpad 1602 are described further below. The one or more analog motion-sensitive signals are converted to one or more digital motion-sensitive signals by an analog-to-digital converter in the handheld controller 414. The processor in the handheld controller 414 applies an encoding protocol to the one or more digital motion-sensitive signals to output one or more encoded signals. For example, the one or more encoded signals have a frequency that corresponds to the touch pad 1602, such that they have a unique relationship. The communication device in the handheld controller 414 applies a wireless transfer protocol to embed the one or more encoded signals into one or more data packets and transmits the one or more data packets to the communication device 1320 via a wireless connection. The one or more data packets are shown in FIG. 16A as user input 1612. The communication device 1320 applies a wireless transfer protocol to the one or more data packets received from the handheld controller 414 to obtain encoded signals and transmits the encoded signals to the input processor 1314.
[0246] The input processor 1314 determines or identifies that the touchpad 1602 has been selected by the user 1 in a manner similar to that described with reference to Figure 13. For example, in response to determining that one or more digital motion-sensitive signals encoded in one or more encoded signals are received at a frequency from the handheld controller 414 that corresponds to a predetermined frequency channel, the input processor 1314 determines that the touchpad 1602 has been selected. In this example, the predetermined frequency channel is stored in the memory device 1316. Also in this example, the input processor 1314 determines the frequency by applying a Fourier transform to the encoded signals.
[0247] The input processor 1314 further determines a location of a selection on the touchpad 1602 by user 1 relative to the reference coordinate system of the touchpad 1602. For example, the input processor 1314 determines that the touchpad 1602 is selected by user 1 at coordinate (x1,0) relative to the reference coordinate system of the touchpad 1602 based on a voltage amount received from a first potentiometer of the handheld controller 414. Further, in this example, the input processor 1314 determines that the touchpad 1602 is selected by user 1 at coordinate (0,y1) relative to the reference coordinate system of the touchpad 1602 based on a voltage amount received from a second potentiometer of the handheld controller 414. In this example, the first and second voltage amounts are received in one or more coded signals. To illustrate, one or more analog motion-sensitive signals output from the motion-sensitive hardware of the handheld controller 414 include information representative of the first and second voltage amounts. The identifier of the selected touchpad 1602 and the location of the selection on touchpad 1602 by User 1 are examples of input device information, such as initial input device information and additional input device information.
[0248] The input processor 1318 provides the identifier of the selected touchpad 1602 and the selection location on the touchpad 1602 to the input emulator 1318. The input emulator 1318 accesses the mapping 1610 to identify a corresponding button on the handheld controller 1306 based on the identifier of the selected touchpad 1602. For example, the input emulator 1318 determines from the mapping 1610 that the selected button on the handheld controller 1306 has a unique relationship to the touchpad 1602. The mapping 1610 is an example of the default mapping 1308 (FIG. 13) or the current mapping 1404 (FIG. 14A-2).
[0249] Also, the input emulator 1318 sends a request to the block dispatcher 302 to obtain the size of an image frame having a virtual scene of the legacy game N. As an example, the image frame of the legacy game N, whose size is requested, is generated after the user input 1614 is generated based on the user input 1612. To explain, the first image frame of the legacy game N, whose size is requested, is generated based on the user input 1614, and no other image frames are generated based on the user input 1614 before the first image frame is generated. The block dispatcher 302 provides the size of the image frame to the input emulator 1318. For example, the block dispatcher 302 provides the size of the image frame to be the same as the preceding image frame generated immediately before the image frame generated based on the user input 1614. To explain, there are no other image frames generated by the block dispatcher 302 between the generation of the image frame generated based on the user input 1614 and the previous image frame. The input emulator 1318 is coupled to the block dispatcher 302.
[0250] The input emulator 1318 determines a location of a virtual image of the crosshairs in an image frame that is generated based on user input 1614. The location of the virtual image of the crosshairs in the image frame corresponds to a selection location on the touchpad 1602. An explanation of how the location of the virtual image of the crosshairs in the image frame has a one-to-one relationship with a selection location on the touchpad 1602 is described with reference to FIG.
[0251] The input emulator 1318 generates user input 1614 having an indication of a selection button on the handheld controller 1306 and a location of a virtual image of a crosshair within an image frame that is generated based on the user input 1614. The user input 1614 is provided from the input emulator 1318 to the block compiler 104.
[0252] Upon receiving the user input 1614, the block compiler 104 determines whether one or more of the basic blocks 1-n for processing the user input 1614 are stored in the cache 102 (FIG. 1). In response to determining that one or more of the basic blocks 1-n are stored in the cache 102, the block dispatcher 302 (FIG. 3) executes one or more of the basic blocks 1-n to generate one or more image frames for processing the user input 1614. For example, the block dispatcher 302 (FIG. 3) receives a location of a virtual image of a crosshair in the generated image frame from the block compiler 104 and executes one or more of the basic blocks 1-n to generate the crosshair as a virtual object in the image frame. Further, the crosshair is generated in the image frame at a location corresponding to the selection location of the touchpad 1602 by the user 1.
[0253] In one embodiment, the input emulator 1318 generates a command for the block dispatcher 302 to determine the location of the virtual image of the crosshairs within the image frame and sends the command to the block dispatcher 302. For example, the command indicates to the block dispatcher 302 that the image frame generated by the block dispatcher 302 is to include the virtual image of the crosshairs at a location proportional to the location of the selection on the touchpad 1602 by User 1. The command includes the location of the selection on the touchpad 1602 by User 1. The input emulator 1318 is coupled to the block dispatcher 302. In this embodiment, the user input 1614 does not include the location of the virtual image of the crosshairs within the image frame. The block dispatcher 302 executes one or more of the basic blocks 1-n to generate the image frame. Further, the block dispatcher 302 determines the location of the virtual image of the crosshairs within the image frame in the same manner as described herein, such as with reference to FIG. 16B, where the input emulator 1318 determines the position of the virtual image of the crosshairs within the image frame based on the location of the selection on the touchpad 1602 by user 1.
[0254] 16B is a diagram of an embodiment of a system 1650 showing a correspondence between the location of a selection on a touchpad 1062 by a user 1 and the location of a crosshair 1652 on an image frame 1654 having a virtual environment 452. The correspondence is determined by an input emulator 1318 (FIG. 16A). The image frame 1654 is displayed on a display screen of a display device 410 (FIG. 4B). The system 1650 includes a touchpad 1602 and an image frame 1654.
[0255] The touchpad 1602 has a top edge 1666A, a right edge 1666B, a bottom edge 1666C, and a left edge 1666D. The top edge 1666A is adjacent to and connected to the right and left edges. The bottom edge 1666C is adjacent to and connected to the right and left edges. The bottom edge 1666C faces the top edge 1666A, and the right edge 1666B faces the left edge 1666D.
[0256] A selection location on the touchpad 1602 has coordinates (x1, y1). The touchpad 1602 is associated with a reference coordinate system that has an origin (0,0) at a corner of the touchpad 1602. The touchpad 1602 has four corners, such as the four vertices of a rectangle. As an example, the touchpad 1602 is a rectangle with vertices. The x-axis of the reference coordinate system for the touchpad 1602 is along the bottom edge 1666C of the touchpad 1602, and the y-axis of the reference coordinate system is along the left edge 1666D of the touchpad 1602. Coordinate x1 is the distance along the x-axis from the origin (0,0), and coordinate y1 is the distance along the y-axis from the origin (0,0). The x-axis of the reference coordinate system is perpendicular to the y-axis of the reference coordinate system.
[0257] Similarly, image frame 1654 has a top edge 1668A, a right edge 1668B, a bottom edge 1668C, and a left edge 1668D. Top edge 1668A is adjacent to and connected to the right and left edges, and bottom edge 1668C is adjacent to and connected to the right and left edges of image frame 1654. Bottom edge 1668C faces top edge 1668A, and right edge 1668B faces left edge 1668D.
[0258] The location of the crosshairs 1652 is at coordinates (X1,Y1). The image frame 1654 is associated with a reference coordinate system that has an origin (0,0) at a corner of the image frame 1654. The image frame 1654 has four corners, such as the four vertices of a rectangle. As an example, the image frame 1654 is a rectangular image frame with vertices. The X-axis of the reference coordinate system of the image frame 1654 is along the bottom edge 1668C, and the Y-axis of the reference coordinate system of the image frame 1654 is along the left edge 1668D. The coordinate X1 is the distance along the X-axis from the origin (0,0) at a corner of the image frame 1654, and the coordinate Y1 is the distance along the Y-axis from the origin (0,0).
[0259] The input emulator 1318 determines that the distance along the X-axis of the reference coordinate system of the image frame 1654 is proportional to the distance along the x-axis of the reference coordinate system of the touchpad 1602. For example, the input emulator 1318 determines that the width of the image frame 1654 is P times the width of the touchpad 1602, where P is an integer greater than zero. The width of the image frame 1654 is measured along the X-axis of the reference coordinate system of the image frame 1654 and the width of the touchpad 1602 is measured along the x-axis of the reference coordinate system of the touchpad 1602. In this example, the input emulator 1318 determines that the height of the image frame 1654 is Q times the height of the touchpad 1602, where Q is an integer greater than zero. The height of the image frame 1654 is measured along the Y-axis of the reference coordinate system of the image frame 1654 and the height of the touchpad 1602 is measured along the y-axis of the reference coordinate system of the touchpad 1602. In this example, the input emulator 1318 determines that the distance along the X axis is P times the distance along the x axis of the reference coordinate system of the touchpad 1602. Also in this example, the input emulator 1318 determines that the distance along the Y axis is Q times the distance along the x axis of the reference coordinate system of the touchpad 1602.
[0260] 17 is a diagram of an embodiment of a system 1700 illustrating the use of different mappings with different types of handheld controllers. System 1700 includes a wheel controller 1702, a handheld controller 414, and an input processor system 407. Wheel controller 1702 has the shape of a driving wheel and includes updated input devices such as a steering wheel 1708, a directional pad 1706, a square button, a triangle button, a circle button, and an X button.
[0261] It should be noted that the wheel controller 1702 is of a different type than the handheld controller 414. For example, the handheld controller 414 has a left handle 1704A and a right handle 1704B, but does not include a steering wheel. The handles 1704A and 1704B extend downward from the body of the handheld controller 414. In comparison, the wheel controller 1702 has a steering wheel 1708, but does not include left or right handles.
[0262] Additionally, when both a first handheld controller and a second handheld controller are of the same type, but manufactured by different entities, the subtype of the first handheld controller is a different subtype from the subtype of the second handheld controller. For example, a wheel controller manufactured by Logitech® corporation is a different subtype of wheel controller than a wheel controller manufactured by Sony® corporation. As another example, a handheld controller manufactured by Sony® corporation is a different type of handheld controller than a handheld controller manufactured by Microsoft® corporation.
[0263] The wheel controller 1702 has internal components that are the same as or similar to the internal components of the handheld controller 414. For example, the wheel controller 1702 includes motion sensitive hardware, an analog-to-digital converter, a processor, and a communication device. The internal components of the wheel controller 1702 are coupled to each other in the same manner that the internal components of the handheld controller 414 are coupled to each other.
[0264] When user 1 couples an updated controller, such as handheld controller 414 or wheel controller 1702, to input processor system 407, user 1 operates one or more updated input devices of the updated controller to provide the updated controller type and subtype. For example, user 1 disconnects handheld controller 414 from input processor system 407 and couples wheel controller 1702 to input processor system 407 instead. When the updated controller is operated by user 1, the updated controller generates an identification signal having the updated controller type and transmits the identification signal to input processor system 407. The updated controller also generates an identification signal having the updated controller subtype and transmits the identification signal to input processor system 407. The input processor 1314 receives the updated controller type and subtype and provides the type and subtype to input emulator 1318.
[0265] Upon receiving the updated controller type and subtype, the input emulator 1318 selects mapping 1710 or mapping 1712 based on the type and subtype. For example, the input emulator 1318 selects mapping 1710 when the updated controller type is handheld controller 414 and the updated controller subtype is handheld controller 414 manufactured by Sony® corporation. The input emulator selects mapping 1712 when the updated controller type is wheel controller 1702 and the updated controller subtype is wheel controller 1702 manufactured by Sony® corporation. An example of mapping 1710 is a default mapping for user account 1 or a current mapping for user account 1. An example of mapping 1712 is a default mapping for user account 1 or a current mapping for user account 1.
[0266] During play of the legacy game N, the input processor system 407 receives user input 419 from a wheel controller 1702 coupled to the input processor system 407 after detaching the handheld controller 414. The user input 419 is generated by the wheel controller 1702 in the same manner, except that the driving wheel 1708 is rotated instead of the thumbsticks 1302A and 1302B, and the user input 419 is generated by the handheld controller 414. For example, the user 1 manipulates one or more of the updated input devices of the wheel controller 1702. The motion-sensitive hardware of the wheel controller 1702 generates one or more analog motion-sensitive signals in response to the manipulation of the one or more updated input devices of the wheel controller 1702. The analog-to-digital converter of the wheel controller 1702 converts the one or more analog motion-sensitive signals to one or more digital motion-sensitive signals. The processor of the wheel controller 1702 applies an encoding protocol to encode the one or more digital sensitive signals to output one or more encoded signals. The communication device of the wheel controller 1702 applies a wireless transfer protocol to the one or more encoded signals to generate one or more data packets and transmits the one or more data packets to the communication device 1320.
[0267] The communication device 1320 applies a wireless transport protocol to the one or more data packets to extract one or more encoded signals and transmits the one or more encoded signals to the input processor 1314. The input processor 1314 identifies from the one or more encoded signals which of the updated input devices of the wheel controller 1702 are being operated by the user 1 and provides one or more identifiers of the one or more updated input devices to the input emulator 1318. The input emulator 1318 accesses and applies the mapping 1712 to identify the corresponding one or more legacy input devices of the handheld controller 1306 (FIG. 13) to generate the user input 420.
[0268] 18 is a diagram of an embodiment of a system 1800 for illustrating communication of a current mapping from a game console 402 to a game console 1202. The system 1800 includes a game console 402, a computer network 408, a server system 404, and a game console 1202.
[0269] The game console 1202 includes an input processor system 1207, which includes an input processor 1804, an input emulator 1806, a memory device 1808, and a communication device 1810. Illustratively, the input emulator 1806, when implemented within the game console 1202, enables playing of a legacy game N by using a handheld controller 1212. An example of the communication device 1810 is a communication device that applies a close-range communication protocol.
[0270] The communication device 1810 is coupled to the input processor 1804. The input processor 1804 and the input emulator 1806 are coupled to a memory device 1808. The input processor system 1207 is coupled to the bus 1218.
[0271] The input processor 1314 of the game console 402 accesses (e.g., reads) the current mapping 1404 from the memory device 1316 and transmits the current mapping 1404 to the NIC 1212. For example, within a predetermined period of time after the current mapping 1404 is generated, the input processor 1314 transmits the current mapping 1404 to the NIC 1212. Explaining, immediately after the current mapping 1404 is created, the current mapping 1404 is stored in the memory device 1316 and simultaneously transmitted to the NIC 1212. The NIC 1212 applies a network communication protocol to the current mapping 1404 to generate one or more data packets and transmits the one or more data packets to the server system 404 via the computer network 408 for storage in one or more memory devices of the server system 404.
[0272] User 1 logs into user account 1 using handheld controller 1212 and accesses legacy game N via game console 1202. Additionally, prior to or during play of legacy game N, user 1 selects one or more updated input devices of handheld controller 1212 to generate an indication (e.g., a request) to access current mapping 1404 from server system 404. Upon receiving the selection of the one or more updated input devices of handheld controller 1212, handheld controller 1212 applies a wireless transfer protocol to generate one or more data packets and transmits the one or more data packets to communication device 1810. Communication device 1810 applies a wireless transfer protocol to obtain an indication that current mapping 1404 should be retrieved for user account 1 and stores the indication in memory device 1808 of game console 1202.
[0273] A CPU, such as CPU 922 (FIG. 9B) of game console 1202, accesses instructions for obtaining current mapping 1404 from memory device 1808 and sends the instructions to NIC 1210. NIC 1210 applies a network communication protocol to the instructions for obtaining current mapping 1404 to generate one or more data packets and sends the one or more data packets over computer network 408 to server system 404.
[0274] In response to receiving the indication of the current mapping 1404 from the game console 1202, the server system 404 accesses the current mapping 1404 from one or more memory devices of the server system 404 and transmits the current mapping 1404 to the game console 1202 over the computer network 408. For example, the processor of the server system 404 determines that the indication to obtain the current mapping 1404 corresponds to user account 1. To illustrate, the processor of the server system 404 determines that the indication corresponds to user account 1, determining that the indication was received while user 1 was logged into user account 1. Continuing with the example, the processor of the server system 404 accesses the current mapping 1404 from one or more memory devices of the server system 404 and transmits the current mapping 1404 to a network interface controller of the server system 404. The network interface controller of the server system 404 applies a network communication protocol to the current mapping 1404 to generate one or more data packets and transmits the one or more data packets to the game console 1202 over the computer network 408.
[0275] The NIC 1210 of the game console 1202 extracts the current mapping 1404 from the one or more data packets received from the server system 404 and stores the current mapping 404 in the memory device 1808. The user input 1452 is received from the handheld controller 1212 by the input processor system 1207 in the same manner that the user input 1452 is received from the handheld controller 414 by the input processor system 407. Once the user input 1452 is received, the input emulator 1806 accesses the current mapping 1404 from the memory device 1808 to determine the user input 1454. For example, the user input 1454 is determined by the input emulator 1806 from the user input 1452 in the same manner that the user input 1454 is determined by the input emulator 1318 from the user input 1452. The input emulator 1806 transmits the user input 1454 to the emulation processor system 1206 via the bus 1218.
[0276] Upon receiving the user input 1454, the basic block compiler of the emulation processor system 1206 checks the cache 1208 of the emulation processor system 1206 to determine whether the cache 1208 includes one or more of the basic blocks 1-n of the game code GCN for processing the user input 1454. If it is determined that the cache 1208 does not include one or more of the basic blocks 1-n for processing the user input 1454, the basic block compiler of the emulation processor system 1206 accesses one or more of the emulated PU code instructions 1-M from one or more memory devices of the emulation processor system 1206 for processing the user input 1454, and compiles one or more of the emulated PU code instructions 1-M to generate one or more of the basic blocks 1-n for processing the user input 1454. The basic block compiler of the emulation processor system 1206 sends one or more of the basic blocks 1-n to a block dispatcher of the emulation processor system 1206. Upon receiving one or more basic blocks 1-n, the block dispatcher of the emulation processor system 1206 executes one or more of the basic blocks 1-n to generate one or more image frames based on the user input 1454. Meanwhile, upon determining that the cache 1208 includes one or more of the basic blocks 1-n for processing the user input 1454, the basic block compiler of the emulation processor system 1206 sends instructions to the block dispatcher of the emulation processor system 1206. In response to receiving the instructions, the block dispatcher of the emulation processor system 1206 executes one or more of the basic blocks 1-n to process the user input 1454.
[0277] 19A is a screen shot of an image 1900 illustrating receipt of an updated type and subtype of a handheld controller, such as handheld controller 414 or wheel controller 1702 (FIG. 17) or handheld controller 1212 (FIG. 18). Image 1900 includes a virtual environment 452 of legacy game N (FIG. 4B).
[0278] During or before playing legacy game N, user 1 detaches a first updated handheld controller from input processor system 407 and couples a second updated handheld controller to input processor system 407. User 1 then manipulates one or more updated input devices of the second updated handheld controller to access a menu 1902 that includes a setting option. A CPU, such as CPU 922 (FIG. 9B) of emulation processor system 409 instructs a GPU, such as GPU 924 (FIG. 9B) of emulation processor system 409 to display menu 1902 on a display screen of display device 410 (FIG. 4B). User 1 further manipulates one or more updated input devices of the second updated handheld controller to select a setting option. Once the setting option is selected, the CPU of emulation processor system 409 instructs the GPU of emulation processor system 409 to display another menu on a display screen of display device 410 (FIG. 4B). The other menu includes control and peripheral options. User 1 then operates one or more updated input devices of the second updated handheld controller to select the control and peripheral option. Once the control and peripheral option is selected, the CPU of emulation processor system 409 instructs the GPU of emulation processor system 409 to display menu 1912 (FIG. 19B) on the display screen of display device 410.
[0279] 19B is a screenshot of image 1910 to illustrate menu 1912. Menu 1910 includes a reassign button option to reassign buttons, a gun handheld controller option to indicate to input processor system 407 (FIG. 4B) that a gun shaped controller is to be used as the second updated handheld controller with input processor system 407, and a wheel handheld controller option to indicate to input processor system 407 that a wheel controller is to be used as the second updated handheld controller with input processor system 407 (FIG. 4B). A gun controller is an example of a type of updated handheld controller.
[0280] User 1 manipulates one or more updated input devices of the second updated handheld controller to select a wheel handheld controller option. When the wheel handheld controller option is selected, the CPU of emulation processor system 409 instructs the GPU of emulation processor system 409 to display a menu including a first subtype controller option and a second subtype controller option on the display screen of display device 410. The first subtype controller and the second subtype controller are examples of subtypes of a wheel controller.
[0281] User 1 manipulates one or more updated input devices of the second updated handheld controller to select a second subtype controller option. Upon receiving the selection of the second subtype controller option, the CPU of emulation processor system 409 instructs the GPU of emulation processor system 409 to continue or begin playing legacy game N.
[0282] 20 is an embodiment of a screenshot of an image 2000 to illustrate a mapping 2002, such as a default mapping. The image 2000 includes a virtual environment 452 of a legacy game N. The mapping 2002 is an example of a default mapping 1308 (FIG. 13). During or before playing the legacy game N, the CPU of the emulation processor system 409 instructs the GPU of the emulation processor 409 to display the mapping 2002 on the display screen of the display device 410.
[0283] The mapping 2022 includes an inherent relationship, such as a correspondence or one-to-one relationship, between an input device of the legacy handheld controller and an input device of the updated handheld controller. For example, a legacy input device 1 of the legacy handheld controller, such as a triangle button, is mapped to an updated input device 1 of the updated handheld controller, such as a triangle button. As another example, a legacy input device 2, such as a touchpad 1602 (FIG. 16A) of the updated handheld controller, corresponds to an updated input device 2, such as a select button of the legacy handheld controller. As yet another example, a legacy input device 3 of the legacy handheld controller has a one-to-one relationship with an updated input device 2 of the updated handheld controller.
[0284] 21A is a diagram of an embodiment of a system 2100 to illustrate receiving updated graphics parameters from User 1. The system 2100 includes a display device 410, a handheld controller 414, and a game console 402.
[0285] Game console 402 includes an emulation processor system 409, a memory device 406, and a GPU memory device 2104. Emulation processor system 409 includes a CPU 922 and a GPU 924. GPU memory device 2104 is coupled to bus 930. GPU memory device 2104 is an example of a memory device in memory system 926 (FIG. 9B).
[0286] After User 1 logs into User Account 1, during or before playing Legacy Game N, GPU 924 controls display device 410 to display a list of one or more updated graphic parameters, such as UGP1 and UGP2. For example, during playing Legacy Game N, GPU 924 renders the updated graphic parameters UGP1 and UGP2 on display device 410.
[0287] After the updated graphic parameters are displayed on the display device 410, the user 1 operates one or more updated input devices of the handheld controller 414 to select one or more updated graphic parameters, and the handheld controller 414 generates a user input 2102 based on the operation. The user input 2102 includes an indication of the selection of the one or more updated graphic parameters. The user input 2102 is transmitted from the handheld controller 414 to the CPU 922 of the game console 402. The CPU 922 obtains the one or more updated graphic parameters from the user input 2102 and provides the one or more updated graphic parameters to the GPU 924. The GPU 924 emulates the one or more legacy graphic parameters into the one or more graphic parameters.
[0288] In one embodiment, any number of updated graphical parameters are displayed on the display device 410.
[0289] 21B-1 is a diagram of an embodiment of a system 2120 for illustrating emulation of one or more legacy graphics parameters to one or more updated graphics parameters. The system 2120 includes a handheld controller 414 and a game console 402.
[0290] The GPU 924 includes a GPU emulator 2122. As an example, as described herein, the GPU emulator is hardware or software, or a combination thereof. Illustratively, the GPU emulator is a processor, PLD, ASIC, or controller. As another example, the GPU emulator is a computer program.
[0291] When an indication of selection of one or more updated graphics parameters is received from the handheld controller 414, the CPU 922 provides the updated graphics parameters to the GPU 924. The GPU 924 stores the updated graphics parameters in the GPU memory device 2104. In response to the user input 420, the GPU 924 executes one or more of the basic blocks 1-n of the updated code to determine that one or more legacy graphics parameters, such as legacy graphics parameter LGP1 and legacy graphics parameter LGP2, are applied to the virtual object and virtual background in the image frame 2123 of the legacy game N. The image frame 2123 is the updated image frame. It should be noted that the user input 419 from which the user input 420 is generated is received before the user input 2102 for applying the one or more updated graphics parameters is received.
[0292] Upon determining the one or more legacy graphics parameters by the GPU 924 to generate one or more legacy image frames, such as image frame 422 (FIG. 4B), the GPU emulator 2122 applies the one or more updated graphics parameters to the legacy image frames instead of the one or more legacy graphics parameters, and outputs image frame 2123. The application of the one or more updated graphics parameters may be referred to as rendering an image frame of the virtual environment of the legacy game N. For example, the GPU emulator 2122 identifies an updated graphics parameter UGP1 from the GPU memory device 2104 as corresponding to the legacy graphics parameter LGP1, and an updated graphics parameter UGP2 as corresponding to the legacy graphics parameter LGP2. In this example, the updated graphics parameter UGP1 and the legacy graphics parameter LGP1 are of the same type. To explain, the updated graphics parameter UGP1 is an updated resolution of the virtual image of the virtual object, and the legacy graphics parameter LGP1 is a legacy resolution of the virtual image of the virtual object. As another example, the updated graphic parameters UGP2 are a set of colors applied to the virtual object from a 32-bit color palette, and the legacy graphic parameters LGP2 are a set of colors applied to the virtual object from a 16-bit color palette. As another example, both the graphic parameters LGP1 and UGP1 include the same identifier (e.g., an alphanumeric code) of a type of graphic parameter. In this example, the GPU emulator 2122 determines that the updated graphic parameters UGP1 correspond to the legacy graphic parameters LGP1 in response to determining that both the graphic parameters UGP1 and LGP1 include the same identifier. The GPU 924 displays an image of the virtual environment, such as a virtual environment having virtual objects and a virtual background of the virtual environment 452, on the display device 410 according to the image frame 2123. For example, the GPU 924 controls the light emitters of the display device 410 to display the image according to the image frame 2123.An image of the virtual environment is displayed on a display device 410 to show the play of legacy game N.
[0293] In one embodiment, the legacy GPU 904 (FIG. 9A) executes one or more of the emulated PU code instructions 1-M. Execution of the emulated PU code instructions 1-M may not generate updated graphics parameters. Rather, legacy graphics parameters are generated.
[0294] In one embodiment, the CPU 922 executes the basic block compiler 104 and the GPU 924 executes the block dispatcher 302 .
[0295] In one embodiment, the CPU 922 executes the basic block compiler 104 and the block dispatcher 302 .
[0296] In one embodiment, the updated graphics parameters do not correspond to legacy graphics parameters (e.g., there is no legacy graphics parameter corresponding to the updated graphics parameter UPG1). Instead of applying any of the legacy graphics parameters, the GPU emulator 2122 applies the updated graphics parameter UPG1 during rendering operations performed during play of the legacy game N.
[0297] In one embodiment, the GPU emulator 2122 applies an interpolation operation, such as nearest, or bilinear, or bicubic, or a combination thereof, to the legacy image frame to estimate updated graphics parameters to be applied to legacy image pixels of the legacy image frame for which the legacy graphics parameters are unknown. For example, the GPU emulator 2122 interpolates color and intensity values of a first image pixel of the virtual environment 452 to a second image pixel of the virtual environment 452 to output an updated image frame having the first and second image pixels. In this example, upon execution of one or more basic blocks 1-n of the updated code, the legacy graphics parameters, such as color and intensity values, of the second image pixel are unknown. In this example, the first and second image pixels are part of an updated image frame, such as one of the updated image frames 2123.
[0298] In one embodiment, the GPU emulator 2122 applies one or more interpolation operations and one or more convolution operations to the legacy image frame to estimate updated graphic parameters to be applied to the legacy image pixels of the legacy image frame. For example, after applying bicubic interpolation, the GPU emulator 2122 applies a 9X9 convolution, a 1X1 convolution, and a 5X5 convolution to the legacy image frame to output an updated image frame. For example, in an nXm convolution, the GPU emulator 2122 takes nXm image pixels of the legacy image frame, convolves the nXm image pixels with nXm kernels, and outputs the resulting values of the updated graphic parameters, where n and m are positive integers, where n represents the row number of the image pixel or kernel pixel, and m represents the column number of the kernel pixel of the image pixel. For example, n is 4 and m is 2. As another example, both m and n are 4. The values of the updated graphics parameters of the nX1 pixels of the legacy image frame to which the nXm kernel is applied are replaced with the resulting values.
[0299] In one embodiment, instead of applying the convolution to the image pixels of the legacy image frame, the GPU emulator 2122 divides the image pixels into sub-pixels and applies the convolution to the sub-pixels. In this embodiment, the convolution is applied to the sub-pixels before or after applying the convolution to the image pixels.
[0300] In one embodiment, the GPU 924 applies a deep learning approach. In the deep learning approach, the GPU emulator 2122 receives a legacy image frame, accesses a database in which a comparison image frame is stored, compares the legacy image frame with the comparison image frame, and outputs an updated image frame. As an example, the comparison image frame has one or more virtual objects that are the same as the legacy image frame. In this example, the comparison image frame has a comparison fighter plane, and the legacy image frame has a virtual object 454. Continuing with this example, the GPU emulator 2122 compares the comparison fighter plane with the virtual object 454 to determine whether the virtual object 454 is modified based on the comparison fighter plane. Continuing with this example, the GPU emulator 2122 determines that at least a portion (e.g., a wing) of the virtual object 454 is replaced with at least a corresponding portion (e.g., a wing) of the comparison fighter plane. In this example, upon receiving the decision from the GPU emulator 2122, the GPU 924 replaces at least a portion of the virtual object 454 with at least a corresponding portion of the comparison fighter jet to output the resulting fighter jet, and further outputs an updated image frame. The updated image frame is stored in a database by the GPU 924. Note that the database is stored in the GPU memory device 2104.
[0301] In one embodiment, the GPU applies a recursive learning technique. The recursive learning technique includes a deep learning technique. In the recursive learning technique, the GPU emulator 2122 receives an additional legacy image frame, accesses a database in which updated image frames are stored, compares the additional legacy image frame with the updated image frame, and outputs the additional updated image frame. In one example, the updated image frame has one or more of the same virtual objects as those of the additional legacy image frame. In this example, the updated image frame has a resultant fighter plane, and the additional legacy image frame has a virtual object 454. Continuing with this example, the GPU emulator 2122 compares the resultant fighter plane with the virtual object 454 to determine whether the virtual object 454 is modified based on the resultant fighter plane. Continuing with this example, the GPU emulator 2122 determines that at least a portion (e.g., a wing) of the virtual object 454 should be replaced with at least a corresponding portion (e.g., a wing) of the resultant fighter plane. In this example, upon receiving the decision from GPU emulator 2122, GPU 924 replaces at least a portion of virtual object 454 with at least a corresponding portion of the resulting fighter jet to output an additional resulting fighter jet, and also outputs an additional updated image frame, which is stored by GPU 924 in a database for application to a next legacy image frame, which is received by GPU emulator 2122 subsequent to receipt of the additional legacy image frame.
[0302] FIG. 21B-2 is a diagram of an embodiment of a system 2130 to illustrate generation of an updated image frame 2123 from an image frame 422, such as a legacy image frame output from a block dispatcher 302. The system 2130 includes a block dispatcher 302 and a GPU emulator 2122. The block dispatcher 302 is coupled to the GPU emulator 2122. The block dispatcher 302 executes one or more of the basic blocks 1-n of the updated code to generate an image frame 422 including legacy graphics parameters. The GPU emulator 2122 receives the image frame 422, analyzes the image frame 422 to identify legacy graphics parameters, converts the legacy graphics parameters to updated graphics parameters, and outputs the updated image frame 2123.
[0303] In one embodiment, each image frame is an image file, for example a first image frame is a first image file and a second image frame is a second image file.
[0304] FIG. 21C-1 is an embodiment of a list of legacy and updated graphic parameters, and the correspondence between the legacy and updated graphic parameters. An example of the legacy graphic parameters LGP1 or LGP2 is a legacy image resolution, such as 640X480 pixel resolution, 720X480 pixel resolution, and 720X586 pixel resolution of a virtual image of a virtual environment. In this example, the updated graphic parameters UGP1 corresponding to the legacy graphic parameters LGP1, or the updated graphic parameters UGP2 corresponding to the legacy graphic parameters LGP2 are updated image resolutions. To explain, the updated resolutions include a high definition (HD) resolution, which is four times the pixel resolution of the legacy graphic parameters, and a 4K resolution, which is eight times the pixel resolution of the legacy graphic parameters.
[0305] Another example of the legacy graphic parameter LGP1 or LGP2 is the thickness of a legacy line. A legacy line is a line image of one or more parts of one or more virtual objects of a virtual scene of a legacy game N and a part of a virtual background of the virtual scene. In this example, the updated graphic parameter UGP1 corresponding to the legacy graphic parameter LGP1 or the updated graphic parameter UGP2 corresponding to the legacy graphic parameter LGP2 is an updated line thickness. By way of illustration, the updated line thickness is a multiple or factor of the legacy line thickness.
[0306] Yet another example of the legacy graphic parameter LGP1 or LGP2 is a legacy line count, which is defined by the number of legacy lines having pixels that form an image of the virtual environment of the legacy game N. In this example, the updated graphic parameter UGP1 corresponding to the legacy graphic parameter LGP1, or the updated graphic parameter UGP2 corresponding to the legacy graphic parameter LGP2, is an updated line count. By way of illustration, the updated line count is a multiple or factor of the legacy line count.
[0307] Yet another example of an updated graphic parameter UGP1 or UGP2 is an updated scanline thickness, which is defined by each scanline thickness overlaid on an image frame of a virtual image of a virtual environment of a legacy game N. Explained, each scanline is overlaid on a legacy image frame. Each scanline is overlaid so as to extend from one edge of the virtual image to an opposite edge of the virtual image. Note that in this example, there is no corresponding legacy graphic parameter that corresponds to the updated scanline thickness.
[0308] Another example of an updated graphics parameter UGP1 or UGP2 is an updated scanline count, which is defined by the number of scanlines that are overlaid on the virtual image of the virtual environment of legacy game N. Note that in this example, there is no corresponding legacy graphics parameter that corresponds to the updated scanline count.
[0309] However, examples of the legacy graphic parameters LGP1 or LGP2 are a legacy anti-aliasing filter and a legacy video filter. The legacy anti-aliasing filter and the legacy video filter reduce or remove noise, such as blurred color and brightness, that is output and evident within the pixels of the virtual image of the virtual environment of the legacy game N. In this example, the updated graphic parameters UGP1 corresponding to the legacy graphic parameters LGP1, or the updated graphic parameters UGP2 corresponding to the legacy graphic parameters LGP2, are an updated anti-aliasing filter and an updated video filter. By way of explanation, the updated anti-aliasing filter and the video filter reduce or remove noise at a higher resolution than the resolution at which the legacy anti-aliasing filter and the legacy video filter remove or reduce noise. In this explanation, the noise is removed or reduced by smoothing the polygon edges of the polygons used to represent the virtual objects of the legacy game N, or by interpolating the brightness of the pixels of the virtual background of the virtual scene of the legacy game N.
[0310] Yet another example of the legacy graphic parameters LGP1 or LGP2 is a legacy color scale, such as a legacy color space or a legacy color palette, which is defined by the number of bits that represent each color in the legacy color scale. An explanation of the legacy color space is the video graphics array (VGA) color space. Each color in the legacy color palette is represented by 16 bits. The colors in the legacy color palette may be applied to represent images of a virtual environment of a legacy game N. In this example, the updated graphic parameters UGP1 corresponding to the legacy graphic parameters LGP1, or the updated graphic parameters UGP2 corresponding to the legacy graphic parameters LGP2, are the updated color scales. As an explanation, the updated color scale is a 32-bit color palette. Each color in the updated color scale is represented by 32 bits instead of 16 bits.
[0311] Another example of the legacy graphic parameters LGP1 or LGP2 is a legacy rendering rate, such as 20 frames per second (fps) or 30 fps, at which the legacy image frames are rendered and displayed on the display device 410. In this example, the updated graphic parameters UGP1 corresponding to the legacy graphic parameters LGP1 or the updated graphic parameters UGP2 corresponding to the legacy graphic parameters LGP2 are the updated rendering rates, which are multiples of the legacy rendering rates. By way of illustration, the updated rendering rate is 60 fps or 120 fps, at which the updated image frames are rendered and displayed on the display device 410.
[0312] Yet another example of the legacy graphic parameters LGP1 or LGP2 is a legacy upscale factor, which is defined by the legacy image resolution and the size of a standard television, such as a cathode ray tube, used in the legacy machine 900 (FIG. 9A). The legacy upscale factor scales the image of the virtual object or virtual background to fill the display pixels of the standard television after the legacy resolution is applied to the image. In this example, the updated graphic parameters UGP1 corresponding to the legacy graphic parameters LGP1, or the updated graphic parameters UGP2 corresponding to the legacy graphic parameters LGP2, are the updated upscale factors. For illustrative purposes, the updated upscale factors include HD resolution or 4K resolution. Also, for illustrative purposes, the updated upscale factor scales the image of the virtual object or virtual background to fill the display pixels of the HD display device or the 4K display device. The HD display device and the 4K display device are examples of the display device 410.
[0313] FIG. 21C-2 is a diagram illustrating an embodiment of a method performed by the GPU emulator 2122 (FIG. 21B-1) to change the resolution of a virtual object during a rendering operation. In the absence of the GPU emulator 2122, the position and orientation 2150 of the virtual object 2152 in the virtual scene of the legacy game N is resized to be larger and rendered by the GPU 924 using four-color pixels. For example, the virtual object 2152 is rendered by the legacy GPU 904 (FIG. 9A) that does not include the GPU emulator 2122.
[0314] When the GPU emulator 2122 is used, the position and orientation 2150 is resized and rendered as a virtual object 2154 in a virtual scene of the legacy game N. As an example, the virtual object 2154 has a higher resolution than the virtual object 2152. To explain, the virtual object 2154 has a larger number of image pixels than the virtual object 2152. To further explain, the GPU emulator 2122 effectively replaces one or more larger color pixels of the virtual object 2152 with one or more corresponding sets of smaller pixels to output the virtual object 2154. As another example, the virtual object 2154 has a more pronounced contour 2156 compared to the contour 2158 of the virtual object 2152. As yet another example, the virtual object 2154 appears cartoon-like compared to the virtual object 2152. To explain, the virtual object 2154 is rendered as a two-dimensional virtual object and the virtual object 2152 is rendered as a three-dimensional virtual object. As another example, contour 2156 has smoother edges and vertices compared to contour 2158. As another example, virtual object 2154 has higher contrast compared to virtual object 2152. As yet another example, virtual object 2154 has more variety of intensity levels compared to the intensity levels of virtual object 2152.
[0315] FIG. 21D is a diagram illustrating an embodiment of a method for generating an updated image frame 2160 based on updated graphics parameters. The CPU 922 (FIG. 21B-1) executes one or more of the basic blocks 1-n of the game code GCN to determine the positions and orientations of virtual objects and a virtual background in a virtual scene of the legacy game N. The image frame 2160 is an example of an image frame 2123 (FIG. 21B-1). The GPU 924 (FIG. 21B-1) executes one or more of the basic blocks 1-n of the game code GCN to generate a legacy image frame, such as the image frame 422 having the legacy graphics parameters. The legacy graphics parameters are applied by the GPU 924 to the positions and orientations of the virtual objects and a virtual background in the virtual scene of the legacy game N. Instead of providing the legacy image frame to the display device 410 to display an image of the virtual environment 452, the GPU 924 applies (e.g., executes) the GPU emulator 2122 to determine updated graphics parameters corresponding to the legacy graphics parameters. The GPU 924 generates an updated image frame 2160 based on the updated graphics parameters instead of the legacy graphics parameters, and sends the updated image frame to the display screen 410 to display an image of the legacy game N.
[0316] 22A is a diagram of an embodiment of a system 2200 to illustrate receiving updated audio parameters from User 1. The system 2200 includes a display device 410, a handheld controller 414, and a game console 402.
[0317] The game console 402 includes an emulation processor system 409 and an audio memory device 2206. The emulation processor system 409 includes an APU 931 that includes an audio emulator 2204. As an example, the audio emulator described herein is hardware or software, or a combination thereof. Illustratively, the audio emulator is a processor, or a PLD, or an ASIC, or a controller. As another illustration, the audio emulator is a computer program.
[0318] The audio memory device 2206 and the APU 931 are coupled to the bus 930. As an example, the APU 931 is implemented within the updated machine 920. Also as an example, the audio memory device 2206 is implemented within the memory system 926 (FIG. 9B).
[0319] After User 1 logs into User Account 1, during or before playing Legacy Game N, GPU 924 controls display device 410 to display a list of one or more updated audio parameters, such as UAP1 and UAP2. For example, during playing Legacy Game N, GPU 924 renders the updated audio parameters UAP1 and UAP2 on display device 410.
[0320] After the updated audio parameters are displayed on the display device 410, the user 1 operates one or more updated input devices of the handheld controller 414 to select one or more updated audio parameters, and the handheld controller 414 generates a user input 2208 based on the operation. The user input 2208 includes an indication of the selection of the one or more updated audio parameters. The user input 2208 is sent from the handheld controller 414 to the CPU 922. The CPU 922 obtains the one or more updated audio parameters from the user input 2208 and sends the one or more updated audio parameters to the APU emulator 2204 of the APU 931. The APU emulator 2204 emulates the one or more legacy audio parameters into the one or more updated audio parameters.
[0321] In one embodiment, any number of the updated audio parameters are displayed on the display device 410.
[0322] 22B-1 is a diagram of an embodiment of a system 2250 for illustrating emulation of one or more legacy audio parameters to one or more updated audio parameters. The system 2120 includes a handheld controller 414 and a game console 402.
[0323] When an indication of selection of one or more updated audio parameters is received via user input 2208 from handheld controller 414, CPU 922 provides the updated audio parameters to APU 931 via bus 930. APU 931 stores the updated audio parameters in audio memory device 2206. In response to user input 420, APU 931 executes one or more of basic blocks 1-n of updated code to determine that one or more legacy audio parameters, such as legacy audio parameter LAP1 and legacy audio parameter LAP2, are applied to the virtual object and virtual background in image frame 2123 of legacy game N. APU 931 further generates one or more legacy audio frames based on the one or more legacy audio parameters that are output in synchronization with image frame 2123 (FIG. 21B-1). It should be noted that user input 419, from which user input 420 is generated, is received before user input 2208 for applying one or more updated audio parameters is received.
[0324] When the APU 931 generates a legacy audio frame to be output together with the image frame 2123, the audio emulator 2204 applies the one or more updated audio parameters to the legacy audio frame instead of the one or more legacy audio parameters to generate the one or more audio frames 2252 that are updated audio frames. The application of the one or more updated audio parameters may be referred to as generating an audio frame to be output in the virtual environment of the legacy game N. For example, the audio emulator 2204 identifies an updated audio parameter UAP1 as corresponding to the legacy audio parameter LAP1 and an updated audio parameter UAP2 as corresponding to the legacy audio parameter LAP2 from the audio memory device 2206. In this example, the updated audio parameter UAP1 and the legacy audio parameter LAP1 are of the same type. To explain, the updated audio parameter UAP1 may be an updated sampling rate based on which a sound is output from a virtual image of a virtual object of the legacy game N, and the legacy audio parameter LAP1 may be a legacy sampling rate based on which a sound is output from the virtual image. The updated sampling rate is greater than the legacy sampling rate. For further explanation, the legacy sampling rate is 10,000 samples / second and the updated sampling rate is 96,000 samples / second. For another explanation, the updated audio parameters UAP2 allow sounds to be output from the virtual object of the legacy game N as stereo, mono, or surround sound. In this explanation, the legacy audio parameters LGP2 allow sounds to be output from the virtual object only as stereo. For yet another explanation, the updated audio parameters UAP1 have a larger sample space for audio sounds compared to the sample space for audio sounds of the legacy audio parameters LAP1.The larger sample space has more variety of audio sounds compared to the sample space of the audio sounds of the legacy audio parameters LAP1. As another explanation, the updated audio parameters UAP2 have different reverb compared to the legacy audio parameters LAP2. To further explain, the updated audio parameters UAP2 exhibit more or less echo compared to the amount of echo output based on the legacy audio parameters LAP2.
[0325] As another example, both audio parameters LAP1 and UAP1 include the same identifier (e.g., an alphanumeric code) of a type of audio parameter. In this example, the audio emulator 2204 determines that the updated audio parameter UAP1 corresponds to the legacy audio parameter LAP1 in response to determining that both audio parameters UAP1 and LAP1 include the same identifier. The GPU 924 transmits the image frame 2123 and the APU 931 transmits the audio frame 2252 to the display device 410 to display an image of the virtual environment (e.g., virtual environment 452) on the display device 410 in accordance with the image frame 2123 and in accordance with the sound output based on the audio frame 2252. For example, the GPU of the display device 410 controls the lights of the display device 410 to display the image in accordance with the image frame 2123, and the APU of the display device 410 controls the speakers of the display device 410 to output the sound in accordance with the audio frame 2252. The image of the virtual environment is displayed on the display device 410 to display the play of the legacy game N.
[0326] In one embodiment, APU 931, which does not include audio emulator 2204, is a component of legacy machine 900 (FIG. 9A). For example, APU 931 is coupled to memory device 906. As another example, the same audio processing hardware is used in both legacy machine 900 and updated machine 920 (FIG. 9B). As yet another example, APU 931 has the same hardware and software specifications as legacy APU 910.
[0327] In one embodiment, the CPU 922 , or the GPU 924 , or the APU 931 , or a combination of two or more thereof, executes the basic block compiler 104 .
[0328] In one embodiment, the CPU 922 , or the GPU 924 , or the APU 931 , or a combination of two or more thereof, executes the block dispatcher 302 .
[0329] In one embodiment, the updated audio parameters do not correspond to the legacy audio parameters. For example, there is no legacy audio parameter corresponding to the updated audio parameter UAP1. Instead of applying any of the legacy audio parameters, the APU 931 determines the updated audio parameter UAP1 to generate the audio frame 2252 of the legacy game N.
[0330] In one embodiment, the legacy APU 910 (FIG. 9A) executes one or more of the emulated PU code instructions 1-M. Executing the emulated PU code instructions 1-M may not generate updated audio parameters. Rather, legacy audio parameters are generated.
[0331] In one embodiment, when the emulation processor system 409 is implemented in the server system 404 (FIG. 4B), a client device such as a display device 410 or HMD is used. The emulation processor system 409 is coupled to the client device via a computer network 408. The NIC of the server system 404 applies a network communication protocol to the updated image frame 2123 to generate one or more data packets and transmits the data packets to the client device via the computer network 408. The NIC of the client device applies a network communication protocol to the data packets to extract the updated image frame 2123 from the data packets and provides the updated image frame 2123 to a display screen of the client device for display of the virtual environment of the legacy game N.
[0332] FIG. 22B-2 is a diagram of an embodiment of a system 2264 for illustrating generation of an updated audio frame 2252 from a legacy audio frame 2266 output from a block dispatcher 302. The system 2264 includes a block dispatcher 302 and an APU emulator 2204. The block dispatcher 302 is coupled to the APU emulator 2204. The block dispatcher 302 executes one or more of basic blocks 1-n to generate a legacy audio frame 2266 including legacy audio parameters. The APU emulator 2204 receives the legacy audio frame 2266, analyzes the legacy audio frame 2266 to identify the legacy audio parameters, converts the legacy audio parameters to updated audio parameters, and outputs the updated audio frame 2252.
[0333] FIG. 22C-1 is a diagram for illustrating an embodiment of a method for generating an updated audio frame 2260 based on a legacy audio parameter. The APU 931 executes the block dispatcher 302 (FIG. 3) to determine the legacy audio parameter and generate a legacy audio frame based on the legacy audio parameter. The APU emulator 2204 (FIG. 22B-1) extracts the legacy audio parameter from the legacy audio frame, converts the legacy audio parameter of the legacy audio frame into the updated audio parameter, and so on to emulate. The APU 931 generates an updated audio frame 2260 having the updated audio parameter. The updated audio frame 2260 is to be output together with an image frame of the virtual scene of the legacy game N. For example, the updated audio frame 2260 is sent from the APU 931, and the updated image frame 2160 (FIG. 21D) is sent from the GPU 924 to the display device 410 to display an image of the virtual scene of the legacy game N with the sound. The updated audio frame 2260 is an example of one of the updated audio frames 2252 (FIG. 22B-1). In a similar manner, additional updated audio parameters are determined based on the additional legacy audio parameters, and the additional updated audio parameters are integrated into the updated audio frame 2260.
[0334] FIG. 22C-2 is a diagram of an embodiment of a system 2270 for illustrating processing of audio frames of a legacy game N by a client device 2272 when the emulation processor system 409 is implemented within a game console 402 (FIG. 4A). An example of a client device 2272 is a display device 410 or an HMD. The client device 2272 includes a digital-to-analog converter (DAC) 2274, a filter 2276, an amplifier 2278, and a speaker 2280. The DAC 2274 is coupled to the filter 2276, which is coupled to the amplifier 2278. The amplifier 2278 is coupled to the speaker 2280. The DAC 2274 is also coupled to the audio emulator 2204.
[0335] The DAC 2274 receives the updated audio frame 2260 from the audio emulator 2204. Upon receiving the updated audio frame 2260, the digital audio data of the updated audio frame 2260 is converted from digital to analog format by the DAC 2274 to output analog audio data. The filter 2276 filters the analog audio data, such as by reducing or removing noise from the analog audio data, to output noise-reduced audio data. The amplifier 2278 amplifies, such as by modifying the amplitude of the noise-reduced audio data, to output amplified audio data. The speaker 2280 converts the electrical energy of the amplified audio data into sound. In a similar manner, the updated audio frame 2252 (FIG. 22B-2) is processed by the client device 2272.
[0336] In one embodiment, the client device 2272 includes multiple speakers coupled to an amplifier 2278 .
[0337] In one embodiment, when the emulation processor system 409 is implemented in the server system 404 (FIG. 4B), another client device, such as a combination of a game console 402 and a display device 410, or a combination of a game console 402 and an HMD, or a client device 2272 is used. In this embodiment, the NIC of the server system 404 receives the updated audio frame 2260 from the audio emulator 2204, applies a network communication protocol to the updated audio frame 2260 to generate one or more data packets, and transmits the data packets to the other client device via the computer network 408. The NIC of the server system 404 is coupled to the APU 931 via the bus 930. The NIC of the other client device receives the data packets. As an example, the NIC of the other client device is located in the HMD or the display device 410 or the game console 402. The NIC of the other client device applies a network communication protocol to the data packets to obtain the updated audio frame 2260 from the data packets, and transmits the updated audio frame 2260 to the DAC 2274. The NICs of the other client devices are coupled to the DAC2274.
[0338] 22C-3 is a diagram of an embodiment of a system 2280 illustrating synchronization of displaying images based on updated image frames 2123 and outputting sounds based on updated audio frames 2252 for a plurality of virtual scenes of a legacy game N. The system 2280 includes an APU 931, a GPU 924, and a client device 2272. The client device 2272 includes a video audio (VA) synchronizer 2282, a GPU 2284, and a display screen 2286. The VA synchronizer 2282 is coupled to the DAC 2274 and to the GPU 2284, which is coupled to the display screen 2286.
[0339] When the emulation processor system 409 is implemented within a game console 402, the APU 931 and the CPU 924 are coupled to a communication device of the game console 402. The communication device receives the audio frames 2252 and the image frames 2123, applies a close range communication protocol to the frames 2252 and 2123 to generate one or more data packets, and transmits the one or more data packets to the display device 410. The display device 410 includes a communication device that receives the one or more data packets and applies a close range communication protocol to the data packets to extract the frames 2252 and 2123. The communication device of the display device 410 transmits the frames 2252 and 2123 to the VA synchronizer 2282.
[0340] The VA synchronizer 2282 synchronizes the display order of the image frame 2123 with the display order of the audio frame 2252. For example, the VA synchronizer 2282 determines that the updated audio frame 2260 and the updated image frame 2160 are output simultaneously based on a first time identifier embedded in the updated audio frame 2260 and a second time identifier embedded in the updated image frame 2160. Illustratively, the VA synchronizer 2282 determines that the updated audio frame 2260 and the updated image frame 2160 are output simultaneously by determining that the first time identifier matches the second time identifier. Also, in this illustration, each of the first and second identifiers is an alphanumeric code. In this illustration, the first time identifier is generated by the APU 931, and the second time identifier is generated by the GPU 924. Also, in this illustration, both the APU 931 and the GPU 924 generate the first and second time identifiers based on the same clock signal. To further explain, the clock signal may be generated by the APU 931 and sent to the GPU 924, or may be generated by the GPU 924 and sent to the APU 931, or may be generated by a clock source and sent to the APU 931 and the GPU 924. One example of a clock source is the Internet.
[0341] After synchronizing the image frame 2123 with the audio frame 2252, the VA synchronizer 2282 outputs the image frame 2123 to the GPU 2284 and outputs the audio frame 2252 to the DAC 2274. For example, the VA synchronizer 2282 provides the updated image frame 2160 to the GPU 2284 while providing the updated audio frame 2260 to the DAC 2274. The GPU 2284 controls the display pixels of the display screen 2286 to display images of the multiple virtual scenes of the legacy game N according to the image frame 2123. Also, the audio frame 2252 is processed by the DAC 2274, the filter 2276, the amplifier 2278, and the speaker 2280 ( FIG. 22C-2 ) in the same manner as the updated audio frame 2260 is processed, in order to output the sound in synchronization with the display of the images of the multiple virtual scenes of the legacy game N.
[0342] In one embodiment, when the emulation processor system 409 is implemented within a server system 404 (FIG. 4B), the audio frame 2252 is sent from the APU 931 to the NIC of the server system 404. The APU 931 and the GPU 924 are coupled to the NIC of the server system 404. Also, the image frame 2123 is sent from the GPU 924 to the NIC of the server system 404. The NIC of the server system 404 applies a network communication protocol to the frames 2123 and 2252 to generate one or more packets and transmits the packets over the computer network 408 (FIG. 4B) to the other client device described above with reference to FIG. 22C-2. The NIC of the other client device applies a network communication protocol to the one or more packets to extract the frames 2252 and 2123 from the packets. The NIC of the other client device sends frames 2252 and 2123 to VA synchronizer 2282 for processing in the same manner as described above with reference to FIG. 22C-3.
[0343] FIG. 22D is a diagram of an embodiment of a display device 410 for displaying a legacy mode and a normal mode. User 1 operates the handheld controller 414 to generate a user input. Upon receiving a user input before or during playing the legacy game N, the GPU 924 generates an image frame and transmits the image frame to the display device 410. Based on the image frame, the display device 410 displays an image 2290 including an item 2292 for receiving a selection of a legacy mode and an item 2294 for receiving a selection of a normal mode. When User 1 operates the handheld controller 414 to select the item 2292, the GPU 924 generates an image frame 2123 (FIG. 21B-1) based on the updated graphic parameters. On the other hand, when User 1 operates the handheld controller 414 to select the item 2294, the GPU 924 generates an image frame 422 (FIG. 4B) based on the legacy graphic parameters.
[0344] 23 is an illustration of an embodiment of a display device 410 for displaying a virtual environment 2300 in which a menu 2302 is displayed. Virtual environment 2300 is the same as virtual environment 452 (FIG. 4B), except that virtual environment 2300 includes a menu 2302. Menu 2302 is displayed by the GPU of display device 410 when user input is received from handheld controller 414 (FIG. 4B) during play of legacy game N.
[0345] The menu 2302 includes a number of updated graphic parameters, such as an X position, a Y position, an upscale factor, and an aspect ratio. User 1 operates the handheld controller 414 to select one or more of the updated graphic parameters displayed in the menu 2302. For example, user 1 selects an X position to change the position of the virtual environment 452 relative to the horizontal level of the display screen of the display device 410. As another example, user 1 selects a Y position to change the position of the virtual environment 452 relative to the vertical level of the display screen of the display device 410. The horizontal level of the display screen is perpendicular to the vertical level. To explain, the horizontal level is along the x-axis and the vertical level is along the y-axis. As another example, user 1 selects an upscale factor to change the updated upscale factor. As yet another example, user 1 selects an aspect ratio to change the ratio between the height of the virtual environment 452 and the width of the virtual environment 452 on the display screen of the display device 410. The height is measured along the vertical level of the display screen and the width is measured along the horizontal level of the display screen.
[0346] In one embodiment, menu 2302 includes a sliding scale for each of the updated graphical parameters displayed in menu 2302. User 1 uses handheld controller 414 to slide a slider on a corresponding sliding scale of one of the updated graphical parameters of menu 2302 to select a level or amount of the updated graphical parameter.
[0347] 24A is an illustration of an embodiment of a display device 410 for displaying a virtual environment 2400 in which a menu 2402 is displayed. Virtual environment 2400 is the same as virtual environment 452 (FIG. 4B), except that virtual environment 2400 includes a menu 2402. Menu 2402 is displayed by the GPU of display device 410 when user input is received from handheld controller 414 (FIG. 4B) during play of legacy game N.
[0348] The menu 2402 includes a number of updated graphic parameters, such as enable scanline, scanline depth, scanline width, scanline brightness, dither depth, etc. User 1 operates the handheld controller 414 to select one or more of the updated graphic parameters displayed in the menu 2402. For example, User 1 selects "enable scanline" to overlay scanlines on top of the virtual environment 452. Upon receiving an indication of the selection of the "enable scanline" option, the GPU 924 applies the GPU emulator 2122 to overlay the scanlines. As another example, User 1 selects "scanline depth" to change the depth of each "scanline overlay" or overlay on the virtual environment 452. The depth of the scanline is along the horizontal level of the display screen of the display device 410. To explain, the depth of the scanline is along the z-axis, which is perpendicular to both the x-axis and the y-axis. As another example, user 1 selects scanline width to change the width of each "scanline overlay" or overlay on the virtual environment 452. The width of the scanline is along the vertical level of the display screen of the display device 410. As yet another example, user 1 selects "scanline brightness" to change the amount of brightness of the scanline overlay or overlay on the virtual environment 452. Upon receiving an instruction to change the brightness of the scanline, the GPU 924 applies the GPU emulator 2122 to change the level of brightness. As yet another example, user 1 selects a dither depth to add noise to reduce quantization error in the scanline generation. The quantization error is reduced by randomizing the quantization. Upon receiving an instruction to change the dither depth from the handheld controller 414, the GPU 924 applies the GPU emulator 2122 to change the level of dither depth to generate the scanline. To illustrate, the GPU emulator 2122 samples data to generate the scanline at a first sampling rate that is less than a second sampling rate.The second sampling rate is also for sampling data to generate scanlines. When the first sampling rate is applied, a first amount of sample points output from the sampling is lower compared to a second amount of sample points output from the second sampling rate. Explained, the first sampling rate is lower than the Nyquist sampling rate. The GPU emulator 2122 applies a certain amount of dither depth to the first amount of sampling points to increase noise, thereby making the scanlines more noticeable. The scanlines are more noticeable than when the dither depth is not applied.
[0349] In one embodiment, menu 2402 includes a sliding scale for each of the updated graphical parameters displayed in menu 2402. User 1 uses handheld controller 414 to slide a slider on a corresponding sliding scale of one of the updated graphical parameters of menu 2402 to select a level or amount of the updated graphical parameter.
[0350] 24B is a diagram of an embodiment of a display device 410 for displaying a virtual environment 2410 having multiple scan lines 2412, such as scan line 2412A, scan line 2412B, and scan line 2412C. The virtual environment 2410 is the same as the virtual environment 452 (FIG. 4B), except that the virtual environment 2410 includes scan lines 2412. Each of the scan lines 2412 extends from a left edge 2414 of the image of the virtual environment 452 to a right edge 2416 of the image of the virtual environment 452. For example, each scan line 2412 is horizontal. The left edge 2414 is perpendicular to a top edge 2418 of the image of the virtual environment 452 and to a bottom edge 2420 of the image of the virtual environment 452.
[0351] In one embodiment, instead of scanline 2412, a vertically oriented scanline is generated and overlaid on the virtual environment 452. For example, the vertically oriented scanline extends from the top edge 2418 to the bottom edge 2420. The vertically oriented scanline is generated by GPU 924 for display on display device 410.
[0352] 25A is an illustration of an embodiment of a display device 410 for displaying a virtual environment 2500 in which a menu 2502 is displayed. Virtual environment 2500 is the same as virtual environment 452 (FIG. 4B), except that virtual environment 2500 includes a menu 2502. Menu 2502 is displayed by the GPU of display device 410 when user input is received from handheld controller 414 (FIG. 4B) during play of legacy game N.
[0353] The menu 2502 includes a number of updated graphic parameters, such as brightness, contrast, gamma, hue, and saturation. User 1 manipulates the handheld controller 414 to select one or more of the updated graphic parameters displayed within the menu 2402. For example, user 1 selects a brightness to modify, such as to increase or decrease the brightness of an image pixel of the virtual environment 452. Upon receiving an indication of the selection of "brightness," the GPU 924 applies the GPU emulator 2122 to modify the brightness. As another example, user 1 selects "contrast" to modify the contrast of a virtual object of the virtual environment 452 compared to a virtual background of the virtual environment 452. As another example, user 1 selects gamma to provide an indication of how quickly colors change in the image of the virtual environment 452. Upon receiving an indication of the selection of gamma, the GPU 924 applies the GPU emulator 2122 to modify colors at a higher or lower frequency. As yet another example, user 1 selects a hue to modify the colors filling the virtual environment 452. Upon receiving the indication to change the scanline intensity, GPU 924 applies the GPU emulator 2122 to the amount of color used to generate the image pixels of virtual environment 452. Illustratively, GPU emulator 2122 changes a first hue that includes red, blue, green, and yellow to a second hue that includes red, blue, green, orange, and yellow. As yet another example, user 1 selects saturation to change the amount of intensity or purity of the color of each image pixel of virtual environment 452. When the saturation of a color is increased, the color appears purer. When the saturation is decreased, the color appears to be replaced with gray.
[0354] In one embodiment, menu 2502 includes a sliding scale for each of the updated graphical parameters displayed in menu 2502. User 1 uses handheld controller 414 to slide a slider on a corresponding sliding scale of one of the updated graphical parameters of menu 2502 to select a level or amount of the updated graphical parameter.
[0355] 25B is an illustration of an embodiment of display device 410 for displaying virtual environment 2510 in which menu 2512 is displayed. Virtual environment 2510 is the same as virtual environment 452 (FIG. 4B), except that virtual environment 2510 includes menu 2512. Menu 2512 is displayed by the GPU of display device 410 when user input is received from handheld controller 414 (FIG. 4B) during play of legacy game N.
[0356] The menu 2512 includes a number of updated graphic parameters, such as a shading level applied by a cel shader, a saturation applied by a cel shader, a brightness applied by a cel shader, a contour applied by a cel shader, an intensity applied by a cel shader, and a smoothness applied by a cel shader. The cel shader is an example of a GPU emulator 2122 (FIG. B). User 1 operates the handheld controller 414 to select one or more of the updated graphic parameters displayed in the menu 2512. For example, user 1 selects a shading level to change, such as increasing or decreasing the amount of shading of image pixels of a contour, such as a perimeter or boundary of a virtual object in the virtual environment 452. Explaining, the virtual object is a virtual character controlled by user 1 via the handheld controller 414. Explaining further, the virtual object is a fighter jet. Shading changes the color or intensity of the color of the contour. As an example, the contour of a virtual object in a virtual scene distinguishes the virtual object from other virtual objects and the virtual background in the virtual scene. Upon receiving an indication of a selection of the shading level option, GPU 924 applies GPU emulator 2122 to modify the shading level of the contour of the virtual object. As another example, User 1 selects saturation to modify (e.g., increase or decrease) the amount of intensity or purity of color of each image pixel of the contour of a virtual object in virtual environment 452. Upon receiving an indication of a selection of the saturation option, GPU 924 applies GPU emulator 2122 to modify the amount of intensity or purity of color of each image pixel of the contour of the virtual object.
[0357] As another example, User 1 selects brightness to modify (e.g., increase or decrease) the brightness of image pixels of the contours of virtual objects in virtual environment 452. Upon receiving an indication of the selection of the saturation option, GPU 924 applies GPU emulator 2122 to modify the brightness of image pixels of the contours of virtual objects.
[0358] As yet another example, user 1 selects contour to indicate whether to generate a contour of a virtual object in the virtual environment. Upon receiving the contour instruction, GPU 924 applies GPU emulator 2122 to include a contour of the virtual object. The contour forms a perimeter of the virtual object and contrasts the boundary of the virtual object to distinguish the virtual object from the virtual background of the virtual object and from other virtual objects in virtual environment 452. As yet another example, user 1 selects intensity to change the amount of transparency of the contour of the virtual object in virtual environment 452. When the transparency of the contour decreases, the contour appears opaque, and when the transparency increases, the contour appears transparent. In this manner, the contour switches from opaque to semi-transparent to transparent or vice versa. Upon receiving the contour instruction, GPU 924 applies GPU emulator 2122 to change the amount of transparency of the contour of the virtual object in virtual environment 452.
[0359] As another example, user 1 selects smooth to modify (e.g., increase or decrease) the amount of smoothness of image pixels of the contour of a virtual object in virtual environment 452. Explaining, smoothness is increased by increasing the number of image pixels of the contour and is decreased by decreasing the number of image pixels of the contour. Upon receiving an indication of the selection of the smooth option, GPU 924 applies GPU emulator 2122 to modify the smoothness of image pixels of the contour of the virtual object.
[0360] In one embodiment, menu 2512 includes a sliding scale for each of the updated graphical parameters displayed in menu 2512. User 1 uses handheld controller 414 to slide a slider on a corresponding sliding scale of one of the updated graphical parameters of menu 2512 to select a level or amount of the updated graphical parameter.
[0361] 25C is an illustration of an embodiment of display device 410 for displaying virtual environment 2520 in which menu 2522 and menu 2524 are displayed. Virtual environment 2520 is the same as virtual environment 452 (FIG. 4B), except that virtual environment 2520 includes menus 2522 and 2524. Menus 2522 and 2524 are displayed by the GPU of display device 410 when user input is received from handheld controller 414 (FIG. 4B) during play of legacy game N.
[0362] The menu 2522 includes a number of updated graphic parameters, such as a type 1 video filter and a type 2 video filter. The user 1 operates the handheld controller 414 to select one or more of the updated graphic parameters displayed in the menu 2522. For example, the user 1 selects a type 1 filter or a type 2 filter to be applied to image pixels of the virtual environment 452. A description of the type 1 filter is a Mitchell Netravali® filter, and a description of the type 2 filter is a Butterworth® filter. Other descriptions of the filter types of the video filters include Chebyshev® and Bessel®. Upon receiving an indication of the selection of the type 1 or type 2 video filter, the GPU 924 applies the GPU emulator 2122 to apply the type of video filter to generate image pixels of the virtual object, or image pixels of the virtual background, or a combination thereof, of the legacy game N.
[0363] Similarly, the menu 2524 includes a number of updated graphic parameters, such as an anti-aliasing filter type 1 and an anti-aliasing filter type 2. The user 1 operates the handheld controller 414 to select one or more of the updated graphic parameters displayed in the menu 2524. For example, the user 1 selects a type 1 filter or a type 2 filter to be applied to image pixels of the virtual environment 452. The description of the type 1 filter is an FXAA® filter, and the description of the type 2 filter is a Line Thickness® filter. Upon receiving an indication of the selection of the type 1 or type 2 anti-aliasing filter, the GPU 924 applies the GPU emulator 2122 to apply the type of anti-aliasing filter to generate image pixels of a virtual object, or image pixels of a virtual background, or a combination thereof, of the legacy game N.
[0364] In one embodiment, each menu 2522 and 2524 includes a sliding scale for each of the updated graphical parameters displayed in the menu. User 1 uses handheld controller 414 to slide a slider on the corresponding sliding scale of one of the updated graphical parameters of menu 2522 or 2524 to select the level or amount of the updated graphical parameter.
[0365] 25D is an illustration of an embodiment of a display device 410 for displaying a virtual environment 2530 in which a crosshair 2532 is displayed. The virtual environment 2532 is the same as virtual environment 452 (FIG. 4B), except that the virtual environment 2532 includes a crosshair 2532. The crosshair 2532 is displayed by the GPU of the display device 410 when user input is received from the handheld controller 414 (FIG. 4B) during play of the legacy game N. For example, the user input indicates a size of the crosshair 2532.
[0366] User 1 pauses legacy game N using handheld controller 414. Once legacy game N is paused, user 1 manipulates handheld controller 414 to generate user input including an instruction that the size of crosshair 2532 should be changed. The size of the crosshair is an example of an updated graphics parameter. Upon receiving the instruction, GPU 924 applies GPU emulator 2122 (FIG. 21B-1) to change the size of crosshair 2532 to match the size received via user input.
[0367] FIG. 26 is a flow diagram conceptually illustrating various operations performed to stream a cloud video game to a client device, according to an implementation of the present disclosure. Examples of client devices include a combination handheld controller and display device, a smartphone, a combination handheld controller, display device and game console, a combination handheld controller and HMD, a combination handheld controller, game console and HMD, a tablet, and a computer. A game server 2602 executes a game program, such as a video game or game code GCN, and generates raw (uncompressed) video 2604 and audio 2606. The virtual environment 452 (FIG. 4A) and the audio output in the representation of the virtual environment 452 are examples of the video 2604 and audio 2606. The game server 2602 is an example of a server system 404 (FIG. 4A). The video 2604 and audio 2606 are captured and encoded for streaming, as indicated by reference number 2608 in the illustrated figure. The encoding provides compression of the video and audio streams to reduce bandwidth usage and optimize the gaming experience. Examples of encoding formats include H.265 / MPEG-H, H.264 / MPEG-4, H.263 / MPEG-4, H.262 / MPEG-2, WMV, VP6 / 7 / 8 / 9, etc.
[0368] The encoded audio 2610 and encoded video 2612 are further packetized into network packets, as indicated by reference numeral 2614, for purposes of transmission over computer network 2620, which is an example of computer network 408 (FIG. 4A). In some embodiments, the network packet encoding process also employs a data encryption process, thereby providing enhanced data security. In the illustrated implementation, audio packets 2616 and video packets 2618 are generated for transfer over computer network 2620.
[0369] The game server 2602 further generates haptic feedback data 2622, which is also packetized into network packets for network transmission. In the illustrated implementation, haptic feedback packets 2624 are generated for transfer over the computer network 2620.
[0370] The aforementioned operations of generating the raw video and audio and haptic feedback data are performed on the game server 2602 of the data center, and the operations of encoding the video and audio and packetizing the encoded audio / video and haptic feedback data for transmission are performed by a streaming engine of the data center. As shown, the audio, video, and haptic feedback packets are transmitted over a computer network 2620. As shown at reference numeral 2626, the audio packets 2616, the video packets 2618, and the haptic feedback packets 2624 are decomposed, e.g., parsed, by the client device to extract the encoded audio 2628, the encoded video 2630, and the haptic feedback data 2622 from the network packets at the client device. If the data is encrypted, the data is also encrypted. The encoded audio 2628 and the encoded video 2630 are then decoded by the client device, as shown at reference numeral 2634, to generate client-side raw audio and video data for rendering on the display device 2640 of the client device. The haptic feedback data 2622 can be processed by a processor of the client device to generate haptic feedback effects in the controller device 2642 or other interface device, such as an HMD, through which the haptic effects are rendered. The controller device 2642 is an example of a handheld controller, such as the client device's handheld controller 414 (FIG. 4A) or 1212 (FIG. 12). An example of a haptic effect includes a vibration or rumble of the controller device 2642.
[0371] It will be appreciated that the video game responds to player or user input, so a similar procedural flow as described above for the transfer and processing of player input is performed, but in the reverse direction from the client device to the server. As shown, the controller device 2642 or another updated controller, such as an HMD, or a combination thereof, generates input data 2648. The input data 2648 is packetized at the client device for transfer as user input over the computer network 2620 to the data center. The input data packets 2646 are depacketized and reassembled by the game server 2602 to define the input data 2648 at the data center. The input data 2648 is provided to the game server 2602, which processes the input data 2648 to generate the game state of the legacy game N.
[0372] During the transfer of the audio packets 2616, video packets 2618, and haptic feedback packets 2624 over the computer network 2620, in some embodiments, the transmission of data over the computer network 2620 is monitored to ensure quality of service. For example, network conditions of the computer network 2620, including both upstream and downstream network bandwidth, are monitored as indicated by reference numeral 2650, and the game streaming is adjusted in response to changes in available bandwidth. That is, the encoding and decoding of network packets is controlled based on the current network conditions, as indicated by reference numeral 2652.
[0373] FIG. 27 is a block diagram of an embodiment of a game console 2700, which is compatible with interfacing with a display device of a client device and can communicate with a game hosting system, such as server system 404 (FIG. 4A), via computer network 2620 (FIG. 26). Game console 2700 is an example of game console 402 (FIG. 4A) and game console 1202 (FIG. 12). Game console 2700 is located in a data center or where a player, such as user 1 or 2, is located. In some embodiments, game console 2700 is used to execute game code GCN and display legacy game N on HMD 2705. Game console 2700 is provided with various peripherals that can be connected to game console 2700. The game console 2700 includes a Cell processor 2728, a dynamic random access memory (XDRAM) unit 2726, a reality synthesiser graphics processor unit 2730 with dedicated video random access memory (VRAM) unit 2732, and an input / output (I / O) bridge 2734. The game console 2700 also includes a Blu Ray® disc read only memory (BD-ROM) optical disc reader 2740 for reading from disc 2740a, and a removable slot-in hard disk drive (HDD) 2736 accessible via the I / O bridge 2734. Optionally, the game console 2700 also includes a memory card reader 2738 for reading CompactFlash® memory cards, Memory Stick® memory cards, etc., which are also accessible via the I / O bridge 2734. The I / O bridge 2734 also connects to a USB 2.0 port 2724, a Gigabit Ethernet port 2722, an IEEE 802.11b / g wireless network (Wi-Fi™) port 2720, and a Bluetooth® wireless link port 2718 that can support Bluetooth® connections.
[0374] In operation, the I / O bridge 2734 handles all wireless, USB, and Ethernet data, including data from the game controller and from the HMD 2705. For example, when a player is playing a legacy game N generated by execution of a portion of game code, such as game code GCN, the I / O bridge 2734 receives input data or input signals described herein from the game controller 2642 and / or from the HMD 2705 via a Bluetooth link and directs the input data to the Cell processor 2728, which updates the current state of the legacy game N accordingly. By way of example, a camera in the HMD 2705 captures the player's gestures and generates images representative of the gestures.
[0375] The wireless, USB, and Ethernet ports also provide connections for other peripherals, such as game controllers 2642 and 2703, HMD 2705, and other peripherals, such as, for example, a remote control 2704, a keyboard 2706, a mouse 2708, a portable entertainment device 2710, such as, for example, a Sony Playstation Portable entertainment device, etc., a video camera, such as, for example, an EyeToy video camera 2712, etc., a microphone headset 2714, and a microphone 2715. The portable entertainment device 2710 is an example of a game controller. In some embodiments, such peripherals are wirelessly connected to the game console 2700, for example, the portable entertainment device 2710 communicates via a Wi-Fi ad-hoc connection, while the microphone headset 2714 communicates via a Bluetooth link.
[0376] The provision of these interfaces means that the game console 2700 may also be compatible with other peripheral devices such as digital video recorders (DVRs), set-top boxes, digital cameras, portable media players, voice over Internet Protocol (IP) phones, mobile phones, printers, scanners, etc.
[0377] Additionally, a legacy memory card reader 2716 is connected to the game console 2700 via a USB port 2724, allowing for the reading of memory cards 2748 of the type used by the game console 2700. The game controllers 2642 and 2703, and the HMD 2705 are operable to communicate wirelessly with the game console 2700 via a Bluetooth® link 2718, or are operable to be connected to the USB port 2724, thereby also receiving power, thereby charging the batteries of the game controllers 2642 and 2703, and the HMD 2705. In some embodiments, each of the game controllers 2642 and 2703, as well as the HMD 2705, includes memory, a processor, a memory card reader, fixed memory such as, for example, flash memory, lights such as, for example, an illuminated spherical section, light emitting diodes (LEDs), or infrared lights, etc., a microphone and speaker for ultrasonic communication, an acoustic chamber, a digital camera, an internal clock, a recognizable shape such as, for example, a spherical section facing the game console 2700, and wireless devices using protocols such as, for example, Bluetooth®, Wi-Fi®, etc.
[0378] Game controller 2642 is a controller designed to be used with both hands by a player, such as user 1 or 2, and game controller 2703 is a one-handed controller with an attachment. HMD 2705 is designed to fit over the head or in front of the player's eyes. In addition to one or more analog joysticks and traditional control buttons, each game controller 2642 and 2703 is susceptible to three-dimensional location determination. Similarly, HMD 2705 is susceptible to three-dimensional location determination. As a result, in some embodiments, gestures and actions by a player using game controllers 2642 and 2703 and HMD 2705 are translated as inputs to the game in addition to or instead of traditional button or joystick commands. Optionally, other wireless-enabled peripheral devices, such as, for example, a Playstation® portable device, are used as controllers. In the case of a Playstation® portable device, additional game or control information, such as control instructions or survival count, is provided on the display screen of the device. In some embodiments, other alternative or supplemental control devices are used, such as, for example, a dance mat (not shown), a light gun (not shown), a steering wheel and pedals (not shown), a custom controller, etc. Examples of custom controllers include a single or several large buttons (also not shown) for fast response trivia games.
[0379] The remote control 2704 is also operable to communicate wirelessly with the game console 2700 via a Bluetooth® link 2718. The remote control 2704 includes controls suitable for operating a Blu Ray® disc BD-ROM reader 2740 and for navigating the disc contents.
[0380] The Blu Ray® disc BD-ROM reader 2740 is operable to read CD-ROMs compatible with the game console 2700, in addition to conventional pre-recorded and recordable CDs, and so-called Super Audio CDs. The Blu Ray® disc BD-ROM reader 2740 is also operable to read Digital Video Disc-ROMs (DVD-ROMs) compatible with the game console 2700, in addition to conventional pre-recorded and recordable DVDs. The Blu Ray® disc BD-ROM reader 2740 is further operable to read BD-ROMs compatible with the game console 2700, as well as conventional pre-recorded and recordable BluRay® discs.
[0381] The game console 2700 is operable to provide audio and video generated or decoded through the Reality Synthesizer graphics unit 2730 to a display and sound output device 2742, such as a monitor or television, having a display screen 2744 and one or more loudspeakers 2746, via an audio connector 2750 and a video connector 2752, or to a display device of the HMD 2705 via a Bluetooth wireless link port 2718. The audio connector 2750, in various embodiments, includes conventional analog and digital outputs, while the video connector 2752 variously includes component video, S-video, composite video, and one or more high definition multimedia interface (HDMI) outputs. The resulting video output may be in a format such as Phase Alternating Line (PAL) or National Television System Committee (NTSC), or in high definition at 2720p, 1080i, or 1080p. Audio processing, e.g., generation, decoding, etc., is performed by the cell processor 2708. The game console 2700 operating system supports Dolby® 5.1 surround sound, Dolby® Theater Surround (DTS), and decoding of 7.1 surround sound from Blu-Ray® discs. The display and sound output device 2742 is an example of the display device 410 (FIG. 4A).
[0382] In some embodiments, a video camera, such as video camera 2712, includes a single charge-coupled device (CCD), an LED indicator, and a hardware-based real-time data compression and encoding device such that the compressed video data is transmitted in a suitable format, such as an intra-image based Moving Picture Experts Group (MPEG) standard, for decoding by the game console 2700. The LED indicator of the video camera 2712 is positioned to illuminate, for example, in adverse lighting conditions, in response to suitable control data from the game console 2700. Some embodiments of the video camera 2712 connect to the game console 2700 via a USB, Bluetooth, or Wi-Fi communication port. Various embodiments of the video camera include one or more associated microphones and can also transmit audio data. In some embodiments of the video camera, the CCD has a resolution suitable for high-definition video capture. In use, images captured by the video camera are incorporated into the game or interpreted as game control inputs. In another embodiment, the video camera is an infrared camera suitable for detecting infrared light.
[0383] In various embodiments, appropriate software, such as device drivers, are provided to enable successful data communication through one of the communication ports of the game console 2700 with a peripheral device, such as a video camera or a remote control.
[0384] In some embodiments, the aforementioned system devices including the game console 2700, the game controller 2642 (FIG. 26) or 2703, and the HMD 2705 enable the HMD 2705 to display and capture video of an interactive session of the legacy game N. The system device initiates an interactive session of the legacy game N, an interactive session that defines interactivity between player 1 and other players, and an interactive session of the legacy game N. The system device further determines an initial position and orientation of the game controller 2642 (FIG. 26) or 2603 operated by the player, and / or the HMD 2705. The game console 2700 determines a current state of the game based on the interactivity between the player and the legacy game N. The system device tracks the position and orientation of the game controller 2642, 2703, and / or the HMD 2705 during the player's interactive session with the legacy game N. The system device generates a spectator video stream for the interactive session based on the current state of the legacy game N and the tracked positions and orientations of the game controllers 2642, 2703 and / or the HMD 2705. In some embodiments, the HHC renders the spectator video stream on a display screen of the HHC. In various embodiments, the HMD 2705 renders the spectator video stream on a display screen of the HMD 2705.
[0385] Referring to FIG. 28, a diagram showing components of an HMD 2802 is shown. The HMD 2802 is an example of the HMD 2705 (FIG. 27). The HMD 2802 includes a processor 2800 for executing program instructions. A memory device 2802 is provided for storage purposes. Examples of the memory device 2802 include volatile memory, non-volatile memory, or a combination thereof. A display device 2804 is included that provides a visual interface, such as displaying image frames generated from stored data for a player to view. A battery 2806 is provided as a power source for the HMD 2802. A motion detection module 2808 includes any of various types of motion sensing hardware, such as a magnetometer 2810, an accelerometer 2812, and a gyroscope 2814.
[0386] Accelerometers are devices for measuring acceleration and reaction forces caused by gravity. Single-axis and multi-axis models are available to detect the magnitude and direction of acceleration in different directions. Accelerometers are used to sense tilt, vibration, and shock. In one embodiment, three accelerometers 2812 are used to provide the direction of gravity, which provides an absolute reference for two angles, e.g., world space pitch and world space roll.
[0387] The magnetometer measures the strength and direction of the magnetic field in the vicinity of the HMD 2802. In some embodiments, three magnetometers 2810 are used in the HMD 2802 to ensure an absolute reference of the yaw angle in world spac...
Claims
1. 1. A method for emulating graphics parameters of a legacy game, comprising: receiving user input from a handheld controller during play of the legacy game; determining whether one or more basic blocks of code for processing the user input are stored in a cache; compiling the one or more basic blocks of code from one or more emulated processing unit (PU) code instructions upon determining that the one or more basic blocks are not stored in the cache; executing the one or more basic blocks of code to generate one or more legacy graphics parameters; generating one or more image frames having one or more updated graphics parameters for emulating the one or more legacy graphics parameters to display one or more images of the legacy game on a display device; The method comprising:
2. The method of claim 1 , wherein the one or more emulated PU code instructions, when executed, are incapable of generating the one or more updated graphics parameters.
3. receiving user input indicating the one or more updated graphics parameters to be applied to the one or more legacy graphics parameters; The method of claim 1 further comprising:
4. The method of claim 1 , further comprising overlaying one or more additional updated graphical parameters over the one or more image frames.
5. The method of claim 4 , wherein the one or more additional updated graphics parameters include a plurality of scan lines extending across the one or more image frames.
6. The method of claim 4 , wherein the one or more additional updated graphical parameters is a size of a crosshair displayed on the one or more images.
7. The method of claim 1 , wherein the one or more updated graphical parameters are contours of a virtual object in the one or more images.
8. 1. A system for emulating graphics parameters of a legacy game, comprising: One or more processors, receiving user input from a handheld controller during play of the legacy game; determining whether one or more basic blocks of code for processing the user input are stored in a cache; compiling the one or more basic blocks of code from one or more emulated processing unit (PU) code instructions upon determining that the one or more basic blocks are not stored in the cache; Executing the one or more basic blocks of code to generate one or more legacy graphics parameters; emulating the one or more legacy graphics parameters to generate one or more image frames having one or more updated graphics parameters for displaying one or more images of the legacy game on a display device; the one or more processors configured to a memory device coupled to the one or more processors; The system comprising:
9. 9. The system of claim 8, wherein the one or more emulated PU code instructions, when executed, are incapable of generating the one or more updated graphics parameters.
10. the one or more processors: The system of claim 8 , further configured to receive user input indicating the one or more updated graphics parameters to be applied to the one or more legacy graphics parameters.
11. the one or more processors: The system of claim 8 , further configured to overlay one or more additional updated graphical parameters over the one or more image frames.
12. The system of claim 11 , wherein the one or more additional updated graphics parameters include a plurality of scan lines extending across the one or more image frames.
13. The system of claim 11 , wherein the one or more additional updated graphical parameters is a size of a crosshair displayed on the one or more images.
14. The system of claim 8 , wherein the one or more updated graphical parameters are contours of a virtual object in the one or more images.
15. 1. A method for emulating audio parameters of a legacy game, comprising: receiving user input from a handheld controller during play of the legacy game; determining whether one or more basic blocks of code for processing the user input are stored in a cache; compiling the one or more basic blocks of code from one or more emulated processing unit (PU) code instructions upon determining that the one or more basic blocks are not stored in the cache; executing the one or more basic blocks of code to generate one or more legacy audio parameters; emulating the one or more legacy audio parameters to generate one or more audio frames having one or more updated audio parameters, the one or more audio frames being output as sound along with one or more images of the legacy game displayed on a display device; The method comprising:
16. The method of claim 15 , wherein the one or more emulated PU code instructions are not capable of generating the one or more updated audio parameters.
17. The method of claim 15, wherein the sound is mono, stereo, or surround.
18. The method of claim 15 , wherein the one or more updated audio parameters include a size of reverb output from one or more speakers.
Citation Information
Patent Citations
Game information storage medium and game system using the same
JP2001340640A
Method and apparatus for emulation enhancement
JP2009072601A
Register mapping when emulating the target system on the host system
JP2009535745A
Method and apparatus for enhancing emulation
JP2014513997A
Hand-held video game platform emulation
US20040157664A1
