Controller for Virtual Console Games

Abstract

A system and method are provided for displaying on a touchscreen display (122) of a mobile device (106) a virtual game controller (116) having at least one virtual button (118) configured to control a plurality of actions of an avatar (126) representing a player in a video game executed by a remote gaming server (102). Player interactions with the at least one virtual button are received via the touchscreen display and used to generate control signals based on the received player interactions. The control signals are interpretable by the video game as instructions to perform a plurality of actions in the video game. The control signals are transmitted from the mobile device for reception by the remote gaming server executing the video game.

Description

Background Art

[0001] Background The hardware required to display video game content or receive input from a player has evolved from a mechanism that executes a physical hard copy of game software on a dedicated computing device or console to a cloud-based service that executes on a remote server accessed by a player via the Internet. In cloud-based games, resource-intensive aspects of the game, such as graphics rendering, are performed on the server rather than on the player's local device. Therefore, in general, even with mobile devices such as mobile phones and tablets that do not have as much computing power or memory as a console or desktop computer, cloud-based video games can be played without sacrificing image quality or overusing local processing resources. Thus, currently, since video games can be played on most devices with an Internet connection, players are no longer restricted by the necessity of owning dedicated hardware for playing video games.

[0002] In video games, first-person games are based on a graphical view rendered from the perspective of the player's avatar or from the driver's seat or cockpit of a vehicle operated by the player's avatar. Games that aim to shoot or fight against the player's enemies are known as first-person shooter (FPS) games. In many FPS games, the player's avatar's hand and the selected weapon are displayed in the main view, and health status, ammunition, position, and other details are displayed on a heads-up display beside the main view. To perform actions such as running, dodging, looking around, aiming, and firing, the player uses a combination of a keyboard and a mouse or a physical game controller (such as a gamepad) equipped with designated buttons and sticks to move their own avatar within the game. Since some of these actions, such as aiming and firing, need to be performed simultaneously, a player who controls their avatar using a keyboard, mouse, or physical game controller has to activate multiple buttons and / or sticks simultaneously using multiple fingers.

[0003] Brief Description of the Drawings The present disclosure will be better understood by reference to the accompanying drawings, and numerous features and advantages thereof will become apparent to those skilled in the art. Like or identical items are denoted by the same reference numerals in the separate drawings.

Brief Description of the Drawings

[0004]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0005] Detailed Description When a player plays a cloud-based video game on a mobile device such as a mobile phone or a tablet computer equipped with a touch screen display, the touch screen display of the mobile device serves both to display the graphics of the game and to receive user input by touching to control the user's avatar. To give the user control over operating their avatar, a virtual game pad can be overlaid and displayed on top of the game itself. The virtual game pad is composed of virtual buttons corresponding to the buttons and sticks of a physical game pad controller. Thus, when the player holds a physical game pad controller, the user can control their avatar by tapping or pressing the displayed virtual buttons in substantially the same manner. However, since the player's palm and fingers are often already occupied in holding and fixing the mobile device during play, the player may only be able to use their thumb to operate the virtual buttons of the virtual game pad. As a result, the player cannot operate multiple virtual buttons simultaneously, which impairs the user experience of the player while playing the video game.

[0006] In the game system shown in FIGS. 1-7, the virtual buttons of the virtual game controller displayed on the mobile device are configured to control multiple actions of the player's avatar during the gameplay of the video game. Thus, the player can effectively provide a wide range of player input via the touch screen display of the mobile device using only the thumb, or more generally, using only one or two touch points. Further, in some embodiments, the virtual area of the virtual game controller is configured to enable the player to control the actions of their avatar within the video game as if the player were using a physical game controller having at least one stick controller.

[0007] In the context of an FPS video game, a virtual game controller typically includes virtual buttons that are displayed on the left side of the virtual controller. The virtual buttons are used to aim the weapon held by the player's avatar of and to move the player's avatar within the game scene. The virtual controller also typically includes virtual buttons that are displayed on the right side of the virtual controller. The virtual buttons are configured to adjust the view of the avatar in the video game environment (i.e., the rendered natural or urban landscape where the avatar is located within the video game) and to fire or activate the weapon held by the player's avatar. The area on the same side of the virtual game controller as this button is also configured to adjust the direction of the avatar's line of sight within the video game. The player input received at each button and in the specially configured area is converted into one or more control signals. The one or more control signals are translated by the video game as if they were provided to the video game by a physical game controller. That is, the virtual game controller displayed on the mobile device's display operates to emulate the operation of a physical game controller without requiring more than three simultaneous touch inputs in accordance with the player's operation of the virtual controller via the mobile device's touch screen. Thus, the virtual game controller enables the player to play the video game on the player's own mobile device without the need to simultaneously operate a plurality of virtual buttons or areas on the same side of the virtual controller using only the player's thumb to control the actions of the player's own avatar within the video game.

[0008] FIG. 1 shows a cloud game system 100 for cloud-based play of a video game 110 via a remote client device 106 with a touch screen-enabled virtual game controller, according to some embodiments. The cloud game system 100 includes a data center 102 having a game server 104 connected to the client device 106 via one or more packet-switched networks 108. The remote game server 104 operates to execute a video game software application 110 (hereinafter referred to as "video game 110"), and as a result, a sequence of video frames is rendered in real time. This sequence of video frames is encoded and optionally formatted by the game server 104 so as to be included in a video stream 112 transmitted to the client device 106. The client device 106 receives the video stream 112, decodes the encoded video frames included therein, and operates to sequentially display the decoded video frames to present a video display of the rendered graphical content to the player. Audio content is generated by the game server 104 by executing the video game 110 and provided to the client device 106 to be output to the player in a similar manner.

[0009] The client device 106 includes a network interface 124 connected to the network 108 to receive video data and audio data transmitted from the game server 104 via the network 108, and computing resources, storage resources, and display resources (not shown) for decoding and displaying video frames of the transmitted video stream 112 and outputting corresponding audio content. The client device 106 includes any of a variety of mobile or portable electronic devices held by the player during operation. These electronic devices include, for example, a computing-enabled mobile phone (i.e., a "smartphone") or a tablet computer, or, in the case of a computing-enabled wristwatch (i.e., a "smartwatch") or other wearable electronic device, are generally used by the player to prevent more than three simultaneous touches on the touch screen.

[0010] In some embodiments, the processing and display of video content and audio content from the video game 110 executed on the game server 104 are facilitated by a software application executed on the client device 106. This software application may include, for example, a software application specific to cloud gaming. In other embodiments, a more general software application is utilized, and the video stream 112 and an audio stream (not shown) are formatted and transmitted by the game server 104 to be compatible with this software application. For example, in one embodiment, the client device 106 utilizes a web browser that uses a hypertext markup language (HTML) format for the display of video content represented by the video stream 112 and the output of audio content represented by the associated audio stream, and the game server 104 provides the video stream 112 in a format such as mp4 or MOV that is compatible with the HTML format for display via the web browser.

[0011] Since video game 110 is interactive, it uses player input to at least partially direct the gameplay experience of the game session with the player. The player input is received at least in part via virtual game controller 116 displayed on touch screen display 122 of client device 106. Touch screen display 122 includes an electronic visual display for presenting rendered content to the player. The electronic visual display is coupled directly or by using a stylus or other device to a touch screen panel composed of sensors for detecting pressure, capacitance changes, or interruption of a light beam resulting from the player's contact with the screen. Virtual game controller 116 includes one or more player-operable components 118. In response to being operated by the player via touch screen display 122 of client device 106, player-operable component 118 causes corresponding player feedback data to be generated for the processing component of virtual game controller 116 or a processing component associated with virtual game controller 116, and supplies the player feedback data to be transmitted via packet-switched network 108 to game server 104 as part of player input stream 120. Player-operable component 118 is typically displayed as graphical content overlaid in the transmitted video stream 112. Virtual game controller 116 is generally configured to emulate a physical game pad controller, as will be further described below with reference to FIG. 4.

[0012] For ease of reference, the player-operable component 118 is generally referred to herein as the "virtual button 118" or more generally as the "button 118", and "pressing" the button, or a "press" of the button, results in the client device 106 translating a touch interaction as a specified player interaction with the virtual game controller 116, thereby generating and transmitting at least one corresponding control signal in such a manner that a player's touch or other contact-based interaction with the touch screen display 122 in the area associated with the position of the player-operable component 118 on the touch screen display 122 is understood to be.

[0013] At least some of the virtual buttons 118 of the virtual game controller 116 are configured to control a plurality of actions of the player's avatar 126 in the gameplay of a video game, which is common, for example, in the context of a first-person shooter (FPS) video game where a player may need to aim and shoot simultaneously. The virtual game controller 116 enables a player to perform a wide range of player inputs by using only one or two touch points, thereby facilitating video game gameplay on a mobile device where a player can typically use only two thumbs to perform player inputs. Thus, the virtual controller enables a player to play a video game on their own mobile device using only their thumbs to control the actions of the player's own avatar in the video game without the need to simultaneously operate multiple virtual buttons or regions on the same side of the virtual controller.

[0014] FIG. 2 shows a schematic hardware configuration 200 and a software configuration 202 of the game server 104 in FIG. 1. As shown by the hardware configuration 200, the game server 104 includes one or more processors, for example, one or more central processing units (CPUs) 204, one or more graphics processing units (GPUs) 206, one or more application-specific integrated circuits (ASICs) (not shown), and the like. The hardware configuration 200 further includes a network interface 208 coupled to the network 108, one or more input / output (I / O) devices 210 (such as a keyboard, a mouse, or a display), one or more mass storage devices 212 (such as a hard drive, an optical disk drive, or a network-connected storage), and the like. The hardware configuration 200 further includes at least one random access memory (RAM) or other system memory 214 accessible by one or more processors. The system memory 214 stores one or more software applications including program code representing executable instructions for operating one or more processors to execute various operations described herein. These software applications include the video game 110 described above and a set of one or more software applications collectively referred to as platform software 216.

[0015] Platform software 216 operates to facilitate the execution of video game 110 on game server 104. In some implementations, the game developer of video game 110 has specifically configured video game 110 to be executed by the game server, in which case platform software 216 provides an application programming interface (API) and other "hooks" that provide a more transparent interaction between hardware configuration 200 and video game 110. In other embodiments, video game 110 is developed for a specific local hardware platform such as a personal computer (PC) platform or a game console platform, in which case platform software 216 emulates a typical interface between video game 110 and its expected local hardware platform, thereby making the actual hardware configuration 200 underlying game server 104 less transparent to video game 110. An exemplary configuration of platform software 216 and its interaction with video game 110 are shown by software configuration 202.

[0016] In the illustrated software configuration 202, the platform software 216 includes an operating system (OS) 218, a game server module 220, a media server module 222, an encoder module 224, a player interaction type module 226, and various other modules for supporting the execution of cloud games known in the art. The OS 218 manages the overall operation of the game server 104 and operates to function as an interface between the hardware components and a higher-level software layer. The game server module 220 operates to provide server-side functions related to the video game 110, such as setting up a game session, storing session state data and other game-related data, processing game play inputs, and rendering game play outputs in response to the game play inputs. The media server module 222 hosts a media streaming site, receives parallel auxiliary or supplementary media streams related to an online game session, and operates to provide a parallel media stream (e.g., the video stream 112 in FIG. 1) to the client device 106 for display simultaneously with the game session supported during the execution of the video game 110. The encoder module 224 operates to encode the media stream and transmit it to the client device 106. Although illustrated as a software module executed by the processor resources of the game server 104, in other embodiments, the encoder module 224 can be implemented in whole or in part as a hard-coded integrated circuit such as an ASIC or a programmable logic integrated circuit.

[0017] The player interaction type module 226 extracts player input data in the player input stream 120 and operates to present the player input data to the video game 110 in a format appropriate as player input. By way of example, in an implementation where the platform software 216 emulates a PC, a game console, or another local game environment, the player interaction type module 226 emulates the input such that the input to one or more instances of the video game 110 appears to come from a physical game controller for each instance rather than from the virtual game controller 116 that is remote from the game server 104 and connected via the network 108, as if it were player input coming from a physical game controller.

[0018] Figure 3 is a block diagram showing an exemplary implementation of the client device 106 according to some embodiments. The client device 106 includes one or more processing units (CPUs) 302, one or more network interfaces 304, a memory 306, and one or more communication buses 308 for interconnecting these components (sometimes referred to as a chipset). The client device 106 includes one or more input devices 310 that facilitate player input, such as a keyboard, a voice command input unit, a touch sensor type input pad, a gesture capture camera, or other input buttons or controls. The client device 106 also includes one or more output devices 312 that include one or more speakers and / or one or more visual displays that enable presentation of the user interface and display of content. The client device 106 also includes a touch screen display 122 configured to display content to the player and receive input from the player at the interface.

[0019] In some embodiments, the client device 106 further includes a position detection device 314, such as a global positioning satellite (GPS) receiver or other geolocation receiver, for determining the position of the client device 106. The client device 106 may also include a proximity detection device 315, such as an infrared (IR) sensor, for determining whether other client devices are in proximity. The client device 106 may also include one or more sensors 313 (such as an accelerometer, a gyroscope, etc.) for detecting the movement, orientation, and other parameters of the client device 106 that can be used as inputs.

[0020] The memory 306 includes a random access memory and may further include non-volatile memory, such as one or more magnetic disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid state storage devices. The memory 306 stores various programs, modules, and data structures, or subsets or supersets thereof. Exemplary programs stored in the memory 306 include an operating system 316 that includes procedures for processing various basic system services and procedures for performing hardware-dependent tasks, and a network communication module 318 for connecting the client device 106 to other devices (such as the game server 104 or other client devices) via one or more network interfaces 304 (wired or wireless) and one or more networks 108, such as the Internet, other wide area networks, local area networks, metropolitan area networks, etc.

[0021] In some embodiments, the memory stores a user interface module 320 for enabling the presentation of information (e.g., a graphical user interface for presenting applications, widgets, video game content, etc.) at the client device 106 via one or more output devices 312 (e.g., the touch screen display 122). The user interface module 320 includes an input processing module 322 for detecting one or more player inputs or interactions from one of the one or more input devices 310 and translating the detected input or interaction. In some embodiments, the memory 306 stores a client-side game module 328 for providing client-side functions related to one or more game titles, including but not limited to, displaying a virtual game controller such as the virtual game controller 116, setting up a game session, locally storing session state data and other game-related data, receiving game play inputs from the input device 310 and / or the touch screen display 122, and providing game play outputs in response to the game play inputs. In this case, in some implementations, the client-side game module 328 is configured to provide game play outputs to a remote server 104 that implements the video game 110.

[0022] In some embodiments, the memory 306 stores client device settings 334 for storing information related to the client device 106 itself, including common device settings (e.g., service layer, device model, storage capacity, processing power, communication capabilities, etc.). The client device settings include game application settings 338 for storing information related to the player accounts of the game application, including one or more of account access information, basic settings of in-game players, game play history data, and information about other players, and game controller configurations 340 for storing information related to the configuration of the client-side game module 328 for the game application 328, and local game session data 344. The local game session data 344 includes information related to online game sessions executed on the client device 106 in relation to various game titles, such as the latest status information, texture pages, vertex pages, constant pages, and frame data of one or more image frames associated with each of the online game sessions.

[0023] Each of the elements identified above can be stored in one or more of the aforementioned memory devices and corresponds to a set of instructions for performing the functions described above. The modules or programs (i.e., sets of instructions) identified above need not be implemented as separate software programs, procedures, modules, or data structures, and thus, in various implementations, various subsets of these modules may be combined or reconfigured. In some implementations, the memory 306 stores a subset of the modules and data structures identified above. Additionally, the memory 306 may store additional modules and data structures not described above.

[0024] FIG. 4 shows an exemplary configuration of a virtual game controller 416 (an embodiment of virtual game controller 116) for receiving player input to control a video game such as video game 110. The virtual game controller 416 is configured to display virtual buttons 118 on either or both the right and left sides of the touch screen display 122 of the client device 106, such that the player can operate the virtual buttons with his or her thumb and other fingers when holding the device. The virtual buttons 118 are configured to “receive” player input (via the touch screen display 122) to manipulate the player's own avatar 126 and objects within the control range of the player's own avatar in video game 110, as if the player were using a physical game controller to provide player input to video game 110. A number of virtual buttons 118, and combinations thereof, can be implemented by the virtual game controller 416 and can be configured to receive player input to control various aspects of video game 110 and the player's gaming experience. For ease of explanation, however, only some of the virtual buttons 118 and their functions are described in detail below, and the functions and operations of the other buttons 118 will be readily understood based on the following description.

[0025] In the context of a first-person shooter (FPS) video game, the left trigger button 402 is used to aim down sight (ADS) at a weapon 410 or object held by the player's avatar 126. The left stick button 404 is configured to move the player's avatar 126 within the video game 110. The right stick button 406 is configured to pan the viewpoint 412 of the player's avatar 126 within the video game 110, and the right trigger button 408 is configured to activate an object or fire a weapon 410 within the control range of the avatar 126 within the video game 110. In a physical game controller, the right stick provides velocity-based feedback to the video game 110 such that as the stick moves farther from its original center position, the field of view 412 of the avatar in the video game 110 environment changes faster and farther. The right stick button 406 of the virtual game controller 116 is configured to emulate the velocity-based function of the right stick on a physical game pad by adjusting the field of view 412 of the avatar based on how far the right stick button 406 is dragged by the player from its original position. As a result, a relatively small movement of the right stick button 406 is translated into a large change in the field of view 412 of the avatar, and thus it becomes difficult to accurately move the field of view of the avatar.

[0026] In the described configuration, the player generally needs to simultaneously operate several combinations of the virtual buttons 118 to achieve specific movements and actions of the avatar 126. For example, to identify and aim at the position of a target within the video game 110, the player typically needs to hold the left trigger button 402 while simultaneously operating the direction in which their avatar 126 is moving by operating the left stick button 404, in order to look through the sight of the weapon 410 held by their avatar 126 and aim. Also, for example, it is common for the player to need to simultaneously operate the right stick button 406 and the right trigger button 408 in order to change the view 412 of the avatar while firing the weapon 410. However, when holding the device, the player can generally only use their thumb to operate the virtual buttons 118, which makes it difficult to simultaneously control multiple virtual buttons 118.

[0027] FIG. 5 shows an exemplary configuration of a virtual controller 516 (another embodiment of the virtual game controller 116) of the client device 106 according to some embodiments. The virtual controller 516 is configured to display the virtual buttons 118 on both the right and left sides of the touch screen 122 of the client device 106, thereby enabling the player to operate the buttons with only their own thumb when holding the device. In one embodiment, a draggable aiming and movement button 502 is provided and is displayed on the left side of the touch screen so as to be operable by the player's left thumb. The aiming and movement button 502 is configured to be draggable within the area of the touch screen 122. When the player drags the aiming and movement button 502, as a result, the player's avatar 126 can move an object 510 within its control range ofSimultaneously with aiming, the player's avatar 126 moves within the video game 110. Thus, the aiming and movement button 502 combines the functions of the left trigger button 402 and the left stick button 404 of the virtual game controller 416 shown in FIG. 4. Although the aiming and movement button 502 is illustrated and described as being positioned on the left side of the touch screen 122 of the client device 106, the button layout of the virtual controller 516 can be configured by the player to be displayed at any position on the touch screen 122.

[0028] In one embodiment, a draggable point and a fire button 504 are provided and are displayed on the right side of the touch screen 122 of the client device 106. The point and fire button 504 is configured to be draggable within the area of the touch screen 122. The point and fire button 504 is also configured to activate an object 510, such as a rifle held by the player's avatar 126, as long as the player holds the button 504 when the player first touches the button 504. The point and fire button 504 is further configured to adjust the direction in which the player's avatar is looking in response to the player dragging the point and fire button 504 within the area of the touch screen 122. Thus, the player can direct the line of sight of his or her avatar 126 while firing or activate an object within the control range of the avatar by pressing and dragging the point and fire button 504 within the area of the touch screen 122. The point and fire button 504 of the virtual controller 516 essentially combines the functions of the right trigger button 408 and the right stick button 406 of the virtual controller 416 shown in FIG. 4. The point and fire button 504 is illustrated and described as being positioned on the right side of the touch screen 122 of the client device 106, but the layout of the buttons 118 of the virtual controller 516 can be configured by the player to be displayed at any position on the touch screen 122.

[0029] In one embodiment, the virtual controller 516 is configured with a defined area 506 (hereinafter referred to as the "view panning area 506") associated with the right side of the touch screen 122. The view panning area 506 enables the player to adjust the direction of the avatar's view 512 in the video game 110 environment based on the player touching the touch screen 122 within the view panning area 506. The view panning area 506 is configured such that a specific player input provided by a touch contact to the area of the touch screen display 122 associated with the view panning area 506 moves the avatar's view 512 to the player's touch position on the touch screen 122 within the view panning area 506 and continues to pan the avatar's view 512 as the player moves the contact point within the view panning area 506. By using the view panning area 506, the player can accurately adjust the view 512 of the player's own avatar in the video game 110 environment. To be conveniently accessible to the player holding the client device 106, the view panning area 506 can be sized and positioned such that the player's thumb can reach the view panning area 506 while the palm and fingers of the player's same-side hand are blocked from holding the client device 106.

[0030] FIG. 6 shows a method 600 for controlling the gameplay of a video game 110 in a cloud-based gaming system 100 based on player input provided via a virtual game controller 116 displayed on a client device 106, according to some embodiments. After a player starts a gaming session, in block 602, video game content and the virtual game controller 116 are displayed on the client device 106. As described above with reference to the embodiment of FIG. 5, the virtual game controller 116 may include, for example, one or more virtual buttons 118. The one or more virtual buttons 118 include, at least, aiming and movement buttons 502 disposed on one side (e.g., the left side) of the touch screen display 122, and pointing and shooting buttons 504 disposed on the opposite side (e.g., the right side) of the touch screen display 122. In certain embodiments, the virtual game controller 116 includes a field-of-view panning region 506 disposed on the right side of the touch screen 122.

[0031] Player input is received through player contact with the virtual game controller 116's virtual buttons 118 and player interaction with the field-of-view panning region 506 through player contact with the relevant area of the touch screen display 122, as represented by one or more of blocks 604, 610, and 616. For example, the player can operate the aiming and movement button 502 and the point-and-shoot button 504 by contacting the areas of the touch screen display 122 associated with each of the buttons 502, 504, and while remaining in contact with the touch screen display 122 and with the player's palm and fingers blocking to hold the client device 106, moving the buttons 502, 504 within the area of the touch screen display 122 reachable by the player's thumb. Similarly, the player can instruct the video game 110 to change the direction of the avatar's line of sight by contacting the portion of the touch screen 122 associated with the field-of-view panning region 506 and moving the contact point within the field-of-view panning region 506 while remaining in contact with the touch screen 122.

[0032] Next, the received player input is converted into a control signal, and the converted control signal is transmitted, received by the game server 104, provided to the video game 110 as part of the player input stream 120, and further translatable by the video game 110 as instructions for performing an action in the video game 110. For example, the position information regarding the movement of the player's touch point associated with the virtual button 118 of the virtual game controller 116 is converted into a control signal and provided to the video game 110, and will be translated as instructions for realizing a specific activity in the video game 110. In one embodiment, at block 606, the player input received through the player operation of the aiming and movement buttons 502 is converted into a control signal translatable by the video game 110 as instructions for moving the player's avatar 126 within the video game 110 environment based on the direction and distance in which the player moves the aiming and movement buttons 502. The player input received via the aiming and movement buttons 502 is also, at block 608, converted into a control signal translatable by the video game for instructions to aim at an object within the control range of the avatar of is converted into a control signal translatable by the video game as instructions for aiming.

[0033] In one embodiment, the player input received through the player operation of the point and shoot buttons 504 is, at block 612, converted into a control signal translatable by the video game 110 as instructions for changing the direction of the line of sight of the avatar 126 as if the avatar were moving its own head or eyes to look at various parts of the video game 110 environment based on the direction and distance in which the player moves the point and shoot buttons 504. The player input received via the point and shoot buttons 504 is also, at block 614, converted into a control signal translatable by the video game 110 as instructions for activating an object within the control range of the avatar or for firing a weapon and continuing to fire until the player releases the point and shoot buttons 504.

[0034] In one embodiment, player input received through player contact within the view panning region 506 is translated by the video game 110 into a control signal that can be translated as an instruction to change the direction of the avatar 126's line of sight based on the position where the contact within the view panning region 506 was initiated and the position where the contact was interrupted, at block 618.

[0035] At block 620, a control signal generated based on player input received via the virtual game controller 116 is provided to the game server 104 via the player input stream 120. At block 622, the video game 110 translates the provided control signal. The game server 104 controlled by the video game 110 then renders video content based on the implemented control signal at block 624 and provides video content for display at the client device 106.

[0036] FIG. 7 is a flow diagram showing a method 700 for converting player input received in a field-of-view panning region, as shown in block 618 of FIG. 6, into a control signal to be translated by video game 110 to adjust the line of sight of player avatar 126. The field-of-view panning region 506 is configured to emulate the function of the right stick of a physical game controller. Thus, rapidly moving the contact point within the field-of-view panning region 506 results in a control signal that can be translated by video game 110 as if the right stick of the physical controller has been moved to a position far from the default center position. The resulting action within video game 110 is that the avatar's line of sight pans quickly in the direction the player has moved within the field-of-view panning region 506. Similarly, slowly moving the contact point within the field-of-view panning region 506 results in a control signal that can be translated by video game 110 as if the right stick of the physical controller has been moved to a position near the default center position, and thus the avatar's line of sight pans slowly in the direction the player has moved within the field-of-view panning region 506.

[0037] The generation of a control signal translatable by video game 110 as the movement of the right stick of a physical game controller is achieved by using a heuristic function to convert the linear velocity of the movement of the contact point within the field-of-view panning region 506 into a non-linear signal (e.g., a non-linear signal that exactly matches the signal that video game 110 would receive from the right stick of a physical game controller). Thus, in block 702, as the player moves the contact point at a velocity in a desired direction within the field-of-view panning region 506, player input is received in the form of linear velocity within the field-of-view panning region 506. In block 704, a heuristic function is applied to the velocity input received within the field-of-view panning region 506 to convert the linear velocity input into a non-linear signal. As shown by block 706, if the converted signal is less than a defined threshold, in block 708, the converted signal is ignored by virtual game controller 116 (i.e., not subject to the action of virtual game controller 116) and no control signal is transmitted to video game 110. This can also improve haptic sensation and control, and as a result, only certain actions will be translated as triggers for generating a control signal. This can also reduce motion errors. If the converted signal is above the threshold, in block 710, the converted signal is normalized and limited to a value within the range of signal values (i.e., the signal range) that would be output by the right stick of a physical game controller. Then, in block 712, the normalized and limited control signal is transmitted to video game 110 and will be translated as an instruction to move the field of view of player avatar 126.

[0038] In some embodiments, some aspects of the technologies described above may be implemented by one or more processors of a processing system that executes software. The software includes one or more sets of executable instructions stored on or tangibly embodied on a non-transitory computer-readable storage medium. When executed by one or more processors, the software may include instructions that operate one or more processors to perform one or more aspects of the technologies described above and some data. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid state storage devices such as flash memory, cache, random access memory (RAM), or other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be in source code, assembly language code, object code, or other instruction formats that are translated or executable by one or more processors.

[0039] A computer-readable storage medium can include any storage medium or combination of storage media that is accessible by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-ray disc), magnetic media (e.g., floppy (registered trademark) disk, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer-readable storage medium may be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., magnetic hard drive), removably attached to a computing system (e.g., optical disc or Universal Serial Bus (USB)-based flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0040] Not all of the activities or elements described in the general description are required, some of a particular activity or device may not be required, and it should be noted that one or more additional activities may be performed in addition to those described, or some elements may be included in addition to those described. Further, the order in which activities are recited is not necessarily the order in which they are performed. Also, concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the appended claims. Accordingly, the specification and drawings should be interpreted in an illustrative rather than a limiting sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0041] Advantages, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, these advantages, solutions to problems, and any feature that may bring about or make more prominent any advantage, benefit, or solution should not be construed as an important, necessary, or essential feature of any claim or all claims. Further, since various equivalent modifications and implementations of the subject matter disclosed above will be apparent to those skilled in the art who have the benefit of the teachings of this specification, the specific embodiments disclosed above are merely illustrative. It is not intended to limit the details of the structure or design shown in this specification other than as described in the appended claims. Accordingly, it is clear that the specific embodiments disclosed above can be changed or modified, and all such variations are considered to be within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the appended claims.

Claims

1. A method comprising: displaying a virtual game controller on a touch screen display of a mobile device, the virtual game controller comprising: a first draggable virtual button configured to simultaneously control the movement of an avatar representing a player in a video game executed by a remote game server and the aiming of an object by the avatar in the video game executed by the remote game server; a second draggable virtual button comprising a touch point and configured to simultaneously control the line of sight of the avatar in the video game executed by the remote game server and the firing of an object in the video game executed by the remote game server, the method further comprising: receiving, via the touch screen display, a player interaction with the first draggable virtual button or the second draggable virtual button; generating a control signal based on the received player interaction, the step of generating the control signal including converting a first contact with the touch point of the second draggable virtual button on the touch screen display into the control signal including at least one instruction interpreted by the video game to fire the object in the video game; transmitting the control signal from the mobile device to be received by the remote game server.

2. The step of generating the control signal further includes converting position information related to the player dragging a touch point associated with the first draggable virtual button into instructions for the video game to translate it as the position where the avatar aims at the object in the video game, and the control signal further includes the instructions for the video game to translate it as the position where the avatar aims at the object in the video game. The method according to claim 1.

3. The step of generating the control signal further includes converting position information related to the player dragging a touch point associated with the first draggable virtual button into instructions for the video game to translate it as the movement of the avatar in the video game, and the control signal further includes the instructions for the video game to translate it as the movement of the avatar in the video game. The method according to claim 1.

4. The step of generating the control signal further includes converting position information related to the player dragging a touch point associated with the second draggable virtual button on the touch screen display into instructions for the video game to translate it as moving the line of sight of the avatar in the video game, and the control signal further includes the instructions for the video game to translate it as moving the line of sight of the avatar in the video game. The method according to claim 1.

5. A device, The touch screen display configured to display content from a game server and receive player input through contact with the touch screen display, A network interface configured to communicate with the game server before the player executes a video game represented by an avatar, including at least one processing unit, wherein the at least one processing unit displays a virtual controller including a first draggable virtual button configured to simultaneously control the movement of the avatar and the aiming of an object by the avatar in the video game, and a second draggable virtual button including a touch point and configured to simultaneously control the line of sight of the avatar and the firing of an object in the video game in the video game, configured to generate a control signal to be translated by the video game as instructions for the avatar to perform a plurality of actions in the video game based on received player input associated with the first draggable virtual button or the second draggable virtual button, generating the control signal includes converting an initial contact with the touch point of the second draggable virtual button on the touch screen display into the control signal including at least one instruction to be interpreted by the video game to fire the object in the video game., device.

6. The at least one processing unit further configured to generate the control signal by converting position information related to the player dragging a touch point associated with the first draggable virtual button on the touch screen display into instructions for the video game to translate as the position where the avatar is aiming at the object in the video game, the control signal further includes the instructions for the video game to translate as the position where the avatar is aiming at the object in the video game. The device according to claim 5.

7. The at least one processing unit is further configured to generate the control signal by converting position information associated with the player dragging a touch point associated with the first draggable virtual button on the touch screen display into instructions for translation by the video game as movement of the avatar within the video game, the control signal further including the instructions for translation by the video game as movement of the avatar within the video game. The device according to claim 5.

8. The at least one processing unit is further configured to generate the control signal by converting position information associated with the player dragging the touch point associated with the second draggable virtual button on the touch screen display into instructions for translation by the video game to move the line of sight of the avatar within the video game, the control signal further including the instructions for translation by the video game to move the line of sight of the avatar within the video game. The device according to claim 5.

Images