Virtual console game controller

Abstract

A system and method for displaying a virtual game controller (116) on a touchscreen display (122) of a mobile device (106), the virtual game controller (116) having at least one virtual button (118) configured to control a plurality of actions of an avatar (126) representing a player within a video game executed by a remote game server (102). The player's interaction with the at least one virtual button is received through the touchscreen display and used to generate a control signal based on the received player interaction, the control signal interpretable by the video game as an instruction to perform the plurality of actions in the video game. The control signal is transmitted from the mobile device for receipt by the remote game server executing the video game.

Description

BACKGROUND

[0001] The hardware required to display video game content and receive input from a player has evolved from physical hard copies of game software running on a specialized computing device or console to cloud-based services in which the game software is executed on a remote server accessed by the player over the Internet. In cloud-based gaming, resource-intensive aspects of the game, such as graphics rendering, occur on the server rather than on the player's local device. As a result, mobile devices, such as cell phones and tablets, which typically do not have as much computing power or storage as a console or desktop computer, can be used to play cloud-based video games without sacrificing image quality or overusing local processing resources. As a result, players are no longer required to have specialized hardware to play video games, as video games can now be played on most any device with an Internet connection.

[0002] In video games, first-person games are based on rendered graphics from the perspective of the player's avatar, or presented from the perspective of the driver's seat or cockpit of a vehicle operated by the player's avatar. Games in which the player's objective is to shoot or otherwise fight enemies are referred to as first-person shooter (FPS) games. Many FPS games display the player's avatar's hands and their selected weapon in the main view, with the main view Figure One The player's health, ammunition, position, and other details are displayed in a heads-up display on the side. The player moves their avatar in the game using a combination of a keyboard and mouse or a physical game controller (e.g., gamepad) with designated buttons and sticks to perform actions such as running, strafing, looking, aiming, shooting, etc. Because some of these actions, such as aiming and shooting, need to occur simultaneously, players using a keyboard or mouse or a physical game controller to control their avatar use multiple fingers to activate multiple buttons and / or sticks simultaneously. BRIEF DESCRIPTION OF DRAWINGS

[0003] The present disclosure will be better understood by reference to the accompanying drawings, of which numerous features and advantages of various embodiments will be apparent to those skilled in the art. Like reference numerals refer to similar or identical items throughout the various drawings.

[0004] Figure 1 is a block diagram of a cloud gaming system configured to facilitate a user playing a video game on a remote client device via a virtual game controller displayed on a touchscreen display, in accordance with some embodiments.

[0005] Figure 2 is a block diagram illustrating an example hardware and software configuration of a client device, in accordance with some embodiments.

[0006] Figure 3is a block diagram illustrating another example hardware and software configuration of a client device, in accordance with some embodiments.

[0007] Figure 4 is a diagram illustrating a configuration of a virtual controller to control a video game from a client device, in accordance with some embodiments.

[0008] Figure 5 is a diagram illustrating another configuration of a virtual controller to be displayed on a client device, in accordance with some embodiments.

[0009] Figure 6 is a flowchart illustrating a method for receiving user input at a virtual controller, in accordance with some embodiments.

[0010] Figure 7 is a flowchart illustrating a method of translating user input received at a view panning area into control signals to be interpreted by a video game for adjusting a gaze of an avatar of a player, in accordance with some embodiments. DETAILED DESCRIPTION

[0011] When a player plays a cloud-based video game on a mobile device, such as a mobile phone or tablet with a touchscreen display, the touchscreen display of the mobile device is used to display the graphics of the game and to receive user input via touch in order to control the user's avatar. In order to provide the user with control over the manipulation of their avatar, a virtual gamepad can be displayed over the game itself. The virtual gamepad is configured with virtual buttons that correspond to the buttons and sticks of a physical gamepad controller, allowing the user to control their avatar in roughly the same way that the player would if they were holding a physical gamepad controller, by tapping or otherwise pressing the displayed virtual buttons. However, the user often only have their thumbs available to manipulate the virtual buttons of the virtual gamepad, as their palms and fingers are already used to hold and secure the mobile device during gameplay. This makes it impractical to manipulate multiple virtual buttons at the same time, thereby detracting from the user's experience when playing the video game.

[0012] Figures 1-7 A game system is illustrated in which the virtual buttons of a virtual gamepad displayed on a mobile device are configured to control multiple actions of an avatar of a player in the gameplay of a video game, enabling the player to effectively provide a wide range of player input via the touchscreen display of the mobile device using only their thumbs, or more generally using only one or two touch points. Furthermore, in some embodiments, a virtual area of the virtual gamepad is configured to allow the player to control actions of their avatar within the video game as if the player were using a physical game controller with at least one stick controller.

[0013] In the context of an FPS video game, the virtual game controller includes virtual buttons, typically displayed on the left side of the virtual controller, configured to aim a weapon held by the player's avatar and to move the player's avatar within the game scene. The virtual controller also includes virtual buttons, typically displayed on the right side of the virtual controller, configured to adjust the view of the avatar of the video game environment (i.e., the rendered landscape or cityscape in which the avatar is located within the video game) and to fire or otherwise activate a weapon held by the player's avatar. An area within the same side of the virtual game controller as the buttons is configured to also adjust the direction of the gaze of the avatar within the video game. Player input received at each of the buttons and at the specially configured area is converted to one or more control signals that are interpreted by the video game as if the one or more control signals were provided to the video game by a physical game controller. That is, the virtual game controller displayed on the display of the mobile device, in conjunction with the manipulation of the virtual controller by the player via the touch screen of the mobile device, operates to simulate the operation of a physical game controller without requiring more than two simultaneous touch inputs. Thus, the virtual game controller allows the player to play the video game on their mobile device using only their thumbs without needing to simultaneously manipulate multiple virtual buttons or areas of the same side of the virtual controller in order to control the actions of their avatar within the video game.

[0014] Figure 1 A cloud gaming system 100 for cloud-based game play of a video game 110 via a remote client device 106 that employs a touch screen-enabled virtual game controller is illustrated in accordance with some embodiments. The cloud gaming system 100 includes a data center 102 having a game server 104 connected to the client device 106 via one or more packet-switched networks 108. The remote game server 104 operates to execute a video game software application 110 (hereinafter, "video game 110") that results in real-time rendering of a sequence of video frames, which the game server 104 encodes and otherwise formats for inclusion in a video stream 112 transmitted to the client device 106. The client device 106 operates to receive the video stream 112, decode the encoded video frames contained therein, and display the decoded video frames in sequence to provide a video presentation of the rendered graphical content to the player. Audio content is generated by the game server 104 through execution of the video game 110 and is provided to the client device 106 for output to the player in a similar manner.

[0015] The client device 106 includes a network interface 124 connected to the network 108 to receive video data and audio data transmitted from the game server 104 via the network 108 and resources (not shown) to compute, store, and display video frames for decoding and displaying the transmitted video stream 112 and to output corresponding audio content. The client device 106 includes any of a variety of mobile or portable electronic devices held by a player when in operation, including a computing-enabled cellular telephone (i.e., a “smartphone”) or tablet computer, or otherwise commonly utilized by a player to render more than two simultaneous touches on a touchscreen infeasible, such as in the case of a computing-enabled watch (i.e., a “smartwatch”) or other wearable electronic device.

[0016] In some embodiments, the processing and display of video and audio content from the video game 110 performed at the game server 104 is facilitated by a software application executed at the client device 106. The software application can include, for example, a cloud gaming-specific software application. In other embodiments, a more general-purpose software application is utilized, and the video stream 112 and audio stream (not shown) are formatted and transmitted by the game server 104 to be compatible with the software application. For example, in one embodiment, the client device 106 utilizes a web browser that utilizes a hypertext markup language (HTML) format for displaying video content represented by the video stream 112 and outputting audio content represented by an associated audio stream, and the game server 104 provides the video stream 112 in a format compatible with the HTML format, such as.mp4 or.MOV, for display via the web browser.

[0017] As interactive, the video game 110 uses player input to at least partially direct the gameplay experience of a game session with the player. The player input is received at least in part through a virtual game controller 116 displayed on a touchscreen display 122 of the client device 106. The touchscreen display 122 includes an electronic visual display for presenting rendered content to the player, coupled to a touchscreen panel consisting of sensors for detecting pressure, changes in capacitance, or beam interruptions caused by the player directly or through the use of a stylus or other tool contacting the screen. The virtual game controller 116 includes one or more player-manipulable components 118 that are responsive to being manipulated by the player via the touchscreen display 122 of the client device 106, causing processing components of the virtual game controller 116 or otherwise associated with the virtual game controller 116, to generate and provide corresponding player feedback data for transmission to the game server 104 via the packet-switched network 108 as part of a player input stream 120. The player-manipulable components 118 are typically displayed as overlaid on display graphical content represented in the transmitted video stream 112. The virtual game controller 116 is typically configured to emulate a physical gamepad controller, as described below with reference to Figure 4 Further described.

[0018] For ease of reference, the player-manipulable components 118 are generally referred to herein as "virtual buttons 118" or more generally as "buttons 118," and a "pressing down" of a button or "pressing down" of a button will be understood as the player's touch or other contact interaction with the touchscreen display 122 in an area associated with the location of the player-manipulable component 118 on the touchscreen display 122 in a manner that causes the client device 106 to interpret the touch interaction as designating the player's interaction with the virtual game controller 116, and thus generate and transmit at least one corresponding control signal.

[0019] At least some of the virtual buttons 118 of the virtual game controller 116 are configured to control a plurality of actions of the player's avatar 126 in the gameplay of the video game, which is common in the context of FPS video games, for example, where the player can need to aim and shoot simultaneously. The virtual game controller 116 allows the player to provide a wide range of player input using only one or two touch points, which facilitates playing the video game on a mobile device where the player typically has only two thumbs available to provide player input. Thus, the virtual controller allows the player to play the video game on their mobile device using only their thumbs without needing to simultaneously manipulate multiple virtual buttons or areas on the same side of the virtual controller in order to control the actions of their avatar within the video game.

[0020] Figure 2The diagram shows... Figure 1 The game server 104 comprises an overall hardware configuration 200 and a software configuration 202. As depicted in hardware configuration 200, game server 104 includes one or more processors, such as one or more central processing units (CPUs) 204, one or more graphics processing units (GPUs 206), one or more application-specific integrated circuits (ASICs) (not shown), etc. Hardware configuration 200 further includes a network interface 208 coupled to network 108, one or more input / output (I / O) devices 210 (such as a keyboard, mouse, or monitor), one or more mass storage devices 212 (such as a hard disk drive, optical disk drive, or network attached storage), etc. Hardware configuration 200 further includes at least one random access memory (RAM) or other system memory 214 accessible by the one or more processors. System memory 214 stores one or more software applications, which include program code representing executable instructions that manipulate the one or more processors to perform the various operations described herein. These software applications include the aforementioned video game 110 and a collection of one or more software applications collectively referred to as platform software 216.

[0021] Platform software 216 operates to facilitate the execution of video game 110 at game server 104. In some embodiments, the game developer of video game 110 specifically configures video game 110 for execution by the game server, in which case platform software 216 provides an application programming interface (API) and other "hooks" that provide more transparent interaction between hardware configuration 200 and video game 110. In other embodiments, video game 110 is developed for a specific native hardware platform such as a personal computer (PC) platform or a game console platform, in which case platform software 216 simulates a typical interface between video game 110 and its intended native hardware platform, thereby rendering the actual underlying hardware configuration 200 of game server 104 less transparent to video game 110. An example configuration of platform software 216 and its interaction with video game 110 is illustrated by software configuration 202.

[0022] In the depicted software configuration 202, the platform software 216 includes an operating system (OS) 218, a game server module 220, a media server module 222, an encoder module 224, a player interaction module 226, and other various modules for supporting the execution of cloud gaming as is known in the art. The OS 218 operates to manage the overall operation of the game server 104 and to serve as an interface between hardware components and higher-level software layers. The game server module 220 operates to provide server-side functionality associated with the video game 110, such as setting up game sessions, storing session state data and other game-related data, processing game play inputs, and rendering game play outputs in response to game play inputs. The media server module 222 operates to host a media streaming site, receive simultaneous auxiliary or supplemental media streams associated with online game sessions, and provide the simultaneous media streams (e.g., video streams 112) to the client devices 106 for display contemporaneously with the game sessions supported during execution of the video game 110. The encoder module 224 operates to encode the media streams for transmission to the client devices 106. While illustrated as a software module executed by processor resources of the game server 104, in other embodiments, the encoder module 224 can be implemented in whole or in part as a hard-coded or programmable logic integrated circuit, such as an ASIC. Figure 1

[0023] The player interaction module 226 operates to extract player input data in the player input stream 120 and present the player input data as player input to the video game 110 in an appropriate format. To illustrate, in an implementation in which the platform software 216 emulates a PC, game console, or other local game environment, the player interaction module 226 can operate to emulate input to one or more instances of the video game 110 as if the input came from a player input of a physical game controller for each instance, rather than a virtual game controller 116 remote from the game server 104 and connected via the network 108.

[0024] Figure 3 ​is a block diagram illustrating an example implementation of a client device 106 in accordance with some embodiments. The client device 106 includes one or more processing units (CPUs) 302, one or more network interfaces 304, memory 306, and one or more communication buses 308 for interconnecting these components (sometimes called a chipset). The client device 106 includes one or more input devices 310 that facilitate input into the client device 106, such as a keyboard, voice command input unit, touch-sensitive input pad, gesture capture camera, or other input buttons or controls that facilitate input. The client device 106 also includes one or more output devices 312 that enable presentation of user interfaces and display content, including one or more speakers and / or one or more visual displays. The client device 106 also includes a touchscreen display 122 that is configured to both display content to the player and receive input from the player at the interface.

[0025] In some embodiments, the client device 106 further includes a location detection device 314, such as a GPS (Global Positioning Satellite) receiver or other geo-location receiver, for determining the location of the client device 106. The client device 106 can also include a proximity detection device 315, e.g., an infrared (IR) sensor, for determining the proximity of other client devices. The client device 106 can also include one or more sensors 313 (e.g., accelerometers, gyroscopes, etc.) for sensing the motion, orientation, and other parameters of the client device 106, which can be used as input.

[0026] The memory 306 includes random access memory and further can include non-volatile memory, such as one or more disk storage devices, one or more optical storage devices, one or more flash storage devices, or one or more other non-volatile solid-state storage devices. The memory 306 stores various programs, modules, and data structures, or subsets or supersets thereof. Example programs stored in the memory 306 include an operating system 316 that includes procedures for handling various basic system services and for performing hardware dependent tasks, a network communication module 318 that facilitates connecting the client device 106 to other devices such as the game server 104 or other client devices via the one or more (wired or wireless) network interfaces 304 and one or more networks 108, such as the Internet, other wide area networks, local area networks, metropolitan area networks, and so on.

[0027] In some embodiments, the memory stores a user interface module 320 for enabling presentation of information at the client device 106 via one or more output devices 312 (e.g., a touchscreen display 122) (e.g., a graphical user interface for presenting applications, widgets, video game content, etc.). The user interface module 320 includes an input processing module 322 for detecting one or more player inputs or interactions from one of the one or more input devices 310 and interpreting the detected inputs or interactions. In some embodiments, the memory 306 stores a client-side game module 328 for providing client-side functionality associated with one or more game titles, including but not limited to displaying virtual game controllers such as virtual game controller 116, setting up game sessions, locally storing session state data and other game-related data, receiving game play inputs from input devices 310 and / or touchscreen display 122, and providing game play outputs in response to game play inputs, where in some implementations the client-side game module 328 is configured to provide game play outputs to a remote server 104 that implements a video game 110.

[0028] In some embodiments, the memory 306 stores a client device settings 334 for storing information associated with the client device 106 itself, including general device settings (e.g., service tier, device model, storage capacity, processing power, communication capabilities, etc.). The client device settings include game application(s) settings 338 for storing information associated with a player account of the game application(s) including one or more of account access information, in-game player preferences, game play history data, and information about other players, game controller configuration(s) 340 for storing information associated with the configuration of the client-side game module 328 of the game application(s) 328, local game session data 344 including information associated with online game sessions executed on the client device 106 in association with different game titles, e.g., one or more of latest state information, texture pages, vertex pages, constant pages, and frame data of one or more image frames associated with each of the online game sessions.

[0029] Each of the above elements can be stored in one or more of the aforementioned memory devices and corresponds to a set of instructions for performing the functions described above. The above modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures, modules, or data structures and thus various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 306 stores a subset of the modules and data structures described above. Additionally, memory 306 can store additional modules and data structures not described above.

[0030] Figure 4 An example configuration of virtual game controller 416 (one embodiment of virtual game controller 116) is illustrated to receive player input for controlling a video game, such as video game 110. Virtual game controller 416 is configured to display virtual buttons 118 on one or both of the right and left sides of touchscreen display 122 of client device 106 so that a player can manipulate the virtual buttons with their thumbs and fingers while they hold the device. Virtual buttons 118 are configured to "receive" player input (via touchscreen display 122) to manipulate the player's avatar 126 and objects within their avatar's control in video game 110 as if the player is using a physical game controller to provide player input for video game 110. While many virtual buttons 118 and combinations thereof can be implemented by virtual game controller 416 and configured to receive player input to control various aspects of video game 110 and the player's gaming experience, for ease of description, only some of the virtual buttons 118 and their functions are described in detail below, and the functions and operation of the other buttons 118 will be readily understood based on the following description.

[0031] In the context of a first-person shooter (FPS) video game, the left trigger button 402 is used for down-sight aiming (ADS) of a weapon 410 or object held by the player's avatar 126. The left stick button 404 is configured to move the player's avatar 126 within the video game 110. The right stick button 406 is configured to pan the viewpoint 412 of the player's avatar 126 within the video game 110, and the right trigger button 408 is configured to activate an object or fire a weapon 410 in the avatar's 126 control within the video game 110. In a physical game controller, the right stick provides velocity-based feedback to the video game 110 such that the farther the stick is moved away from its original center position, the farther and faster the avatar's view 412 of the video game 110 environment changes. The right stick button 406 of the virtual game controller 116 is configured to simulate the velocity-based functionality of the right stick on a physical gamepad by adjusting the avatar's view 412 based on how far the right stick button 406 is dragged from its original position by the player. This results in relatively small movements of the right stick button 406 being translated into large changes in the avatar's view 412, making precise movement of the avatar's view difficult.

[0032] With the described configuration, the player is generally required to manipulate certain combinations of the virtual buttons 118 simultaneously in order to accomplish certain movements and actions of the avatar 126. For example, to target and aim at an object within the video game 110, the player is generally required to hold down the left trigger button 402 to down-sight aim with a weapon 410 held by their avatar 126, while simultaneously manipulating the direction in which their avatar 126 is moving by manipulating the left stick button 404. It is also common for the player to need to manipulate the right stick button 406 and the right trigger button 408 simultaneously, for example, to change the avatar's view 412 while firing a weapon 410. However, when holding the device, the player generally only has their thumbs available to manipulate the virtual buttons 118, making simultaneous control of multiple virtual buttons 118 difficult.

[0033] Figure 5 An example configuration of a virtual controller 516 (another embodiment of the virtual game controller 116) of the client device 106 is illustrated in accordance with some embodiments. The virtual controller 516 is configured to display virtual buttons 118 on both the right and left sides of the touchscreen 122 of the client device 106, enabling the player to manipulate the buttons with only their thumbs while they hold the device. In an embodiment, a draggable aim and move button 502 is provided and displayed on the left side of the touchscreen, and is thus manipulatable by the player's left thumb. The aim and move button 502 is configured to be draggable within the area of the touchscreen 122. Dragging of the aim and move button 502 by the player causes the player's avatar 126 to move within the video game 110 while the player's avatar 126 aims at an object 510 within their control. Thus, the aim and move button 502 is a combination of the left trigger button 402 and the left stick button 404 of the virtual game controller 116. In an embodiment, the aim and move button 502 is displayed on the left side of the touchscreen 122, and is thus manipulatable by the player's left thumb while they hold the device. In an embodiment, the aim and move button 502 is displayed on the right side of the touchscreen 122, and is thus manipulatable by the player's right thumb while they hold the device. Figure 4The functions of the left trigger button 402 and the left stick button 404 of the illustrated virtual game controller 416 are combined. While the aim and move button 502 is depicted and described as being located on the left side of the touchscreen 122 of the client device 106, the button layout of the virtual controller 516 can be configured by the player to be displayed at any location on the touchscreen 122.

[0034] In embodiments, a draggable point and shoot button 504 is provided and displayed on the right side of the touchscreen 122 of the client device 106. The point and shoot button 504 is configured to be draggable within the area of the touchscreen 122. The point and shoot button 504 is further configured to activate the object 510, such as a rifle, held by the player's avatar 126 when the player initially contacts the button 504 and as long as the player is holding the point and shoot button 504. The point and shoot button 504 is further configured to adjust the direction in which the player's avatar is looking in response to the player dragging the point and shoot button 504 within the area of the touchscreen 122. Thus, the player is able to direct their avatar's 126 gaze while shooting, or otherwise activate the object in the avatar control by holding and dragging the point and shoot button 504 within the area of the touchscreen 122. The point and shoot button 504 of the virtual controller 516 essentially combines the functions of the right trigger button 408 and the right stick button 406 of the illustrated virtual game controller 416. Figure 4 The functions of the right trigger button 408 and the right stick button 406 of the illustrated virtual game controller 416. While the point and shoot button 504 is depicted and described as being located on the right side of the touchscreen 122 of the client device 106, the button layout of the virtual controller 516 can be configured by the player to be displayed at any location on the touchscreen 122.

[0035] In embodiments, the virtual controller 516 is configured with a defined area 506 associated with the right side of the touchscreen 122 (hereinafter referred to as the "view pan area 506") that allows the player to adjust the direction of the avatar view 512 of the video game 110 environment based on the player contacting the touchscreen 122 within the view pan area 506. The view pan area 506 is configured such that certain player inputs provided by touch contact to the area of the touchscreen display 122 associated with the view pan area 506 are used to move the avatar view 512 to the location of the player's contact with the touchscreen 122 within the view pan area 506 and continue to pan the avatar view 512 as the player moves the touch point within the view pan area 506. By using the view pan area 506, the player is able to precisely adjust the view 512 of the video game 110 environment of their avatar. To facilitate access by the player holding the client device 106, the view pan area 506 can be sized and positioned to be accessible by the player's thumb when the palm of the player's hand and the fingers of the same hand are used to hold the client device 106.

[0036] Figure 6 FIG. 13 illustrates a method 1300 for controlling gameplay of a video game 110 in a cloud-based gaming system 100 based on player input provided via a virtual game controller 116 displayed at a client device 106, in accordance with some embodiments. After a game session is initiated by a player, at block 1302, video game content and a virtual game controller 116 are displayed at the client device 106. As described above with reference to embodiments of the virtual game controller 116, the virtual game controller 116 can include, for example, one or more virtual buttons 118 including, at least, a sight and movement button 502 disposed on one side (e.g., the left side) of a touchscreen display 122 and a point and shoot button 504 disposed on an opposite side (e.g., the right side) of the touchscreen display 122. In embodiments, the virtual game controller 116 includes a view translation region 506 disposed on the right side of the touchscreen 122. Figure 5

[0037] Player input is received via the player's interaction with the virtual buttons 118 and the view translation region 506 of the virtual game controller 116 by the player contacting the associated regions of the touchscreen display 122, as represented by one or more of blocks 1304, 1310, and 1316. For example, the player can manipulate the sight and movement button 502 and the point and shoot button 504 by contacting the regions of the touchscreen display 122 associated with each of the buttons 502, 504 and simultaneously maintaining contact with the touchscreen display 122, moving the buttons 502, 504 within the regions of the touchscreen display 122 that are accessible by the player's thumbs while the player's palms and fingers are used to hold the client device 106. Similarly, the player can indicate a desired direction of gaze for the avatar of the video game 110 by contacting the portion of the touchscreen 122 associated with the view translation region 506 and simultaneously continuing to contact the touchscreen 122, moving the touch point within the view translation region 506.

[0038] ​The received player input is then converted into control signals that are transmitted for receipt by the game server 104 to be provided to the video game 110 as part of the player input stream 120 and then interpretable by the video game 110 as instructions to perform actions in the video game 110. For example, position information regarding player movement of a touch point associated with a virtual button 118 of the virtual game controller 116 is converted into control signals provided to the video game 110 to be interpreted as instructions for effecting certain activities in the video game 110. In embodiments, player input received via player manipulation of the aim and move button 502 is converted at block 606 into control signals interpretable by the video game 110 as instructions for moving a player’s avatar 126 within the video game 110 environment based on the direction and distance the player moves the aim and move button 502. At block 608, player input received via the aim and move button 502 is also converted into control signals interpretable by the video game as instructions for aiming at an object within the avatar control.

[0039] In embodiments, player input received via player manipulation of the point and shoot button 504 is converted at block 612 into control signals interpretable by the video game 110 as instructions for altering the direction in which the avatar 126 is looking, as if the avatar were moving its head or eyes to view different parts of the video game 110 environment based on the direction and distance the player moves the point and shoot button 504. At block 614, player input received via the point and shoot button 504 is also converted into control signals interpretable by the video game 110 as instructions for activating an object or firing a weapon within the avatar control and continuing to do so until the player releases the point and shoot button 504.

[0040] In embodiments, player input received via player contact within the view pan area 506 is converted at block 618 into control signals interpretable by the video game 110 as instructions for altering the direction in which the avatar 126 is looking based on the location within the view pan area 506 where contact was initiated and the location where contact was broken.

[0041] At block 620, the control signals generated based on player input received through the virtual game controller 116 are provided to the game server 104 via the player input stream 120. At block 622, the video game 110 interprets the provided control signals. At block 624, the game server 104 then renders video content and provides the video content for display at the client device 106 based on the implemented control signals as controlled by the video game 110.

[0042] Figure 7 is a flowchart illustrating a method for providing a player input stream to a game server for controlling a video game, in accordance with an embodiment Figure 6The method 700 of converting player input received at the view panning area into control signals to be interpreted by the video game 110 to adjust the gaze of the player's avatar 126 is illustrated by the flowchart of FIG. 18. The view panning area 506 is configured to mimic the functionality of the right stick of a physical game controller such that fast movement of the touch point within the view panning area 506 results in a control signal that can be interpreted by the video game 110 as if the right stick of the physical controller was moved to a position far from the default centered position. The action generated within the video game 110 is that the gaze of the avatar pans quickly in the direction of the movement made by the player within the view panning area 506. Similarly, slow movement of the touch point within the view panning area 506 results in a control signal that can be interpreted by the video game 110 as if the right stick of the physical controller was moved to a position close to the default centered position and, therefore, the gaze of the avatar pans slowly in the direction of the movement made by the player within the view panning area 506.

[0043] The generation of the control signal that can be interpreted by the video game 110 as a movement of the right stick of a physical game controller is achieved by using a heuristic function to convert the linear velocity of the movement of the touch point within the view panning area 506 into a non-linear signal, for example, into a non-linear signal that closely matches the signal that the video game 110 would receive from the right stick of a physical game controller. Thus, at block 702, player input in the form of a linear velocity is received at the view panning area 506 by the player moving the touch point within the view panning area 506 at a velocity and in a desired direction. At block 704, a heuristic function is applied to the velocity input received at the view panning area 506 in order to convert the linear velocity input into a non-linear signal. If the converted signal is less than a defined threshold, as shown at block 706, the converted signal is ignored (i.e., no action is taken on it) by the virtual game controller 116 and no control signal is sent to the video game 110 at block 708. This also enables improved haptics and control such that only certain actions are interpreted as triggers for generating control signals. Thus, operational errors can also be reduced. If the converted signal is equal to or greater than the threshold, the converted signal is normalized and bounded at block 710 to a value within the range of signal values (i.e., signal range) that would be output by the right stick of a physical game controller. At block 712, the normalized and bounded control signal is then sent to the video game 110 to be interpreted as an instruction for moving the view of the player's avatar 126.

[0044] In some embodiments, certain aspects of the techniques described above can implemented by one or more processors of a processing system executing software. The software includes one or more sets of instructions stored or otherwise tangibly embodied on a non-transitory computer- readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium can include, for example, a magnetic or optical disk storage such as a compact disk (CD) or digital versatile disk (DVD), a solid state memory like flash memory, a cache, random access memory (RAM) or other non-volatile storage device, etc. Executable instructions can be, for example, assemblies, instructions, files, code, etc., generated by a compiler or executed by an interpreter. The above-described apparatuses and methods can be implemented using software operating on a computing device, such as a server, a personal computer, a laptop computer, a tablet computer, a personal digital assistant (PDA), a mobile telephone, etc.

[0045] A computer-readable storage medium can include any storage medium, or combination of storage media, accessible by a computer system during use to provide instructions and / or data to the computer system. Such a storage medium can include, for example, an optical medium (e.g., a compact disc (CD), a digital versatile disc (DVD), etc.), a magnetic medium (e.g., a floppy disk or a hard drive), a semiconductor medium (e.g., a solid state drive (SSD) or other solid state memory), etc. The computer-readable storage medium can be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a hard drive), removably attached to the computing system (e.g., an optical disk or flash drive), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0046] Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device can not be required, and that one or more further activities or elements can be performed or included in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to particular embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0047] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems and any feature(s) that can cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Furthermore, the particular embodiments disclosed above are illustrative only as to a possible implementation and are not intended to limit the claimed subject matter in any way. Rather, the claimed subject matter encompasses all techniques, solely inherent in the claimed subject matter and all equivalents thereof. Therefore, it is intended that the disclosed subject matter not be limited to any specific embodiments disclosed in the above description except insofar as restricted by the following claims.

Claims

1. A method for controlling gameplay of a video game, comprising: displaying a virtual game controller on a touchscreen display of a mobile device, the virtual game controller comprising: a first draggable virtual button configured to simultaneously control movement of an avatar representing a player within a video game executed by a remote game server and aim of an object by the avatar within the video game executed by the remote game server; and a second draggable virtual button comprising a touch point and configured to simultaneously control a gaze of the avatar within the video game executed by the remote game server and activation of the object within the video game executed by the remote game server; converting an initial contact on the touchscreen display with the touch point of the second draggable virtual button to a control signal comprising at least an instruction to be interpreted by the video game for activating the object in the video game; and transmitting the control signal from the mobile device for receipt by the remote game server.

2. The method of claim 1, wherein, the first draggable virtual button is configured to simultaneously control movement of the avatar within the video game and aim of the object by the avatar based on the player dragging a touch point associated with the first draggable virtual button on the touchscreen display.

3. The method of claim 2, further comprising: converting location information associated with the player dragging the touch point to an instruction to be interpreted by the video game that the avatar is aiming at a location in the video game where the object is located, wherein the control signal further comprises the instruction to be interpreted by the video game that the avatar is aiming at the location in the video game where the object is located.

4. The method of claim 2, further comprising: converting location information associated with the player dragging the touch point to an instruction to be interpreted by the video game that the avatar is moving within the video game, wherein the control signal further comprises the instruction to be interpreted by the video game that the avatar is moving within the video game.

5. The method of claim 1, wherein, the second draggable virtual button is configured to simultaneously provide control of the gaze of the avatar and activation of the object based on a player contacting and dragging the touch point on the touchscreen display associated with the second draggable button.

6. The method of claim 5, further comprising: converting location information associated with the player dragging the touch point associated with the second draggable virtual button on the touchscreen display to an instruction to be interpreted by the video game for moving the gaze of the avatar within the video game, wherein the control signal further comprises the instruction to be interpreted by the video game for moving the gaze of the avatar within the video game.

7. A device for controlling gameplay of a video game, comprising: a touch screen display configured to display content from a game server and receive player input through contact with the touch screen display; a network interface configured to communicate with the game server executing a video game, the video game player represented by an avatar; and at least one processing unit configured to: display a virtual controller, the virtual controller comprising: a first draggable virtual button configured to simultaneously control movement of the avatar in the video game and aiming of an object by the avatar in the video game; and a second draggable virtual button comprising a touch point and configured to simultaneously control a gaze of the avatar in the video game and activation of the object in the video game; and convert initial contact on the touch screen display with the touch point of the second draggable virtual button to a control signal, the control signal comprising at least an instruction to be interpreted by the video game as used to activate the object in the video game.

8. The apparatus of claim 7, wherein, the first draggable virtual button is configured to simultaneously provide control of movement of the avatar within the video game and aiming of the object by the avatar based on the player dragging a touch point on the touch screen display associated with the first draggable virtual button on the touch screen display.

9. The apparatus of claim 8, wherein, the at least one processing unit is further configured to: convert location information associated with the player dragging the touch point on the touch screen display to an instruction to be interpreted by the video game as a location in the video game at which the avatar is aiming the object, wherein the control signal further comprises the instruction to be interpreted by the video game as the location in the video game at which the avatar is aiming the object.

10. The apparatus of claim 8, wherein, the at least one processing unit is further configured to: convert location information associated with the player dragging the touch point on the touch screen display associated with the first draggable virtual button on the touch screen display to an instruction to be interpreted by the video game as movement of the avatar within the video game, wherein the control signal further comprises the instruction to be interpreted by the video game as movement of the avatar within the video game.

11. The apparatus of claim 7, wherein, the second draggable virtual button is configured to control the gaze of the avatar and activation of the object in the video game based on player contact and dragging the touch point associated with the second draggable button on the touch screen display.

12. The apparatus of claim 11, wherein, the at least one processing unit is further configured to: converting location information associated with the touch point associated with the player dragging the second draggable virtual button on the touch screen display into an instruction to be interpreted by the video game for moving the gaze of the avatar within the video game, wherein the control signal further comprises the instruction to be interpreted by the video game for moving the gaze of the avatar within the video game.

Images