Methods of haptic response and interaction

Abstract

A method and system for providing responses to a player during gameplay of a video game includes detecting an interactive task within a game scenario of the video game that requires an action from the player. In response to detecting the interactive task, identifying a profile of a player playing the video game. A haptic response is provided to the player according to haptic settings defined for the player's player profile. The haptic response is provided to the player via an input device used by the player to provide game input to the video game. The provided haptic response is specific to the player and is provided to guide the player through the interactive task within the game scenario of the video game.

Description

[Technical Field]

[0001] The present disclosure relates to providing notifications to a user during gameplay and alerting the user to portions of a game scenario, and more particularly to providing notifications to a user to perform interactive tasks via a controller used to provide game input during gameplay of a video game, wherein the notifications are customized for the user. [Background technology]

[0002] In recent years, interactive applications such as video games, virtual life simulations, educational applications, and music applications have become increasingly popular. The majority of video games are streaming three-dimensional (3D) video games (also known as massively multiplayer online games—MMOGs). MMOGs are accessed simultaneously by a large number of users connected through a network such as the Internet. Users of MMO applications assume the role of a virtual character or game icon and control the actions of the virtual character or game icon using input provided through input devices such as a keyboard, game controller, or touchscreen. The input enables users to navigate the virtual space, interact with the gaming environment and virtual environment according to the game rules and goals specified in the video game, and interact with other users' virtual characters / game icons. Users may provide input cooperatively with other users (e.g., as part of a team) to achieve a common goal, or they may compete with other users (e.g., competitively) to progress through the video game.

[0003] Video games may be played using any computing device, including desktop computers, laptop computers, mobile computing devices, etc., and may be played using input provided using input devices, such as game controllers associated with the computing device, keyboards, control interfaces provided on a display screen, etc. One of the primary goals of video game applications is to maximize a user's immersion in the video game. However, different users' play styles may vary the user's immersion, resulting in an unsatisfying experience for the user. For example, some users may be slow to respond to game prompts, miss certain game prompts, be unable to interact with game assets that are shown to be beneficial to the user during gameplay, require more assistance during gameplay, become easily distracted or overly focused on certain parts of the game, and miss other parts of the game that may be meaningful or beneficial to the user.

[0004] It is against this background that the embodiments of the present disclosure have been made. Summary of the Invention

[0005] Embodiments of the present disclosure relate to systems and methods for providing a haptic response (HR) to a user playing a video game. In one embodiment, the haptic response is customized for the user. For example, a play style can be studied for different types of gaming actions and contexts, and based on that analysis, information about the haptic response provided to the user is stored in the user's profile. Over time, based on the user's gaming activity, a learning algorithm can be used to update information about the haptic response and how the haptic response was provided to the user. The updated information is also updated in the user's profile. Haptic responses are provided via peripheral devices, such as gaming controllers, to convey information about gaming interactivity. The conveyed information can be context-specific, based on events occurring in the game, etc. In some cases, the information conveyed by the haptic response is intended to inform the user of specific actions required of the user during gameplay. As an example, the haptic response can include a vibration cue to the controller. A vibration cue can be provided to the controller such that a degree of vibration occurs in different or specific parts of the controller. If the information conveyed to the user is to move the controller, or a game object controlled by the user, to the right, more vibration can be provided to the right handle of the controller. In some cases, the vibration provided can be configured to move or shift from one side of the controller to another. As described above, these types of tailored haptic responses can be customized for a user based on play style or learned behavior. For example, if a user needs more assistance in determining where to move the controller, or where to move a game object controlled by the controller, or where to move an input button on the controller, a particular component of the controller can be used to provide a vibration cue. In certain examples, the vibration cue can be provided louder or for a longer period of time. If the user needs less assistance, the vibration cue can be automatically adjusted lower. A particular component of the controller can include haptic elements integrated within the controller, which haptic elements are associated with each of the buttons / joysticks of the controller, and / or associated with the interactive screen of the controller, and / or associated with the controller as a whole.

[0006] In one embodiment, game input provided by a user is used to adjust the game state of the video game and generate gameplay data. The gameplay data includes telemetry data that can be analyzed to determine the speed of gameplay, actions taken by the user, progress made in gameplay in response to actions, the time taken to perform each action, etc., from which the user's play style or other gameplay characteristics can be determined. In addition to game style and gameplay characteristics, the system may also identify interactive tasks within the game scenario of the video game that the user was unable to interact with or that require action from the user. Interactive tasks that the user missed or that require action can be identified by correlating the content of the game scenario with the context of actions taken by the user in the game scenario. In response to detecting an interactive task in the game scenario that requires user interaction, the system provides a haptic response to the user to make the user aware of the existence of an interactive task in the game scenario and, if necessary, guide the user to take action toward achieving the goal of the interactive task. The haptic response provided to the user is customized for the user. In one embodiment, the system learns the user's gaming style and dynamically generates haptic settings that can be used when providing haptic responses. In an alternative implementation, the haptic responses can be customized according to input provided by the user to define the haptic settings, which match feedback requirements. In another implementation, an initial customization of the haptic settings can be performed by the system, and input from the user can be used to perform additional customization. The haptic responses provide the user with sufficient cues to detect and interact with interactive tasks in a game scenario.

[0007] In one embodiment, a method for providing a haptic response to a user during gameplay of a video game is disclosed. The method includes detecting an interactive task within a game scenario of the video game that requires an action from a user. The interactive task is identified by correlating content of the game scenario with a context of actions taken by the user in the game scenario. A user profile of a user playing the video game is identified. A haptic response is generated for the user according to haptic settings defined for the user's user profile. The haptic response is provided to the user via a controller used to provide game input to the video game. The provided haptic response is user-specific and is provided to guide the user through the interactive task within the game scenario of the video game.

[0008] In one embodiment, the haptic response continues until an action from the user is detected in the interactive task.

[0009] In one embodiment, the haptic response includes deactivating a control on the controller to prevent the user from progressing through the video game until an action from the user is detected on the interactive task.

[0010] In one embodiment, the functionality of the controller is used to generate the haptic response.

[0011] In one embodiment, the haptic response includes spatial cues to orient the user to an interactive task in a game scenario, the spatial cues being provided using a three-dimensional representation of the game scenario of the video game.

[0012] In one embodiment, the haptic response is triggered according to haptic settings that are customized for the user.

[0013] In one embodiment, the haptic settings are predefined by the user.

[0014] In one embodiment, haptic settings are dynamically defined based on a user's playstyle. The playstyle is determined using a haptic learning engine that uses machine learning logic. The haptic learning engine is dynamically trained by the user's game input and game progression made by the user in the video game. The haptic settings are dynamically adjusted from the training and applied to the controller when a haptic response is triggered. The dynamic adjustment of the haptic settings is updated to the user's user profile.

[0015] In one embodiment, game progression is determined using telemetry data collected from a user's gameplay of a video game, which is analyzed to extract certain features indicative of the user's playing style or game progression of the video game.

[0016] In one embodiment, the action requested by the user is a movement in a particular direction, and the haptic response provided during gameplay includes a directional cue indicating the direction the user is moving or needs to move in relation to the interactive task.

[0017] In one embodiment, the controller has a plurality of tactile elements, and the directional cue provided in the tactile response includes sequentially activating the plurality of elements such that the tactile response enables movement from one tactile element to a subsequent tactile element of the plurality of tactile elements in a direction specified by the directional cue.

[0018] In one embodiment, the haptic response is defined to provide a change in the feedback provided to the user, the change in feedback being dynamically controlled, indicative of different actions taken by the user in the video game, and intuitive to the user.

[0019] In one embodiment, the haptic response is configured to change over time based on the content of the game scenario of the video game or game input provided by the user.

[0020] In one embodiment, the user's haptic response is generated to correlate the content of the game scenario with the context of the actions taken by the user in the game scenario.

[0021] In one embodiment, the interactive task is to interact with the game assets or avatar of another user playing a video game.

[0022] In another embodiment, a method for providing a haptic response to a user during gameplay of a video game is disclosed. The method includes identifying an interactive task within a game scenario of the video game that requires an action from a user. The interactive task is identified by correlating content of the game scenario with a context of an action taken by the user in the game scenario. A user profile of a user playing the video game is identified. A haptic response is generated for provision to the user according to haptic settings defined for the user's user profile. The haptic response is user-specific and is used to guide the user through the interactive task in the game scenario of the video game.

[0023] In some implementations, tactile responses are provided to the user via the controller used to provide game input to the video game.

[0024] In some implementations, the haptic response is provided to the user via a head-mounted display used to view gameplay of the video game.

[0025] In another embodiment, a method for providing a user with a haptic response during gameplay of a video game is disclosed. The method includes examining gaming actions performed by a user during gameplay of the video game. The examining includes examining a context of the gaming action in relation to content of a game scenario occurring in the video game. Information regarding a haptic response to be provided to the user is identified based on the examination of the gaming actions. The information is used to define a haptic setting that is specific to the user. The defined haptic setting is stored in a user profile of the user. The information regarding the haptic response is dynamically updated based on the user's gaming actions collected during gameplay. The updating of the information causes a corresponding update to the haptic setting defined in the user's user profile. The haptic response is generated for the user in response to detecting an interactive task within the game scenario of the video game that requires an action from the user. The interactive task is identified by correlating content of the game scenario with the context of the gaming action performed by the user in the game scenario. The haptic response is generated according to the haptic setting defined in the user's user profile and is generated to guide the user through the interactive task within the game scenario of the video game.

[0026] Other aspects and advantages of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the disclosure.

[0027] The present disclosure may be best understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0028] [Figure 1] 1 illustrates a simplified concept of a gaming cloud site for collecting user game inputs that are used to determine haptic settings that should be used to provide notifications, according to one embodiment of the present disclosure. [Figure 2] FIG. 1 illustrates a simplified block diagram of a controller haptic profiler used to define haptic settings used to provide a user with haptic responses during gameplay, according to one embodiment of the present disclosure. [Figure 3] 1 illustrates a simplified block diagram of different modules within a controller haptic profiler used to generate a user's haptic responses during gameplay of a video game, according to one embodiment of the present disclosure. [Figure 4] FIG. 1 illustrates a simplified block diagram of different modules within a haptic response customization engine used to generate customized haptic settings for haptic responses provided to a user during gameplay of a video game, according to one embodiment of the present disclosure. [Figure 5A] 1 illustrates a diagram of a game scenario of a video game currently being played by a user, according to one embodiment of the present disclosure. [Figure 5B] 1 shows a simplified perspective view of a haptic response provided to a user using a game controller, according to one embodiment of the present disclosure. [Figure 5C] 1 shows a simplified perspective view of a haptic response provided to a user using a game controller, according to one embodiment of the present disclosure. [Figure 5D] 1 shows a simplified perspective view of a haptic response provided to a user using a game controller, according to one embodiment of the present disclosure. [Figure 6] 1 illustrates a flow diagram of various operations of a method used in generating haptic responses to a user during gameplay of a video game, according to one embodiment of the present disclosure. [Figure 7] 1 illustrates an example implementation of an information service provider architecture, according to one embodiment of the present disclosure. [Figure 8] FIG. 1 illustrates a simplified block diagram of a game cloud server used to define haptic settings used to provide haptic responses to a user via a controller, according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0029] In the following description, several specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure may be practiced without some or all of these specific details. In other instances, well-known process steps have not been described in detail in order to avoid obscuring the present disclosure.

[0030] Currently, a player plays a video game (simply referred to as a "game") by selecting a video game title from a game cloud server and providing game input using input devices, such as a controller, keyboard, mouse, or touchscreen, associated with a computing device, such as a mobile device or laptop device. A video game may be a streaming video game that receives game input and generates frames of game content that are streamed to the player's client device for rendering. Video games running on game cloud servers are capable of live streaming gameplay over a network, such as the Internet. A video game may be a single-user game or a massively multiplayer online game played by multiple players accessing the video game from a single geolocation or multiple geolocations.

[0031] Game inputs provided by a player are used to affect the game state of the video game and generate gameplay data. The player's game inputs correspond to activities performed by the player in the video game, and the activities affect the game state of the video game. The player's game inputs and the activities performed in the video game are part of telemetry data that provides information about the player's gameplay, and from the telemetry data, the player's game style, game progression, game ability, skill level, etc. can be easily inferred. The telemetry data captures characteristics of each game scenario of the video game, characteristics of each activity performed by the player, player attributes, etc., and can be processed to determine the overall game state of the game. Features of a game scenario may include details of game assets (e.g., static objects, moving objects, etc.) defined in the game scenario, where game assets include tasks, graphic objects, game characters, visual tasks, game objects, different locations in the game scenario, paths to one or more game assets, areas in which game assets are located, bushes, trees, front yards, back yards, streets, buildings, other game areas, other players, game movements, static objects, moving objects, non-player characters, player avatars, avatars representing spectators, interactive tasks requiring user interaction, tasks that do not involve user interaction, etc. Interactive tasks may include performing an action on one or more game assets. Some examples of actions that can be performed with game assets may include moving an object, shooting an object, shooting at a visual task, following a specific path, changing the path of a game object's movement, interacting with another player, building a game object, chopping down a tree, walking down a street, throwing a ball, etc. A game scenario may represent locations where game objects, or game assets, background objects, and / or foreground objects are located, which are part of the game scenario, etc., appear in one or more frames of the streamed game content, textual or graphical content currently present or occurring, tasks that need to be performed, representations of the results of action(s) taken on game objects, etc. Characteristics of each activity captured in the telemetry data include details about the type of activity the player attempted, the type of activity the player accomplished, the type of activity the player failed to accomplish, the type of activity the player did not attempt, the game assets assigned to each activity, etc. These characteristics are captured using the context of the game scenario accessed by the player during gameplay of the video game and the actions / game inputs provided by the player. Player attributes that can be determined from the telemetry data include play style (e.g., conservative player, risk taker, innovative moves, response speed, concentration level (e.g., overly involved or highly distracted, etc.), player type (e.g., beginner, expert, etc.), etc. Player attributes can be updated to the player's player profile. The game state of the video game identifies the overall state of the video game at a particular point in time and is affected by the complexity of the player's gameplay. Gameplay data is processed by the game logic of the video game to generate frames of content that are transferred to the player's client device for rendering. If the video game is an MMO game, game inputs from multiple players are used to affect the overall game state of the video game. Telemetry data is also used to identify the player's save data, which includes any game customization provided by the player for the video game.

[0032] Generally, most players who play video games are able to handle the speed of gameplay of the video game and can react with reaction times that correspond to the speed of gameplay of the video game. However, there may be certain players whose reaction times are not as sharp as other players. Slow reaction times may be due to these players being easily distracted or being unable to concentrate on gameplay, or may be due to slow responses. As a result, these players may miss certain interactive tasks (e.g., interacting with one or more visual cues) present in the game scenario of the game. The interactive tasks that the player may miss may be necessary by the player because these interactive tasks may assist the player in accumulating certain game points, rewards, tools, etc. that may be necessary to progress in the video game. For example, the player may fail to observe, pay attention to, or be unable to interact with a treasure chest that is present in a corner or on the extreme side of the game scenario. The treasure chest may contain a key to unlock a game level or a game tool that the player may need in subsequent gameplay. Missing such visual cues may prevent the player from significantly progressing through the game.

[0033] To address such issues, various embodiments of the present disclosure describe systems and methods for creating a sensory interface to provide haptic responses to a player playing a video game. The haptic responses are used to notify the player of the presence of such interactive tasks within the game scenario of the video game. The interactive tasks may identify game objects or game assets with which the player can interact in the game scenario. The haptic responses are provided to the player via the interface in addition to the normal game prompts provided by the game logic. The haptic responses are customized according to the player's play style and are provided to the player via one or more input devices used to provide game input to the video game. The customization may be based on input received from the player, from another player, or from a user associated with the player. For example, a player's play style may be examined for different types of gaming actions and contexts. Based on information gathered from analyzing the player's play style, appropriate haptic responses to be provided to the player are identified and stored in the player's profile. The analysis of play style and identification of appropriate haptic responses are performed by a learning algorithm associated with the game logic. As the player's play style improves over time, the learning algorithm detects the improvement and updates the player's profile. The haptic responses are provided to the player via a peripheral device (e.g., a gaming controller, or simply referred to as a "controller") to direct, nudge, or guide the player through interactive tasks. Notifications of interactive tasks may be provided via functionality available within the peripheral device.

[0034] In some implementations, the input device through which the notification is provided to the player may be a game controller used by the player to provide game input. The notification may use functionality of the game controller. For example, the notification may be provided using buttons on the controller, a touch screen, tactile elements, etc. In some implementations, the notification may provide directional cues to guide the player through an interactive task.

[0035] The haptic responses convey information to the player regarding the interactive task of the game. The conveyed information can be context-specific (e.g., based on events occurring in the video game), time-specific (e.g., a specific time of day or the expiration of a predetermined period of time), player-specific, or any combination thereof. In some cases, the haptic responses convey information to inform the player of specific actions required of the player during gameplay.

[0036] For example, the haptic response may be provided as a vibration cue to the controller. The vibration cue may be provided to the controller so that a degree of vibration occurs in different or specific parts of the controller. For example, if the information to be conveyed to the player is to move the controller, or to direct the player's attention, or to move a game object controlled by the player to the right, a vibration cue may be provided to vibrate the right handle of the controller. In some cases, the vibration cue may be configured to move or shift the vibration provided to the controller from one side of the controller to another. These types of tailored haptic responses can be customized for a player based on the player's play style or learned behavior. For example, if a player needs more assistance in determining which button to press, or which direction a game object should move or turn, or which direction to direct the player's attention on the screen, certain components of the controller can be used to provide vibration cues. In some specific examples, the vibrations may be provided louder or for a longer period of time. If a learning algorithm learns that the player is more comfortable playing the game and does not need additional assistance, the vibration cues may be automatically adjusted lower. Certain components of the controller that can be used to provide haptic responses may include haptic elements built into the controller. The haptic elements may be individual elements associated with different controller buttons / controls / joysticks or an array of elements associated with the controller's interactive screen. In addition to vibration cues, haptic responses may be provided as audio cues, text cues, visual cues, etc.

[0037] With a general understanding of embodiments of the present invention now in place, reference will now be made to the various drawings to describe illustrative details of various implementations.

[0038] FIG. 1 provides an overview of a gaming cloud site 10 used to access games for gameplay. The gaming cloud site 10 includes multiple client devices 100 (100-1, 100-2, 100-3, ... 100-n), which may be distributed across a single geolocation or across different geolocations and communicatively connected to a gaming cloud system 300 via a network 200. The gaming cloud system (GCS) 300 is configured to host multiple games and other interactive applications, such as social media applications and content provider applications (e.g., music streaming applications, streaming video applications, etc.). The GCS 300 may be accessed from a single geolocation or multiple geolocations. The client devices 100 may be any type of client computing device having a processor, memory, and communication capabilities to access the network 200, such as a LAN, wired, wireless, or 4G / 5G, and may be portable or non-portable. Client device 100 may run an operating system and include a network interface for accessing network 200, or may be a thin client with a network interface for communicating with GCS 300 over network 200, with GCS 300 providing the computing functionality. For example, client device 100 may be a smartphone, mobile device, tablet computer, desktop computer, personal computer, wearable device, or hybrid device, or other digital device including a monitor or touchscreen with a portable form factor.

[0039] A client device 100 with 5G communication capabilities may include a mobile device or any other computing device capable of connecting to a 5G network. In one embodiment, the 5G network is a digital cellular network with a coverage area divided into multiple "cells" (i.e., small geographic areas). Analog data generated by the mobile device is digitized and transmitted as radio waves to local antennas within the cell using frequency channels that can be reused by geographically separated cells. The local antennas are connected to the Internet and telephone networks by high-bandwidth optical fiber or other similar wireless communications. Because the 5G network uses high-frequency radio waves for communication, it is capable of transmitting data at higher data rates, resulting in lower network latency.

[0040] A player may access video games available on GCS10 using a user account. In response to a player's request for access to a game to play the game, the player's user account is verified against user accounts 304 maintained in user data store 305. The request is verified against game data store 306 to determine whether the player is eligible to access and play the video game before providing access to the video game. Verification is performed by identifying all game titles available on game cloud system 300 that the player is eligible to view or play and verifying the game title included in the player's request against the identified game titles. Game data store 306 maintains a list of game titles that are or can be hosted on GCS10, and as new games are introduced, the game title, game code, and information about the new games are updated in game data store 306. It should be noted that while various embodiments are described with reference to video games (also referred to as "games"), embodiments may be extended to include any other interactive applications, such as streaming music applications, streaming video applications, etc.

[0041] After successful validation of the user and the request, the gaming cloud system 300 identifies a data center capable of hosting the game and sends a signal to the identified data center to load the game associated with the game title identified in the request. In some implementations, more than one data center may host the game or may be capable of hosting the game. In these implementations, the gaming cloud system 300 identifies a data center that is geographically close to the player's geolocation. The player's geolocation may be determined using a global positioning system (GPS) mechanism within the client device 100, the client's IP address, the client's ping information, the player's social interactions and other online interactions made via the client device 100, to name a few examples. Of course, it should be noted that the above-described methods for detecting a player's geolocation are provided as examples, and other types of mechanisms or tools may be used to determine a player's geolocation. Identifying a data center that is close to a player's geolocation reduces latency when transmitting game-related data between the player's client device 100 and a game running on the identified data center 301. A data center 301 may include multiple game servers 302, and a game server 302 is selected based on the resources available on the game server 302 to host the game. In some implementations, an instance of a game may run on one or more game servers 302, either within the identified data center 301 or across multiple data centers 301.

[0042] In some implementations, the identified data center 301 may not have the necessary resources (e.g., bandwidth, processing, etc.) to host the game. In such implementations, the game cloud system 300 may identify a second data center that is geographically closer to the player's geolocation and has the necessary resources, or may select one of the resources to host the game.

[0043] The gaming cloud system 300 loads games onto one or more game servers 302 in the identified data center(s) 301. The one or more game servers 302 include hardware / software resources to meet the game requirements. The game servers 302 may be any type of server computing device available to the GCS 300, including standalone servers, etc. Additionally, the game servers 302 may manage one or more virtual machines that support game processors that run instances of the player's game on the host.

[0044] In some implementations, one or more servers 302 may include multiple gaming consoles 303, and the gaming cloud system 300 may identify one or more gaming consoles within the identified one or more servers 302 and load the game. Each of the one or more gaming consoles may be an independent gaming console or may be a rack-mounted server or a blade server. And a blade server may include multiple server blades, each having the circuitry and resources necessary to instantiate, for example, a single instance of a game. Of course, the gaming consoles described above are exemplary and should not be considered limiting. Other types of gaming consoles, including other forms of blade servers, may also be involved in running the identified instances of the game. Once one or more gaming consoles or game servers are identified, the game's generic game-related code is loaded onto the identified gaming consoles or game servers and made available to players.

[0045] In other embodiments, a video game may be run locally on the client device 100, and metadata from the running video game may be transmitted over the network 200 to a game cloud server(s) 302 in an identified data center 301 of the GCS 300 to affect the game state and to share gameplay data with other players and spectators.

[0046] Game inputs that affect the game state of the game may be provided from input devices such as a mouse 112, a keyboard (not shown), or a control interface (e.g., a touchscreen, etc.) associated with the client device 100, or from a handheld controller (or simply referred to as a “controller”) 120 or any other peripheral device communicatively connected to the client device 100. Gameplay data collected from a player's gameplay session for the game is used to create a tactile learning model (i.e., an artificial intelligence (AI) model). Telemetry data collected during gameplay of the game is analyzed to extract information (e.g., traits, etc.) indicative of the player's play style, and the information extracted from the analysis is used to generate the tactile learning model. Additional information collected from the player's ongoing game input is used to further train the tactile learning model. The additional information is used to update the player's play style. The player's play style is used to determine whether the player is distracted or losing focus, having difficulty maintaining the pace of gameplay, or failing to interact with interactive tasks within the game scenario. These prevent the player from achieving the game objective of gameplay. The play style is used to determine haptic settings that can be applied to the peripheral devices (i.e., input devices) used by the player to interact with the game. The haptic settings are defined using the controller's capabilities and are customized for the player based on the player's play style. In some implementations, the haptic settings can be further customized using input from the player, or from another player, or from another user associated with the player (e.g., a parent or coach). The customized haptic settings are used to provide haptic responses to the player to alert the player that there are interactive tasks the player needs to interact with in the game scenario and guide the player through the interactive tasks. The haptic responses provide cues to the player that make the player aware of interactive tasks in the game scenario that the player can or should interact with, so that the player can interact with the interactive tasks and gain an interaction benefit.

[0047] The video games executed on the game cloud system 300 may be single-player games or massively multiplayer (MMO) games. A game engine (not shown) communicatively connected to the game logic of the video game may be used to provide the framework for the video game. Generally speaking, a game engine is a software layer that serves as the foundation for games, such as MMO games, and provides the framework used to develop the video game. The game engine abstracts the details of performing common related tasks required for all games (i.e., game engine tasks), while game developers provide the game logic that provides the details of how the game is played. The game engine framework includes multiple reusable components for handling several functional pieces (i.e., core functionality) for a video game that bring the video game to life. Basic core functionality handled by the game engine may include physics (e.g., gravity, friction-based object collision detection, collision response, trajectory, movement, etc.), graphics, sound, artificial intelligence, scripting, animation, networking, streaming, optimization, memory management, threading, localization support, etc. The reusable components include a processing engine used to handle the core functionality identified for the game.

[0048] During gameplay of a game, the game engine manages the game logic of the game and collects and transmits one or more player inputs received from one or more input devices associated with client device 100 to the game logic. Additionally, the game engine manages the allocation and synchronization of functional portions of the game engine to process, in an optimal manner, gameplay data generated by the game logic and generate frames of game content that are sent back to client device 100 for rendering. Various game engines are currently available to provide different core functionality, and an appropriate game engine may be selected based on the functionality specified for running the video game. Haptic responses generated by the haptic response notification engine are processed, encoded, and streamed to the player's client device by the game engine in response to detecting an interactive task in the game scenario of the video game that requires the player's attention (e.g., interaction).

[0049] Game inputs provided by a player during gameplay correspond to activities performed by the player in the video game, which activities affect the game state of the video game. The player's game inputs are part of the telemetry data used to generate gameplay data 308. The gameplay data 308 and the telemetry data are stored in gameplay data store 307. The game state of a video game identifies the overall state of the video game at a particular point in time and is affected by the complexity of the player's gameplay. If the video game is an MMO game, inputs from multiple players are used to affect the overall game state of the video game. The player's saved data includes any game customizations provided by the player for the video game.

[0050] The saved data also includes haptic settings customized for the player, and such data is saved in the player's profile. The haptic settings are defined using the capabilities of the input devices used by the player to provide game input to the video game. The capabilities may include buttons, joysticks, touchscreens, etc. on a handheld controller, or buttons or touchscreens on a head-mounted display, or controls on a peripheral device, etc. The haptic settings are used to inform the player of interactive tasks, or to notify them of events occurring in the real-world environment in which the player is present, or to assist the player in accomplishing a specific task within the game, or to alert them to perform a specific task in the real-world environment, or as a behavioral intervention. For example, a player may become so distracted during gameplay that they forget or miss an interactive task within the game scenario of a video game. Alternatively, a player may be so focused (i.e., so immersed) in the gameplay of a game that they forget to meet a specific appointment or perform a specific task in the real world. In another example, a player may wish to play for a predetermined period of time and want to be notified when the end of the period is approaching. Alternatively, a player may have difficulty dealing with the sensitivity of the input device's functionality. Assistance to the player may be provided to the player as a haptic response. The haptic response may be provided via an input device (e.g., a handheld controller used by the player to provide game input, or a wearable device such as a head-mounted display (HMD)). The haptic response may be used to provide directional or spatial cues that direct the player's attention to a particular portion of the game scenario rendered on the screen of the player's client device. In alternative embodiments where the player requires visual assistance, the haptic response may be in the form of visual cues (e.g., color coding and / or adjustments in color intensity) provided using different input functions (e.g., button presses, directional arrows, etc.). If the video game presents the player with reaction times that are not as sharp or fast as expected, the haptic response may be in the form of vibrations, pulsations, spins, jumps, magnetic action (i.e., a feeling of restricted movement), slowed response times to button presses, or swipe actions on a touchscreen, etc. The haptic response provides a sensory interface for the player using the functionality of an input device such as a handheld controller. The haptic response is tailored to the player and is intuitive, allowing the player to have a satisfying gameplay experience and not be overwhelmed.

[0051] In some implementations, the system used to provide the haptic response may be able to identify interactive tasks by analyzing the context of the game scenario and correlating that context with game actions taken by the player with game input provided via the input device. Based on the analysis, the system may detect interactive tasks requiring action from the player and responsively provide a haptic response that makes the player aware of the interactive task in the game scenario. The system may continue to provide the haptic response until the player visually sees and interacts with the task. In some implementations, the system may prevent the player from progressing through the game until the player interacts with the task by deactivating the functionality of the input device used to provide the game input. The haptic responses may be pre-programmed by the system based on the player's play style. The player's playing style may be obtained from a player profile maintained by the system. Alternatively or additionally, the haptic responses may be programmed by the player, by another player (e.g., an expert player, a coach, etc.), or by another user associated with the player (e.g., a parent of a child player). Different haptic responses may be programmed for different interactive tasks (e.g., events) that a player may encounter in-game or outside of the game (i.e., in the real-world environment). For example, a directional cue may indicate the direction the system expects the player to move to interact with a task or indicate the direction in which a task exists in the game scenario. In this example, a directional pattern may be rendered on a display screen associated with the client device. In an alternative example, the input device itself may vibrate on a side that correlates with the direction the player needs to move in the game scenario. Vibration of the right side or right handle of an input device, such as a handheld controller, may be to instruct the player to move to the right (e.g., take the right path at a fork), or may indicate the location of a task in the game scenario associated with the player, etc.

[0052] Traditionally, input devices such as handheld controllers have generally been configured to provide a player with a sense of the type of action that was occurring in a game scenario. Such feedback has been provided to give the player an immersive experience during gameplay of a video game. For example, a controller may be used to provide rumble feedback when a player is riding a buggy or carriage on an unpaved road. However, the feedback provided by these traditional input devices has not been configured to inform or guide the player to take specific actions in a game scenario.

[0053] Various implementations of the disclosure described herein enable a system to track interactive tasks available in a game scenario, track player interactions, determine which interactive tasks the player has interacted with and which interactive tasks (or simply referred to as "tasks") the player has not interacted with, identify tasks that require action from the player, and guide the player to those tasks. It is understood that not all tasks presented in a game scenario need to be interacted with. The system is configured to distinguish certain tasks that require player interaction and provide haptic responses accordingly. For example, an interactive task may be for a player to interact with a game asset or game object that holds a lock key or tool that may be needed to progress through the game, or may include a tool that can assist the player in gameplay. The system is capable of interacting with the game logic to identify such game assets / game objects and, accordingly, provide a haptic response to guide the player. In one embodiment, a game asset is a part of the game and is represented by a character, object, environment, vehicle, icon, three-dimensional model, sound effect, music, etc. A game object may be a game asset that can be interacted with. In some implementations of the present disclosure, game object and game asset may be used interchangeably to define a game character, object, icon, environment, vehicle, etc., on which a player can take action during gameplay.

[0054] A haptic response system (or simply referred to as a "response system") provides notifications (i.e., feedback) by contextually analyzing the content of a game scenario to identify specific game assets (e.g., interactive tasks) available in the game scenario that require the player's attention. The response system interacts with the game logic, performs contextual analysis to determine which game tasks present in the game scenario are necessary for the player's game progression, tracks the player's interactions with each of the game tasks, and notifies the player when the player overlooks or misses a specific game task necessary for the player's progression. The assistance provided by the response system helps the player achieve their goals in the game, making the game more interesting for the player. Increased player engagement can lead to increased player retention in the game, which can lead to increased revenue for the game developer or game publisher.

[0055] The response system may also be configured to provide reminders to the player of predefined or scheduled events or appointments outside the gaming environment or when the player becomes too engrossed in the video game. Alternatively, the response system may be configured to notify the player of events scheduled to occur in the game (i.e., upcoming events) based on the player's progress in the video game. The response system may also be configured to provide alarms when the player becomes too engrossed in the video game. The response system tracks the player's interactions, determines the amount of time the player is engrossed in game play, and provides an alarm when the player exceeds a predetermined gameplay period. The predetermined period may be set by the player, by another player, by another user, or by game logic. The response system may provide appointment notifications by interacting with a calendar application (such as a social media calendar, an email application, or another calendar application). Additionally, the response system may notify the player of interesting events or actions occurring in game play.

[0056] Additionally, notifications provided to a player are customized to the player's needs and playing style. The notifications may be haptic responses, which may include sound, color, and / or vibration. The intensity of the sound, color, or vibration may be configured to vary to provide directional cues to the player to direct the player to a specific area on a display screen associated with the client device. The haptic responses may be high-fidelity notifications customized for the player based on that player's profile. The player profile may indicate that the player needs visual assistance, or is slow to respond to game prompts (e.g., unable to keep up with the rate at which game prompts are provided in a video game), or unable to keep up with the pace of gameplay. The response system may provide notifications to the player based on the profile. The notifications are provided by the response system by correlating content on the display screen with the context of the on-screen actions. The notifications are provided according to haptic settings defined for the player. The haptic settings may be defined by the response system by learning the player's playing style or from input provided by the player or other users. The response system includes a haptic learning engine, which is a machine learning algorithm, to define the haptic settings. As the player continues to play the game, the haptic learning engine learns the user's performance in the game, identifies the player's playing style, and dynamically adjusts the player's haptic settings. The adjusted haptic settings may be updated to the player's profile and used when the player needs to be notified about interactive tasks.

[0057] In one embodiment, the response system uses telemetry data to determine game progress made by a player and the player's playing style. The telemetry data is processed to extract certain features indicative of the player's progress or lack of progress in the video game. The progress or lack of progress in the video game is determined based on rules defined for the video game. For example, the telemetry data captures features of each activity, such as activities attempted by the player, activities completed by the player, activities failed by the player, and activities not attempted by the player. These features may be used to determine the time taken by the player to complete each activity attempted by the player. The extracted features are sent to a tactile learning engine, which learns the player's playing style from the player's responses. As the player plays the video game, the tactile learning engine continues to learn from the player's responses and refine the player's playing style. Thus, the tactile learning engine can process telemetry data generated from multiple players' gameplay of the video game to determine each player's playing style. In some implementations, the features extracted from the multiple players' telemetry data may also be used to determine the player's playing style. For example, the player's playing style obtained from the tactile learning engine may be compared to the playing styles of other players to determine whether the player is a slow learner, has slow reaction times, is constantly distracted, etc., compared to the other players. By aggregating a player's telemetry data with that of other players, the haptic learning engine can define a set of haptic settings that can be customized or recommended for the player based on the player's playstyle. The player's haptic settings can be selected from predefined settings or can be dynamically defined based on the player's playstyle learned by the haptic learning engine. The haptic learning may be performed on two or more servers in the game cloud system. The telemetry data may be a collection of gameplay data collected from different gameplay sessions of the player and multiple other players playing a video game and processed by one or more servers on which the haptic learning engine is running. The haptic responses provided to the player are in accordance with the haptic settings defined for the player. In some implementations, the haptic responses are provided via the controller 120 used by the player to provide game input. In other implementations, the haptic responses may be provided via a wearable or peripheral device, such as a head-mounted display (not shown), smart glasses (not shown), or the like.

[0058] The characteristics identified in the telemetry data are used to define player attributes, which are updated to the player's profile stored in the user data store 305, and the player attributes are used to identify the player's gaming style.

[0059] 2 illustrates a simplified block diagram of a haptic profiler 400 that, in one embodiment, is used to process telemetry data generated for a video game from a player's gameplay to determine the player's play style. The play style is used to determine the player's haptic settings. The haptic profiler 400 includes multiple modules that assist in analyzing the telemetry data, determine the player's play style, and generate the player's haptic settings. Some exemplary modules may include a gameplay data analyzer module 401, an interactive task detection engine 402, a haptic learning engine 403, a haptic response notification engine 404, and a haptic response customization engine 405. Of course, the aforementioned modules are examples, and fewer or additional modules may be available to define haptic settings for each player playing a video game.

[0060] The haptic profiler 400 receives telemetry data 312 from the gameplay data store 307, which includes player-related data and game-related data. The player-related data is stored in the user data store 305, and the game-related data is stored in the game data store 306. The haptic profiler 400 uses each player's player-related data and the game-related data of the game as inputs and processes various data to determine each player's play style and generate a haptic configuration for each player. The telemetry data 312 includes raw game data of the video game. The raw game data captures the complexity of a player's gameplay, including game inputs provided by the player, game content provided by the game logic in response to the game inputs, game context for different points in gameplay, etc.

[0061] The gameplay data analyzer module 401 processes the telemetry data 312 to extract certain gameplay characteristics indicative of the progress made by the player in gameplay. For example, the certain characteristics may be used to determine whether the player is progressing or not progressing in the video game based on rules defined for the video game, the degree of progress made in the video game, the difficulty of certain portions of the video game, etc. Details regarding the progress made in the video game are used to determine player attributes, which are updated in the player's profile. For example, some of the player attributes determined from the certain characteristics may include the player's level of expertise in the video game (e.g., whether the player is an expert player, a good player, or a novice player), concentration level, playing style, etc. Such determinations may be made based on the time it takes the player to progress through different portions of the game, the number of attempts to acquire or beat game assets, the number of attempts to win game assets, the amount of game assets the player is able to interact with in different portions of the video game, etc. The player attributes may be used to determine the player's interest and / or comfort level in the video game. Specific features of the player's gameplay extracted from the telemetry data 312 are provided as inputs to the interactive task detection engine 402 and the haptic learning engine 403 for further processing.

[0062] In addition to player attributes, the telemetry data 312 may also be used by the gameplay data analyzer 401 to determine gameplay characteristics of the video game. The telemetry data 312 includes image data and gameplay data generated in response to player game inputs. The gameplay data analyzer 401 may perform a contextual analysis of the image data and gameplay data included in the video game telemetry data 312 to identify gameplay characteristics by correlating the content of different game scenarios of the video game with the context of actions occurring in each game scenario. For example, the gameplay characteristics may include the level of gameplay, the video game complexity in different portions of the video game, the game scenarios accessed by the player, the context of each game scenario, the game assets included in each of the different scenarios accessed by the player, game tasks available for interaction in each game scenario, game tasks considered important or considered to assist the player during gameplay, the game state of the video game, etc. The gameplay characteristics identified from the telemetry data are provided to the interactive task detection engine 402 for further processing. The player attributes and gameplay characteristics are also provided to the haptic learning engine 403 to generate and train a haptic learning model.

[0063] The interactive task detection engine 402 receives the gameplay features provided by the gameplay data analyzer 401 and processes the gameplay features to identify interactive tasks available in the game scenario during gameplay. The tasks may include interacting with game assets available in the game scenario of the video game. The game assets in the game scenario may include static game objects (e.g., trees, rocks, hills, mountains, signs, buildings, treasure chests, lockers, magic boxes, etc.) and dynamic game objects (e.g., shooting stars, airplanes or bombers in war games, race cars in racing games, buses, people exercising, jogging, or walking, opponents or teammates of a player in a basketball game, football game, or soccer game (i.e., opponents or teammates from an opposing team, or teammates who are part of the player's team), etc.). One or more game assets (either static game objects or dynamic game objects) may require the player's attention and / or an action to be taken by the player. These game assets may be necessary for the player to progress through the video game or may provide assistance to the player during gameplay. For example, an interactive task (i.e., a game asset) that the player must perform in a game scenario may be a magic chest or treasure chest that holds a key to unlock a room, a castle key, ammunition, a tool, a hint, etc., to assist the player in progressing through the video game. The player may miss this game asset due to the speed of gameplay or the location of the game asset relative to the interactive task in the game scenario. The interactive task detection engine 402 interacts with the game logic to track the player's interactions in the video game and identify game assets available in the game scenario, game assets with which the player has interacted, and game assets (treasure chests, safes, boxes, etc.) that hold tools / hints / keys for the player to use during gameplay. Using this information, the interactive task detection engine 402 can identify the specific game asset that holds the hint / tool / key that the player missed when interacting in the game scenario. The information identified by the interactive task detection engine 402 is provided as input to the haptic learning engine 403 .

[0064] In addition to identifying an interactive task associated with one or more game assets that the player was unable to perform in the game scenario, the interactive task detection engine 402 may also determine why the player was unable to perform the interactive task. The reason for the failure to interact with the interactive task may be due to the player's slow reaction time, the player being overwhelmed by the pace of gameplay (i.e., too many interactive tasks present in the game scenario relative to the player's interactions, the rate at which interactive tasks are presented, etc.), the player running out of time, the player becoming distracted or unable to concentrate, etc. The interactive task detection engine 402 is configured to analyze the player's gameplay data to determine the rate at which the player interacts with the video game (i.e., reaction time), the pace of gameplay (i.e., gameplay intensity), the rate at which interactive tasks are presented, the time available to the player, etc. From the analysis, the interactive task detection engine 402 may identify why the player was unable to interact with a particular game asset associated with the interactive task. Information from the analysis by the interactive task detection engine 402 is provided to the tactile learning engine 403.

[0065] The haptic learning engine 403 includes a machine learning algorithm that generates and trains a haptic learning model (not shown) using the gameplay characteristics and player attributes provided by the gameplay data analyzer 401 and the interactive task-related information provided by the interactive task detection engine 402. The player attributes identify the player's play style. The haptic learning engine 403 trains the haptic learning model using the player's telemetry data 312. The haptic learning model may also use telemetry data 312 of multiple other players playing the game to fine-tune the haptic learning model generated for the player. The fine-tuned haptic learning model is used to identify specific haptic configuration outputs appropriate to satisfy the player's gameplay objectives. The specific haptic configuration outputs are customized for the player's profile based on the player's play style.

[0066] The identified haptic settings output from the haptic learning engine 403 is provided as input to a haptic response notification engine 404. The haptic response notification engine 404 defines haptic settings for different features identified in the haptic setting output. The haptic settings are provided as input to a haptic response customization engine 405 to define and provide customized feedback to the player during gameplay. The customized feedback may be provided via a controller or a wearable device according to the player's haptic settings. Wearable devices include head-mounted displays, smart watches, smart glasses, and other peripheral devices used to provide player game input. The customized feedback is provided to the player to direct the player's attention to game assets that the player was unable to complete a task on. The feedback may continue until the player interacts with the game and completes the task. In some implementations, feedback may prevent a player from progressing through the game until the player interacts with the game asset and completes the task. In alternative implementations, the player may be provided with informational feedback indicating that the player was unable to interact with the game asset. The informational feedback may be in the form of highlighting or visually identifying the game asset, or in the form of textual or audio feedback. In some implementations, the feedback may include directional cues that guide the player to the game asset so that the player can perform the task.

[0067] FIG. 3 illustrates a simplified block diagram of different sub-modules within the various modules of the haptic profile shown in FIG. 2 , according to one embodiment. For example, the gameplay data analyzer 401 includes a player attribute extraction engine 401 a and a game progress detection engine 401 b. Similarly, the interactive task detection engine 402 includes a game scenario evaluation engine 412 for identifying tasks 412 a and 412 b attempted by the player. The haptic learning engine 403 includes multiple classifiers 403 a and a haptic learning model (artificial intelligence model) 403 b for identifying haptic setting outputs related to the player. The haptic response notification engine 404 includes multiple sub-modules, including a game asset / event-specific cue engine 404 a, a time-based cue engine 404 b, a direction-based cue engine 404 c, and a rating-based cue engine 404 d, to name a few. The various modules included within the haptic response customization engine 405 are described in detail with reference to FIG. 4.

[0068] The gameplay data analyzer 401 is configured to analyze the gameplay data contained in the telemetry data 312 to identify gameplay characteristics and player attributes. The telemetry data 312 is stored in a telemetry data store (not shown) and provided as input to the haptic profiler 400 for processing. The player attribute extraction engine 401 a is used to parse the gameplay data to identify and extract specific characteristics related to the player. Some of the player-related characteristics that may be extracted include the player's game input, the time it took the player to respond to game prompts, the game assets that the game input targets, the player's skill level, etc. The extracted player-specific features are used by the gameplay data analyzer 401 to define player attributes for the player (e.g., the player's skill level, the player's reaction time, concentration / distraction level, etc.). The player attributes defined from the telemetry data are used to dynamically update a user profile stored in the user data store 305 and are used as input to the haptic learning engine 403 to identify the player's haptic preference output. The telemetry data 312 can be an aggregate of multiple players' gameplay. Thus, the various modules of the haptic profiler 400 identify the attributes of the current player as well as the attributes of each of the other players and use the attributes of the other players to fine-tune some of the player's attributes.

[0069] The game progress detection engine 401b of the game play data analyzer 401 uses the player's gameplay characteristics and attributes and game details provided by the game logic to determine the game progress made by the player in the video game. In some embodiments, the player's attributes determined from a previous game play session may be dynamically updated using game play data from the current game play session. In some embodiments, the game progress detection engine 401b may also determine the game progress made by the player in relation to the game progress made by other players, taking into account the game play characteristics and attributes of other players playing the video game. The player's updated attributes are stored in the player's profile maintained in the user data store 305. The game progress made by the player is used by the tactile learning engine 403 to define the player's play style, which may be based on reaction times to game prompts, challenges overcome, game assets captured, game assets missed, tasks attempted, tasks not attempted, etc.

[0070] The game progression details and player gameplay data extracted by the game progression detection engine 401b are used by the interactive task detection engine 402 to determine specific interactive tasks missed by the player in a game scenario rendered on the display screen of the client device associated with the player. A game scenario evaluation engine 412 of the interactive task detection engine 402 evaluates the player's gameplay data and game progression data to identify specific interactive tasks that the player failed to interact with but that should be present in the game scenario of the video game. The game scenario evaluation engine 412 correlates the content of the game scenario currently rendering on the player's client device with the context of the actions taken by the player to determine interactive tasks available in the game scenario (i.e., actions on game assets), interactive tasks 412a attempted by the player, and interactive tasks 412b not attempted by the player. Interactive tasks may correspond to actions performed on game assets that are static in nature (i.e., fixed objects such as trees, mountains, houses, streets, etc.) or on game assets that are dynamic in nature (i.e., moving objects such as projectiles / missiles, projectile game objects, moving vehicles, etc.). The game scenario evaluation engine 412 is configured to use information obtained from the game logic to identify specific assets of the game assets that hold keys, objects, or hints that may be necessary to progress in the game and tasks that need to be performed by the player on the game objects. Multiple game assets may exist in a game scenario, and not all game assets need to interact with or hold keys / hints / tools necessary to progress in the game. For example, some game assets may be presented to the player for pure entertainment or to allow the player to earn additional points in the game, and therefore, these game assets may not be necessary to progress in the game. The game scenario evaluation engine 412 uses the context of the content presented in the game scenario and the actions taken by the player to identify all tasks 412b that the player did not attempt and excludes specific tasks from the unattempted tasks 412b to identify interactive tasks that are necessary to progress in the video game or that hold keys or hints.

[0071] The task information from the interactive task detection engine 402, along with the gameplay features and player attributes identified by the gameplay data analyzer 401, are provided as input to a haptic learning engine 403 for generating and training a haptic learning model. The haptic learning engine 403 is a machine learning algorithm that includes multiple classifiers and a haptic learning model, which is an artificial intelligence (AI) model generated by the machine learning algorithm. The haptic learning model is generated by the haptic profiler 400 and learns the player's play style. This learning occurs to determine whether the player needs assistance during gameplay and identify haptic configuration outputs that can be used to notify or provide assistance to the player, for example, to interact with game assets and progress through the game to achieve the player's game objectives. Attributes of the player and gameplay characteristics of the video game are used to define classifiers. Each classifier may be defined using one or more attributes of the player and / or one or more gameplay characteristics of the game. The classifiers are used to generate a tactile learning model 403b. The tactile learning model 403b is fine-tuned using additional gameplay characteristics of the game identified during a current gameplay session of the game. Additional fine-tuning of the tactile learning model 403b is performed using player attributes and gameplay characteristics of multiple other players who have played or are currently playing the game. The fine-tuning of the tactile learning model 403b may be performed dynamically. The features and attributes collected from the gameplay of multiple other players define the play styles of the other players. The play styles of the other players can be used to determine the current player's state during gameplay (e.g., whether the player is distracted, unable to keep up with the game speed, or having difficulty recognizing certain interactive tasks) based on attributes of the interactive tasks (e.g., the location of corresponding game assets in the game scenario, the display attributes of the game assets, and the amount of content present near the game assets in the game scenario) compared to the play style of the current player. A haptic learning model 403b can be generated for each player and fine-tuned using the features and attributes identified from the gameplay of the other players. The fine-tuned haptic learning model 403b can be used to identify haptic setting outputs that match the player's play style.

[0072] While various embodiments are broadly described with respect to providing a haptic response to a player to remind the player to interact with a particular interactive task to progress in a game or to notify the player of the presence of a game asset in a game scenario that requires action, embodiments may be extended to provide feedback or notification to the player about a particular event that is about to occur, should occur, or is currently occurring in the game, or to provide an alarm that notifies the player when a predetermined time period set aside for playing the game or a predetermined time period during which the player should not be disturbed is about to expire, or to provide a notification when a calendar event is due or an event in the real-world environment is scheduled to occur, etc. A haptic configuration output may be identified to provide reactive or proactive notifications to alert the player. For example, a haptic configuration output selected for a player may be a reactive response to notify the player that the player is selecting an incorrect move or an incorrect tool / weapon to perform a particular interactive task in a game scenario during gameplay. Alternatively, the haptic setting output may provide a proactive response to inform the player of particular challenges that lie ahead of them in the game scenario.

[0073] The haptic response output identified by the haptic learning engine 403 is provided as input to a haptic response notification engine 404 for further processing. The haptic response notification engine 404 processes the haptic response output to identify various types of notifications identified in the haptic setting output so that it can provide appropriate cues to customize the haptic response provided to the player. The haptic setting output identified for the current player playing the game can be event-specific, visual object- or game asset-specific, time-specific, direction-specific, rating-specific, etc. The haptic response notification engine 404 includes multiple sub-modules for processing the type of notification included in the haptic setting output so that it can tailor the haptic response provided to the player accordingly. For example, when the haptic configuration output includes an event-specific notification or a game asset-specific notification, by using the game asset / event-specific cue engine 404a to identify an appropriate cue that corresponds to the event-specific notification or the game asset-specific notification, the haptic response to the player may include appropriate customization to inform or guide the player. The customization may be to direct the player's attention to a particular event or a particular game asset. For example, if a player misses a particular game asset, fails to interact with a particular game asset, fails to make a right turn (i.e., a particular event), etc., the game asset / event-specific cue engine 404a identifies an event- or game asset-specific cue that can be used to provide an event- or game asset-specific notification to the player. The cues identified using the game asset / event-specific cue engine 404a may be spatial cues that use a three-dimensional representation of the game scenario to map game assets or events to locations within the game scenario. Spatial cues may be used to inform the player of the location of a game asset or a location where an event is occurring in the game (e.g., a location in the game scenario where the player failed to make a right turn).

[0074] The time-based cue engine 404b may be used to identify time cues that can be used to customize notifications to the player to inform the player whenever a time-related event or action is about to occur, is scheduled to occur, or is set to expire. For example, the player or another player (e.g., a coach, teacher, etc.), or another user associated with the player (e.g., a parent), may define a predetermined period of time during which the player should not be disturbed. During the predetermined period, the player may be involved in gameplay, interacting with another interactive application, or engaging in another activity. Based on the time-based event or action, the time-based cue engine 404b may identify specific time cues to be used to notify the player of a time-based event or action that has expired, is about to expire, or is about to expire. The time cues may be used to generate notifications to provide alarms, which may be audio feedback (e.g., feedback set to a particular melody), haptic feedback, visual feedback, etc.

[0075] The directional-based cue engine 404c may be used to process directional-based notifications included in the haptic configuration output to identify directional cues that can be used to provide a position-based notification to the player. The directional cues may be intended to guide the player to a particular portion of the screen rendering the game scenario of a game or to particular content in an interactive application. The directional cues may be provided using capabilities of the input device used by the player to provide game input. For example, the input device may be a handheld controller or a wearable device, such as a smart watch, glasses, etc. In some implementations, the directional cues may be used to provide notifications on the screen of the client device. In alternative implementations, directional cues may be used to provide tactile notifications on the input device itself. For example, when a player's attention needs to be directed to the right side of the screen or when the player needs to turn right at an intersection, directional cues may be used to provide vibration feedback on the right side of the handheld controller, or alternatively, visual feedback (e.g., an illuminated arrowhead configured to move from left to right) on the interactive touchscreen of the handheld controller. Additionally, the light intensity and / or size of the arrowhead may change to indicate to the player to move in the direction identified in the directional cue to interact with the interactive task. In one embodiment, directional cues may be used to manipulate functions of handheld controller 120 and provide direction-based notifications. In this implementation, directional cues may be used to sequentially activate multiple haptic elements included on controller 120 in the direction specified by the directional cues. The haptic elements may be used in conjunction with other components of controller 120 (e.g., accelerometers, magnetometers, gyroscopes, inertial measurement units (IMUs), other sensors, or combinations thereof) to provide haptic feedback, audio feedback, visual feedback, etc. to the player by activating specific modes of the haptic elements or other components.

[0076] The rating-based cue engine 404d may be used to process the haptic setting output to identify the type of notification that needs to be provided to the player and to adjust the level of feedback provided in the notification to the player. When the haptic response notification engine 404 requests the player to move in a particular direction, the rating-based cue engine 404d may be used to enhance the feedback provided to the player via various elements, components of the controller 120. The enhanced feedback may be provided by adjusting the settings of the haptic elements and / or other components of the controller 120. For example, the rating-based cue engine 404d may be used to increase the haptic response by increasing the revolutions per minute (rpm) of the vibration from a low rpm to a high rpm. The amount of RPM increase may be based on the player's profile and is specific to the player to ensure the player is able to recognize the feedback and react accordingly. Various cue engines may work together to identify cues to provide two or more types of notifications to the player. The type of cue, the intensity of the notification, and the length of the notification may be defined according to the player's profile or according to details provided by the player or another user. Furthermore, notifications continue to be provided to the player until the haptic profiler 400 detects that the player is performing an action or movement required by the game. The notifications may be to ensure the player responds in an appropriate manner when interacting in the game, to assist the player in progressing through the game, to notify the player of specific actions required of the player, or as a behavioral intervention tool to ensure the player's safety or advantage.

[0077] Each cue engine may be configured to increase the intensity of notifications to ensure that players do not ignore different types of notifications. In some implementations, the cue engine may be configured to prevent a player from progressing until the player completes an action related to a game asset or event associated with the notification. In such implementations, the cue engine may be configured to deactivate controls of an input device until the player takes a required action or behaves in a particular manner. Upon detecting a player action or behavior in response to an event, on a game asset, or during gameplay, the cue engine may reactivate controls of the input device, allowing the player to continue playing the game. Haptic settings are used to customize haptic responses transferred to an input device so that the input device can provide a haptic response to the player.

[0078] 4 illustrates an example of a haptic response customization engine 405 that, in one embodiment, can be used to define and / or adjust various settings for different controls of an input device used to provide a haptic response based on cues provided by various cue engines of haptic response notification engine 404. For example, based on cues provided by the cue engines, color / sound control engine 415 of haptic response customization engine 405 can be used to set colors for different controls of an input device, such as handheld controller (or simply referred to as "controller") 120. Color / sound control settings may be defined for each button (415a) or portion of buttons on the controller 120, for each action (415b) (e.g., correct and incorrect direction movement, correct and incorrect weapon selection, etc.), for any touchscreen interface available on the controller 120 (415c) (e.g., a first color setting to inform the player that a swipe action provided by the player is in the correct direction, a second color setting to inform the player that a swipe action provided by the player is in the wrong direction, etc.), and for each event (415d) (e.g., when a predetermined period of time is approaching the end, or when a calendar event is due or scheduled, etc.).

[0079] In one embodiment, in addition to defining visual and / or audio settings (i.e., color / volume settings) for different controls of an input device (e.g., controller 120), the haptic response customization engine 405 may be configured to provide haptic settings for different controls of the input device. A vibration control engine 406 of the haptic response customization engine 405 may be used to set the haptic settings for the different controls. In one embodiment, the vibration control engine 406 may be configured to define vibration settings for each button press of a controller (406a) or input on a touch interface (406b), or to define vibration pattern settings for each button or direction on a touch interface (406c), or to define vibration pattern settings for the input device itself (406d). For example, the vibration pattern settings may include color / sound / haptic feedback for a series of button presses in response to a series of inputs (e.g., swipe gestures) provided on a touchscreen. The vibration pattern may be configured to increase the intensity of the color, sound, or haptic feedback in response to a series of inputs progressing in a particular direction and decrease the intensity in the opposite direction. In another example, the vibration pattern may be configured on an input device such as the controller itself. For example, when the haptic profiler 400 determines that a player wants to turn right at an intersection, it may trigger a haptic response on the controller. When triggered, the haptic response may be customized to vibrate the right side or right handle of the controller (i.e., haptic feedback) to indicate to the player that they need to turn right. Furthermore, as the player approaches an intersection, the vibration may be set to increase, signaling to the player that they are approaching or passing through the intersection. In this example, the haptic response is event-based. For time-based notifications, a haptic response may be provided to the player, for example, by vibrating the entire controller. In addition to haptic feedback, additional feedback in the form of sound, color, text, etc. may also be provided on either the controller 120 or the display screen of the client device.

[0080] The haptic response customization engine 405 may also customize the jump settings 407 based on cues provided by different cue engines. For example, the jump settings 407 may be customized for a player by defining the length, height, speed, etc. of the jump performed when the player activates a jump control during gameplay. For example, one of the controls (e.g., a button) on the controller may be mapped to a jump action, and this button may be customized for the player according to the jump settings defined by the haptic response customization engine 405.

[0081] The haptic response customization engine 405 may provide additional customization when defining pulse settings 408, parent / other user settings 409, magnetic action settings 410, and / or spin settings 411. For example, pulse settings 408 may be customized to provide appropriate physical and visual feedback to the player via an input device. In some implementations, the input device may be a wearable controller, such as wrist gear, and the customized pulse settings may project a visual display onto a desired surface, such as the user's arm or palm, to provide the player with a sense of direction. The visual display may be a static visual or an interactive visual. The pulse settings may be customized for the player so that the player can touch and use the projected image, allowing the player to interact with the visual display. The pulse settings for the wearable controller may also include haptic settings to allow the player to experience physical feedback from interactions made with the projected image.

[0082] In one embodiment, the haptic response customization engine 405 may also define parent / other user settings 409, allowing a parent or other user (e.g., a coach, another player, a teacher, etc.) to provide customization for a player. The parent / other user settings 409 may specify the controls that can be customized. A parent or other user can customize the controls to provide time or access restrictions for a player according to the player's profile, define color / volume settings, vibration pattern settings, jump settings, pulse settings, etc.

[0083] In one embodiment, the haptic response customization engine 405 may define magnetic action settings 410 for an input device, such as the controller 120. The magnetic action settings 410 may define resistance settings for one or more controls of the input device (e.g., buttons on the controller 120) such that when the player presses a button, the player experiences a sensation of restricted movement (e.g., a drag). For example, in a driving game, the player is supposed to turn right at an intersection. However, the player may not have been paying attention and may instead start going straight. As the player approaches or passes through the intersection, the magnetic action settings defined for one or more controls used by the player (e.g., buttons or joysticks used to drive a vehicle) provide a force or resistance to warn the user to turn right and not go straight. The magnetic action settings 410 may be customized to activate a button and provide appropriate resistance when the player selects an incorrect direction. Similarly, when the player moves in the correct direction, the magnetic action settings 410 may ease the resistance, allowing the player to accelerate after moving in the correct orientation. In addition to the resistance provided by the magnetic action settings 410, directional haptics on the controller 120 may also be activated to provide the player with additional notification of the direction the player is traveling and the direction the player needs to move / progress in the game. The directional haptics on the controller are configured based on the direction-based cue engine 404c or the event-specific / game asset-specific cue engine 404a of the haptic response notification engine 404. For example, a directional haptic may provide vibrations on the controller, and the vibrations may be defined to start on the left side (e.g., left handle) and move toward the right side (e.g., right handle) to notify the player of the direction the player needs to move in the game. The directional haptics may also be configured to increase in intensity as the player approaches an intersection where the player needs to turn right. The increased intensity is to ensure that the player can feel the haptic feedback provided by the controller as the haptic feedback moves from left to right within the body of the controller. The controller includes multiple haptic elements, and haptic feedback is provided to the player by configuring the haptic elements of the controller to work together to allow the haptic feedback to progress from one haptic element to another in a desired direction and convey a directional cue. A magnetic action setting 410 can be defined for each button or control 410a and touch interface 410b of the input device. Thus, the magnetic action setting 410 of a button or touch interface can be activated when a game needs to notify the player to perform a specific action.

[0084] In some implementations, the rating of the haptic feedback may increase or decrease based on the rating cues provided by the cue engine. For example, the rating cues may be used to define the spin setting 411 of the haptic feedback provided via the input device. The spin setting 411, in one embodiment, defines the spin speed of the haptic feedback provided to the player. The spin speed may be set by defining a low revolutions per minute (rpm) and a high rpm at which the haptic feedback can spin. The low and high rpm defined for the spin speed may be player-specific and based on the player's profile. The player-specific setting is to ensure that the player is able to feel and distinguish between the haptic feedback spinning at low and high rpm.

[0085] Various types of customization of input device controls performed by the haptic response customization engine 405 are provided as examples, following the cues provided by the cue engine. Additional or fewer control customizations are also conceivable. Control customization is player-specific, customizing controls to provide event-specific notifications, game asset-specific notifications, directional notifications, rating-specific notifications, and / or time-specific notifications to the player. The haptic learning engine 403 uses the player's game input to learn the player's play style, compares the player's play style with other players' play styles, and compares the player's game input with the game logic of the game to determine the type and frequency of notifications that should be provided to the player, and identifies appropriate notification settings when customizing notifications to the player. Notification settings for customizing notifications can be dynamically determined during the player's gameplay, allowing appropriate notification settings to be applied to the input device controls during gameplay. Feedback provided via the input device controls can be haptic feedback that provides directional context for the game, and such feedback can be provided before any action or during an action performed by the player.

[0086] In some implementations, customization of notifications provided using input device controls may be part of the development tools used by the game logic of the game. In this implementation, notifications are not part of the game logic but are available to the game logic for application during gameplay. Customizable profiles for input devices (e.g., controllers) function as alarm or notification devices that provide assistance during gameplay by alerting players to ongoing or upcoming events, actions, or interactive tasks the player needs to perform in the game. Customization of input device profiles allows for setting alarm / notification date, time, and duration; setting feedback preferences (e.g., haptic, audio, text, color, etc.); setting color / sound / volume intensity for each button or control action; allowing other players (e.g., coaches, teachers, parents, etc.) to remotely access and control notifications provided by the input device; and setting different types and levels of haptic feedback (e.g., vibration, pulse, spin, jump, magnetic action, command lag, etc.). The lag command option allows the input device's controls to be configured to slow down response. This option may allow the player to provide actions at a speed that is comfortable for the player. Notifications provided by the input device may be in addition to prompts provided by the interactive application (e.g., normal game prompts provided by the game). Notifications may be provided to encourage or guide the player to perform a specific action in an interactive task (e.g., a visual game object), and may be configured to continue until an action from the player is detected in the interactive task. Input device controls may be customized to provide notifications that are specific to different events, different actions, different controls on the input device, different players, etc., and the customization may be done using pre-programmed settings or as specified by the player or other players / users.

[0087] A player may want to play a game, but may become frustrated when the player is unable to reach a particular goal. This may be because players have different play styles and may need more time to respond to prompts in the game to achieve the goal. To make the game more interesting for the player, the gameplay of the game may be customized to the player's play style. For example, if a player has a slow reaction time to game prompts, the lag command option may be used to customize the controls of the input device so that when the player uses the control, the response provided by the activated control is slower. In response to detecting a slow response, the game logic may automatically slow down the game speed. Customizations defined for input device controls, which may or may not include input from the player, may be intuitive so that when notifications are provided to the player, the player easily understands them. Customizations of input device profiles (e.g., controller profiles) may be made to accommodate players' different play styles, visuals, and / or other gaming needs. The ability to customize controls may allow for setting color schemes and feedback intensity (e.g., how strong or weak the haptic signal is, how bright or dull the colors are, volume settings, etc.).

[0088] In summary, the haptic learning engine 403 of the haptic profiler 400 is configured to learn from the player's game progression and identify cues as to whether the player is able or unable to understand the visual content provided in the game. Based on this learning, the haptic learning engine determines the type of haptic response that needs to be provided to the player. The haptic learning engine identifies haptic setting outputs that are used to identify the haptic settings needed to customize the notifications provided to the player and dynamically applies the haptic settings to an input device, such as a controller, head-mounted display, wearable device, or other peripheral device, as the player plays the game. As the player continues to play the game, the haptic learning engine 403 continues to learn from the player's performance in the game and dynamically adjusts the haptic settings applied to the input device. The haptic settings identified for a player may be based on in-game events / actions or out-of-game events / actions. In some implementations, the haptic settings may be identified based on presets defined in the player's profile. For example, the presets may indicate that a notification should trigger if the player engages in gameplay for longer than a predetermined period of time. In another example, a player may wish to play a game and not want to be interrupted by any distractions for a predetermined period of time. In both examples, the presets defined in the player's profile may be used to generate a notification to the player before or immediately after the predetermined period of time expires. The notification may be customized so as not to interrupt the player's gameplay, but may provide a gentle prompt or nudge using haptic or audio feedback, for example, to alert the user of the expiration of a time period. In addition to haptic or audio feedback, notifications may provide spatial cues of where the player should, should not, or should go in the game world as the player moves around the game.

[0089] In one embodiment, notifications may be configured to provide notifications based on the context of gameplay and detected or expected actions from the player. For example, notifications may be provided to indicate to the player that the player is selecting the correct or incorrect weapon, tool, or move in the game. Additionally, notifications may be configured to indicate how other players, particularly the player's friends or social contacts, navigated the game, interacted with particular game assets, or used weapons or tools to perform tasks. In some implementations, the haptic learning model 403b may be adjusted using gameplay characteristics and attributes of particular ones of the players (e.g., the player's friends or social contacts), such that the haptic settings used to customize feedback to the player may follow what has been learned from the gameplay of the player's friends or social contacts.

[0090] In some implementations, the haptic settings identified for a player allow the input device to be adjusted for varying feedback. For example, the spin setting of a controller can be adjusted from a low rpm (revolutions per minute) to a high rpm, with a low rpm being set to indicate an incorrect move (e.g., selecting the correct weapon or tool or an incorrect direction) and a high rpm being set to indicate a correct move. Furthermore, spin settings can be defined to behave differently for different actions or cues, allowing the spin setting to dynamically change from a first setting to a second setting based on changes in the player's actions detected in the game. Thus, the haptic setting output from the haptic learning engine can be used to set feedback to indicate different meanings to the player, and such feedback can be intuitive to the player. Intuitive and variable feedback to the player is provided by adjusting various elements of the controller (i.e., input device) used to provide haptic, visual, audio, and other feedback.

[0091] 5A illustrates, in one example implementation, a game scenario for a video game generated from game input provided by a player. The game scenario includes a vehicle traveling along path A, and various paths (e.g., path B and path C) branch off from path A that the player can choose to follow during game play. As the player travels along path A, tasks or game objects or paths to achieve goals may be identified along path C. As the player approaches a branch point on path C, the player may be provided with a notification instructing them to follow path C rather than continuing along path A. 5B-5D illustrate some of the setting options that may be used in some implementations to customize notifications using the controls on a handheld controller. As previously described, customization is performed using the haptic setting outputs identified by the haptic learning model 403b. Customization settings can be defined for the controller and for each control button or input interface of the controller 120. FIG. 5B illustrates one such option, option A, where in one embodiment, notification settings are defined for the controller. The notification settings are defined to enable a haptic response to be provided on the side of the controller 120 that corresponds to the side the player needs to progress to. In the example shown in FIG. 5B, the player is guided to take the right side of the fork toward path C (indicated by solid line 520) rather than continuing down path A (indicated by dashed line 510) by vibrating the right side of the controller or the handle of the controller. A similar notification may be provided to the player if the haptic profiler wants the player to pay attention to the right side of the screen. FIG. 5C illustrates another option, Option B, where in one embodiment, notification settings are defined for the input interface of the input device (e.g., the touchscreen of the controller 120). In the example shown in FIG. 5C, a directional cue is provided on the touchscreen using an arrowhead (indicated by a directional arrow) moving from left to right to indicate the direction the haptic profiler wants the player to move or direct their attention in the game scenario rendered on the display screen 100. In one embodiment, the intensity of the directional cue may increase from left to right to provide additional feedback to the player. In addition to visual cues, notifications may also include audio cues. For example, as the arrowhead moves from left to right, the audio cue may increase in volume or provide a distinctive sound. Alternatively, special audio adjustments may be set by or for the player to indicate the direction the player needs to direct their attention in the game or on the display screen rendering the gameplay. In some implementations, separate audio adjustments may be set to indicate correct and incorrect directions.

[0092] FIG. 5D illustrates yet another setting option, Option C, in which, in one embodiment, each button on the controller 120 is configured to provide notification / feedback. The setting for each button may be defined differently to indicate different feedback responses to the player. For example, each button may be configured to provide haptic feedback. Instead of, or in addition to, haptic feedback, each button may also include a visual setting, which may cause each button to light up in a different color or sequence of colors to indicate to the player the different button presses (single presses and multiple presses) the player is making or needs to make. Note that the various setting options described with reference to FIGS. 5B-5D are described in connection with various settings for different controls on the controller to provide haptic and / or visual feedback. Feedback is not limited to the two specified feedbacks or controllers. Furthermore, the setting options illustrated in FIGS. 5B-5D are exemplary, and additional input devices and additional feedback, such as in the form of audio, may also be provided to the player.

[0093] 6 illustrates, in one embodiment, an operational flow of a method for providing notifications to a player during gameplay of a video game to enable the player to have a satisfying gameplay experience. The method begins when an interactive task is detected within a game scenario of the video game that requires action from the player, as shown in operation 610. The player logs into game cloud system 300 and selects a video game for gameplay. Game cloud system 300 verifies the authenticity of the request to play the video game by accessing the player's user account 304 stored in user data store 305 and the game title in game data store 306, and determines whether the player is authorized to access and play the video game. Upon successful authentication, an instance of the video game runs on one or more game servers 302 in one or more data centers 301 of the gaming cloud system 300. Game input provided by the player via an input device, such as a handheld controller, is used to affect the game state of the video game and generate gameplay data 308 that is stored in a gameplay data store 307. The gameplay data 308 is used to generate a game scenario that captures the current game state of the video game. Frames of content of the game scenario are streamed to the player's client device for rendering on a display screen associated with the client device. A game scenario rendered on a display screen includes game objects (also called "game assets"), including static game objects and moving game objects. Some of the game objects included in the game scenario may be provided to allow a player to earn points or to provide challenges to the player. Some other game objects may be provided as part of the game scenario, while some of the remaining game objects may hold tools, hints, or keys and require player interaction (i.e., actions that accomplish interactive tasks) to allow the player to obtain tools or unlock challenges / levels in order to progress through the video game. The game objects available in a game scenario change dynamically as the game state of the video game changes. The haptic profiler 400 is configured to interact with the game logic of the game executing on one or more game servers 302 to determine the game objects currently available in the game scenario of the game rendering on the display screen of the player's client device, the game objects that hold hints, keys, or tools for progressing through the video game, and the game objects with which the player is interacting during gameplay. Based on its interaction with the game logic, the haptic profiler 400 can correlate the content of the current game scenario with the context of actions taken by the player in the game scenario and identify game objects or interactive tasks that require action from the player. The identified game objects or interactive tasks have not yet been interacted with by the player during gameplay.

[0094] In response to detecting a game object or interactive task requiring the player's attention or action, a profile of the player playing the video game is identified, as shown in operation 620. The player's profile is stored in a user data store and includes the player's playing style. As the player plays the video game and other video games more, the player's playing style improves and the improved playing style is updated in the player's profile.

[0095] As shown in operation 630, the player's play style is used to generate haptic responses for the player. A haptic learning engine 403 within the haptic profiler 400 is used to analyze the player's gameplay to learn the player's game progression and select cues regarding the player's ability to understand the visual content provided in the game. The cues selected by the haptic profiler 400 are used to determine the player's play style. The player's play style, the current context of the gameplay content, and personal progress information made by the player in the game are used to determine the type of haptic response to provide to the player. The haptic responses identified for the player are unique to the player, and are used to guide or direct the player to interactive tasks within the game scenario of the game rendering on the display screen of the client device. The haptic responses identified for the player by the haptic learning engine 403 include haptic settings that are dynamically applied to an input device (e.g., a controller or other input device) as the user plays the game.

[0096] Telemetry data generated from the player's gameplay is processed to extract certain features that provide cues indicating whether the player is progressing or not progressing in the game, whether the player is able or unable to understand the visual content provided by the game, or whether the player is able or unable to handle the speed or complexity of the game. The cues can be used to determine whether the player is a novice player, a good player, or an excellent player based on certain game rules, and whether the player needs assistance in playing the game. Such determinations can be made by the amount of time it takes the player to progress through different parts of the game, the number of attempts the player makes to accomplish a task, etc. The haptic learning engine is capable of progressively learning and interpreting different cues not only from the player's responses to game prompts, but also from the responses of other players. The haptic learning engine 403 aggregates telemetry data from multiple players, including the current player, from different gameplay sessions and fine-tunes haptic settings that can be customized or recommended for the player based on the player's gameplay style. The haptic settings identified for the player can be used to adjust the haptic responses or feedback provided to the player for any type of input device, such as a controller, a head-mounted display, a wearable device (e.g., smart glasses, a smartwatch, etc.), or for other peripheral devices that can be used to notify the player. Customized notifications can extend beyond game applications to other interactive applications used by the player.

[0097] 7 illustrates an embodiment of an information service provider architecture. An information service provider (ISP) 702 delivers many information services to geographically dispersed and connected users (i.e., players) 700 via network 200. An ISP may deliver only one type of service, such as stock price updates, or a variety of services, such as broadcast media, news, sports, gaming, etc. Furthermore, the services offered by each ISP are dynamic, i.e., services can be added or removed at any time. Thus, the ISPs that offer a particular type of service to a particular individual may change over time. For example, an ISP near the user may serve the user while the user is in their hometown, but when the user moves to another town, a different ISP may serve the user. The local ISP may forward the necessary information and data to the new ISP, so that the user information "follows" the user to the new town, making the data closer to the user and more accessible. In another embodiment, a master-server relationship may be established between a master ISP that manages the user's information and a server ISP that interfaces directly with the user under the control of the master ISP. In another embodiment, as a client moves around the world, data may be forwarded from one ISP to another, making the ISPs better positioned to serve the users that will deliver these services.

[0098] The ISP 702 includes an application service provider (ASP) 706 that provides computer-based services to customers over a network (e.g., by way of example and not limitation, any wired or wireless network, LAN, WAN, WiFi, broadband, cable, fiber optic, satellite, cellular (e.g., 4G, 5G, etc.), the Internet, etc.). Software provided using the ASP model is sometimes also referred to as on-demand software or software as a service (SaaS). A simple form of providing access to a specific application program (e.g., customer relationship management) uses a standard protocol such as HTTP. The application software resides on the vendor's system and is accessed by the user via a web browser using HTML, by dedicated client software provided by the vendor, or by other remote interfaces such as a thin client.

[0099] Services delivered over wide geographic areas often use cloud computing. Cloud computing is a style of computing in which dynamically scalable, often virtualized resources are provided as a service over the Internet. Users do not need to be experts in the "cloud" technical infrastructure that supports them. Cloud computing can be categorized into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services often provide common business applications online, accessed from a web browser, while software and data are stored on servers. The term cloud is used as a metaphor for the Internet (e.g., using servers, storage, and logic) based on how the Internet is represented in computer network diagrams, and is an abstraction that hides complex infrastructure.

[0100] Additionally, the ISP 702 includes a game processing server (GPS) 708 used by game clients to play single-player and multiplayer video games. Most video games played over the Internet operate through a connection to a game server. Typically, games use dedicated server applications that collect data from players and distribute that data to other players. This is more efficient and effective than a peer-to-peer arrangement, but requires a separate server to host the server application. In another embodiment, the GPS establishes communication and exchanges information between players and their respective gameplay devices without relying on a centralized GPS.

[0101] A dedicated GPS is a server that runs independently of the client. Such servers typically run on dedicated hardware located in a data center, providing more bandwidth and dedicated processing power. Dedicated servers are the preferred method of hosting most game servers for PC-based multiplayer games. Massively multiplayer online games typically run on dedicated servers hosted by the software company that owns the game title, allowing the dedicated server to control and update the content.

[0102] The Broadcast Processing Server (BPS) 710 distributes audio or video signals to viewers. Broadcasting to a very small audience is sometimes called narrowcasting. The final leg of broadcast distribution describes how the signal reaches the listener or viewer. The signal can arrive over the airwaves to an antenna and receiver, similar to a radio or television station, or the signal can arrive via a station, via cable television or cable radio (or "over-the-air cable"), or directly from a network. The Internet can also provide either radio or television to recipients, particularly using multicast, which allows signal and bandwidth sharing. Historically, broadcasting was defined by geographic area, such as national or local broadcast. However, with the proliferation of high-speed Internet, broadcasting is no longer defined by geography, with content reaching nearly every country around the world.

[0103] A storage service provider (SSP) 712 provides computer storage space and related management services. The SSP also provides periodic backups and archiving. By providing storage as a service, users can order more storage as needed. Another major advantage is that the SSP includes backup services, ensuring that users do not lose all of their data if their computer's hard drive fails. Furthermore, multiple SSPs may have full or partial copies of a user's data, allowing users to access data in an efficient manner regardless of where the user is located or the device used to access the data. For example, a user can access personal files on their home computer and, similarly, on their mobile phone while they are on the move.

[0104] Communications providers 714 provide connectivity to users. One type of communications provider is an Internet Service Provider (ISP), which provides access to the Internet. An ISP connects to its customers using any suitable data transmission technology to deliver Internet Protocol datagrams, such as dial-up, DSL, cable modem, fiber, wireless, or dedicated high-speed interconnects. Communications providers may also offer messaging services, such as email, instant messaging, and SMS text. Another type of communications provider is a Network Service Provider (NSP), which sells bandwidth or network access by providing direct backbone access to the Internet. Network service providers can consist of telecommunications companies, data carriers, wireless communication providers, Internet service providers, cable television operators offering high-speed Internet access, etc.

[0105] A data exchange 704 interconnects several modules inside the ISP 702 and connects these modules to users 700 via network 200. The data exchange 704 can cover a small area where all modules of the ISP 702 are in close proximity, or it can cover a large geographic area when different modules are geographically dispersed. For example, the data exchange 704 could include a high-speed Gigabit Ethernet (or faster) network within a cabinet in a data center, or an intercontinental virtual area network (VLAN).

[0106] A user 700 accesses remote services using a client device 720 (i.e., client device 100 of FIG. 1 ), which includes at least a CPU, memory, display, and I / O. The client device may be a PC, a cell phone, a netbook, a tablet, a gaming system, a PDA, etc. In one embodiment, the ISP 702 recognizes the type of device used by the client and adjusts the communication method used. In other cases, the client device accesses the ISP 702 using a standard communication method, such as HTML.

[0107] FIG. 8 illustrates components of an exemplary device 800 that can be used to perform aspects of various embodiments of the present disclosure. The block diagram illustrates device 800, which may incorporate or be a personal computer, video game console, personal digital assistant, server 302, or other digital device suitable for implementing embodiments of the present disclosure. FIG. 8 illustrates an exemplary system with hardware components suitable for training a video game and / or an AI model capable of performing various functions in connection with gameplay of the video game, according to one embodiment of the present disclosure. Device 800 includes a central processing unit (CPU) 802 that runs software applications and, optionally, an operating system. CPU 802 may consist of one or more homogeneous or heterogeneous processing cores. For example, CPU 802 may be one or more general-purpose microprocessors having one or more processing cores. Further embodiments can be implemented using one or more CPUs with a microprocessor architecture particularly suited for highly parallel and computationally intensive applications, such as interpreting queries, identifying contextually relevant resources, and real-time implementation and rendering of contextually relevant resources in video games, media applications and interactive entertainment applications, and applications configured to perform deep learning, content classification, and user classification. For example, CPU 802 can be configured to include a haptic learning engine 403, which is a machine learning algorithm (referred to herein as an AI engine or deep learning engine) configured to support and / or perform learning operations with respect to the video game and / or providing various functions (e.g., predictions, suggestions) related to gameplay of the video game. The CPU 802 further includes an analyzer 840 configured to analyze inputs and interactions and provide analysis results for generating and training a haptic learning model (AI model) 403b. The trained haptic learning model 403b provides outputs in response to a particular set of player inputs, where the outputs depend on predetermined functions of the trained haptic learning model 403b. The trained haptic learning model 403b is used to identify optimal haptic setting outputs to dynamically apply to the input device, allowing the input device to provide appropriate notifications to the player during gameplay. The notifications may guide the player through interactive tasks or provide feedback regarding movements or actions the player needs to or is performing in the game.

[0108] The device 800 may be localized to the player playing the game segment (e.g., a gaming console), or may be remote from the player (e.g., a back-end server processor), or may be remote from one of many servers that use virtualization in a gaming cloud system for remote streaming of gameplay to client devices (or simply called "clients").

[0109] Memory 804 stores applications and data used by CPU 802. Storage 806 provides non-volatile storage and other computer-readable media for applications and data and may include fixed disk drives, removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other optical storage devices, as well as signal transmission and storage media. User input device 808 communicates user input from one or more users to device 800, examples of which may include a keyboard, mouse, joystick, touchpad, touchscreen, handheld controller, wearable controller, still or video recorder / camera, tracking device for gesture recognition, and / or microphone. The network interface 814 enables the device 800 to communicate with other computer systems via electronic communications networks, which may include wired or wireless communications over local area networks and wide area networks such as the Internet. The audio processor 812 is adapted to generate analog or digital audio output from instructions and / or data provided by the CPU 802, memory 804, and / or storage 806. The components of the device 800, including the CPU 802, memory 804, data storage 806, user input device 808, network interface 814, and audio processor 812, are connected via one or more data buses 822.

[0110] A graphics subsystem 820 is further connected to a data bus 822 and to the components of device 800. The graphics subsystem 820 includes a graphics processing unit (GPU) 816 and a graphics memory 818. The graphics memory 818 includes display memory (e.g., a frame buffer) used to store pixel data for each pixel of an output image. The graphics memory 818 can be integrated into the same device as the GPU 816, connected as a separate device from the GPU 816, and / or implemented within memory 804. Pixel data can be provided directly from CPU 802 to graphics memory 818. Alternatively, CPU 802 provides data and / or instructions defining desired output images to GPU 816, which generates pixel data for one or more output images from the data and / or instructions. The data and / or instructions defining the desired output images can be stored in memory 804 and / or graphics memory 818. In some embodiments, GPU 816 includes 3D rendering functionality for generating pixel data for output images from instructions and data defining scene geometry, lighting, shading, texturing, motion, and / or camera parameters. GPU 816 may further include one or more programmable execution units capable of executing shader programs.

[0111] Graphics subsystem 820 periodically outputs pixel data of an image from graphics memory 818 for display on display device 810. Display device 810 may be any device capable of displaying visual information in response to signals from device 800, including CRT displays, LCD displays, plasma displays, and OLED displays. For example, device 800 may provide analog or digital signals to display device 810.

[0112] It should be noted that access services distributed over wide geographic areas, such as providing access to games in this embodiment, often use cloud computing. Cloud computing is a style of computing in which dynamically scalable, often virtualized resources are provided as a service over the Internet. Users do not need to be experts in the technical infrastructure of the "cloud" that supports them. Cloud computing can be categorized into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services often provide common applications online, such as video games, accessed through a web browser, while the software and data are stored on cloud servers. The term cloud is used as a metaphor for the Internet, based on how the Internet is represented in computer network diagrams, and is an abstract concept that hides complex infrastructure.

[0113] In some embodiments, a game server 302 may be used to operate a duration information platform for video game players. Most video games played over the Internet operate through a connection to a game server. Typically, the game uses a dedicated server application that collects data from a player and distributes it to other players. In other embodiments, the video game may be executed by a distributed game engine. In these embodiments, the distributed game engine may run on multiple processing entities (PEs), where each PE executes a functional segment of a given game engine on which the video game runs. Each processing entity is viewed by the game engine as simply a computational node. The game engine typically performs a functionally diverse set of operations and executes the video game application along with additional services experienced by the user. For example, a game engine implements game logic and performs game calculations, physics analysis, geometry transformations, rendering, lighting, shading, audio, and additional in-game or game-related services, which may include, for example, messaging, social utilities, voice communication, gameplay playback functions, help functions, etc. While a game engine may sometimes run in an operating system virtualized by a hypervisor on a particular server, in other embodiments, the game engine itself may be distributed among multiple PEs, each resident on a different server unit in a data center.

[0114] According to this embodiment, each PE to perform may be a server unit, a virtual machine, or a container, depending on the respective needs of the game engine segment. For example, if a game engine segment is responsible for camera transformation, that particular game engine segment may be provisioned with a virtual machine associated with a graphics processing unit (GPU) because it will be performing a large number of relatively simple mathematical operations (e.g., matrix transformations). Other game engine segments, requiring fewer but more complex operations, may be provisioned with PEs associated with one or more powerful central processing units (CPUs).

[0115] By distributing the game engine, the game engine is given flexible computational characteristics that are not constrained by the capabilities of a physical server unit. Instead, the game engine is provisioned with as many or as few computational nodes as needed to meet the demands of the video game. From the perspective of the video game and the video game player, a game engine that is distributed across multiple computational nodes is indistinguishable from a non-distributed game engine running on a single processing entity. This is because a game engine manager or supervisor distributes the workload and seamlessly integrates the results to provide the video game output component to the end user.

[0116] Users access the remote service using a client device that includes at least a CPU, display, and I / O. The client device can be a PC, cell phone, netbook, PDA, mobile device, etc. In one embodiment, a network running on the game server recognizes the type of device used by the user and adjusts the communication method used. In other cases, the client device accesses the game server's application over the Internet using a standard communication method such as HTML.

[0117] It should be understood that a given video game or gaming application may be developed for a particular platform and a particular associated controller device 120 (or simply referred to as a "controller"). However, when such a game is made available through a game cloud system as presented herein, a user (e.g., a player) may access the video game using a different controller 120. For example, a game may be developed for a gaming console and its associated controller, but a user may access a cloud-based version of the game from a personal computer utilizing a keyboard and mouse. In such a scenario, an input parameter setting may define a mapping from inputs that can be generated by the controller 120 (in this case, a keyboard and mouse) available to the user to inputs that are acceptable for execution of the video game.

[0118] In another example, a user may access the cloud gaming system via a tablet computing device, a touchscreen smartphone, or other touchscreen-driven device. In this case, the client device and controller 120 are integrated together on the same device, and input is provided by detected touchscreen input / gestures. For such a device, input parameter settings may define specific touchscreen inputs that correspond to game inputs for the video game. For example, buttons, directional pads, or other types of input elements may be displayed or overlaid during gameplay to indicate locations on the touchscreen that the user can touch to generate game input. Gestures, such as swipes in specific directions or specific touch actions, may also be detected as game input. In one embodiment, a tutorial showing how to provide inputs via the touchscreen for gameplay may be provided to the user, e.g., before beginning gameplay of the video game, to familiarize the user with control operations on the touchscreen.

[0119] In some embodiments, the client device serves as a connection point for the controller 120. That is, the controller 120 communicates with the client device via a wireless or wired connection, transmitting inputs from the controller 120 to the client device. The client device may then process these inputs and then transmit the input data to a game cloud server over a network (e.g., a network accessed through a local network device such as a router). However, in other embodiments, the controller itself may be a network device capable of communicating inputs directly over a network to a game cloud server without the need to communicate such inputs through a client device first. For example, the controller may connect to a local network device (such as the router described above) to send data to and receive data from a game cloud server. Thus, while the client device may still be required to receive video output from the cloud-based video game and render it on a local display, input latency may be reduced by allowing the controller to send inputs directly over the network to the game cloud server, bypassing the client device.

[0120] In one embodiment, networked controllers and client devices can be configured to transmit certain types of input directly from the controller to the game cloud server and other types of input via the client device. For example, inputs whose detection does not rely on any additional hardware or processing, aside from the controller itself, can be transmitted directly from the controller to the game cloud server over the network, bypassing the client device. Such inputs can include button inputs, joystick inputs, built-in motion-sensing inputs (e.g., accelerometers, magnetometers, gyroscopes), etc. However, inputs that utilize additional hardware or require processing by the client device can be transmitted by the client device to the game cloud server. These can include video or audio captured from the game environment, which can be processed by the client device before transmission to the game cloud server. Additionally, inputs from the controller's motion-sensing hardware can be processed by the client device in conjunction with the captured video, and the controller's position and movement can be detected, which the client device then communicates to the game cloud server. It should be understood that the controller 120 according to various embodiments can also receive data (e.g., feedback data) from the client device or directly from the game cloud server.

[0121] It should be understood that the various embodiments defined herein can be combined or incorporated into specific embodiments using various features disclosed herein. Thus, the examples provided are only some possible examples and are not limited to various embodiments in which many more embodiments can be defined by combining various elements. In some instances, some embodiments may include fewer elements without departing from the spirit of the disclosed embodiment or equivalent embodiments.

[0122] Embodiments of the present disclosure may be practiced with a variety of computer system configurations, including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.

[0123] In some embodiments, communication may be facilitated using wireless technology. Such technology may include, for example, 5G wireless communication technology. 5G is the fifth generation of cellular network technology. A 5G network is a digital cellular network in which the service area covered by a provider is divided into smaller geographic areas called cells. Analog signals representing sound and images are digitized in the phone, converted by an analog-to-digital converter, and transmitted as a stream of bits. All 5G wireless devices within a cell communicate over radio waves using a local antenna array and low-power automatic transceivers (transmitters and receivers) within the cell through frequency channels assigned by the transceiver from a pool of frequencies reused in other cells. The local antennas are connected to the telephone network and the Internet by high-bandwidth fiber optic or wireless backhaul connections. As with other cell networks, mobile devices crossing from one cell to another are automatically transferred to the new cell. It should be understood that 5G networks are merely examples of types of communications networks, and that embodiments of the present disclosure may utilize previous generations of wireless or wired communications, as well as subsequent generations of wired or wireless technologies following 5G.

[0124] In view of the above embodiments, it should be understood that the present disclosure can employ various computer-implemented operations involving data stored in computer systems. These operations are operations requiring physical manipulation of physical quantities. Any of the operations described herein that form part of the present disclosure are useful machine operations. The present disclosure also relates to devices or apparatus for performing these operations. The apparatus can be specially constructed for the required purposes, or the apparatus can be a general-purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines can be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct more specialized apparatus to perform the required operations.

[0125] Although the operations of the method have been described in a particular order, it should be understood that other housekeeping operations may be performed between operations, or operations may be coordinated to occur at slightly different times, or operations may be distributed in a system that allows processing operations to occur at various intervals associated with the processing, so long as the processing of telemetry data and game state data to generate a modified game state is performed in the desired manner.

[0126] One or more embodiments may also be fabricated as computer-readable code on a computer-readable medium. The computer-readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of computer-readable media include hard drives, network-attached storage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tape, and other optical and non-optical data storage devices. The computer-readable medium may include computer-readable tangible media distributed across network-connected computer systems, whereby the computer-readable code is stored and executed in a distributed fashion.

[0127] Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications can be practiced within the scope of the appended claims. The present embodiments are therefore to be considered as illustrative and not restrictive, and the present embodiments are not limited to the details given herein, but can be modified within the scope of the appended claims and their equivalents.

[0128] It should be understood that the various embodiments defined herein can be combined or incorporated into specific embodiments using various features disclosed herein. Thus, the examples provided are only some possible examples and are not limited to various embodiments in which many more embodiments can be defined by combining various elements. In some instances, some embodiments may include fewer elements without departing from the spirit of the disclosed embodiment or equivalent embodiments.

Claims

1. 1. A method for providing a haptic response to a user during gameplay of a video game, comprising: Detecting interactive tasks within a game scenario of the video game that require actions from the user during gameplay, the interactive tasks being identified to assist the user with progression of the video game and determined by correlating content of the game scenario with a context of the actions taken by the user in the game scenario; identifying a user profile of the user playing the video game; generating the haptic response specific to the user based on the user's play style, the haptic response being triggered during gameplay according to haptic settings defined for the user profile of the user and being used to provide directional cues to the user via a controller used to provide game input to the video game to direct the user's attention to the interactive task, the haptic response being dynamically adjusted and provided during the gameplay to guide the user to the interactive task within the game scenario requiring the action from the user of the video game; generating said haptic response includes, during gameplay, deactivating one of the controls of said controller used to provide game input for progressing through said video game, while leaving active a control used to perform the action required to accomplish said interactive task, and upon detecting that said action by the user has been completed, reactivating the deactivated control to allow said user to continue playing said video game; the haptic response is dynamically updated in response to detected changes in the play style of the user; The method, wherein the haptic response is used to notify a player that he or she has missed a particular game task necessary for player progression, and the notification is customized with haptic settings identified using a haptic learning model trained using telemetry data of the player and tuned using telemetry data of a plurality of other players, the telemetry data being collected from a user's gameplay of the video game.

2. 2. The method of claim 1 , wherein the haptic response continues until the action required to accomplish the interactive task is performed by the user, and wherein providing the directional cue dynamically varies an intensity of the haptic response provided to the user to move a game object controlled by the user during the gameplay in a particular direction, the intensity dynamically varying based on a direction of movement of the game object controlled by the user during the gameplay in association with the interactive task.

3. The method of claim 1 , wherein functionality of the controller is used to generate the haptic response.

4. 2. The method of claim 1, wherein the haptic response includes spatial cues to orient the user to the interactive task of the game scenario, the spatial cues being provided using a three-dimensional representation of the game scenario of the video game.

5. The method of claim 1 , wherein the haptic response is triggered according to the haptic settings that are customized for the user.

6. The method of claim 5 , wherein the haptic settings are defined based on input provided by the user.

7. 6. The method of claim 5, wherein the play style is determined using a haptic learning engine using machine learning logic, the haptic learning engine being dynamically trained on the user's game inputs and game progress made by the user in the video game, the haptic settings are dynamically adjusted from the training of the haptic learning engine and applied to the controller when the haptic response is triggered, and the adjustments to the haptic settings are updated to the user profile for the user.

8. 8. The method of claim 7, wherein the game progress is determined using the telemetry data, the telemetry data being analyzed to extract certain features indicative of the play style of the user or the game progress of the video game.

9. 2. The method of claim 1 , wherein the action requested from the user is a movement in a particular direction, and the haptic response provided during the gameplay includes the directional cue indicating a direction in which the user needs to move or a direction in which the interactive task is located in the game scenario.

10. 10. The method of claim 9, wherein the controller has a plurality of tactile elements, and the directional cue provided in the tactile response includes sequentially activating the tactile elements from among the plurality of tactile elements such that the tactile response enables movement from one tactile element to a subsequent tactile element of the plurality of tactile elements in the direction specified by the directional cue.

11. 2. The method of claim 1, wherein the haptic response is defined to provide a change in feedback provided to the user, the change in the feedback being dynamically controlled based on the action taken by the user in the video game and unique to the user.

12. The method of claim 1 , wherein the haptic response is configured to change over time based on content of the game scenario of the video game or game input provided by the user.

13. The method of claim 1 , wherein the haptic response of the user is generated to correlate content of the game scenario with a context of the actions taken by the user in the game scenario.

14. The method of claim 1 , wherein the interactive task is the action taken by the user to interact with game assets or an avatar of another user playing the video game.

15. 1. A method for providing a haptic response to a user during gameplay of a video game running on a server of a game cloud system, comprising: identifying interactive tasks within a game scenario of the video game that require actions from the user during gameplay, the interactive tasks being identified to assist the user with progression of the video game and determined by correlating content of the game scenario with a context of the actions taken by the user in the game scenario; identifying a user profile of the user playing the video game; generating the haptic response specific to the user based on the user's play style, the haptic response being triggered during game play according to haptic settings defined for the user's user profile and being used to provide directional cues to the user to direct the user's attention to the interactive task, the haptic response being dynamically adjusted and provided to guide the user to the interactive task within the game scenario of the video game; generating said haptic response includes, during gameplay, deactivating one of the controls of an input device used to provide game input for progressing through said video game, while leaving active a control used to perform said action required to accomplish said interactive task, and upon detecting completion of said action by said user, reactivating said deactivated control to allow said user to continue said gameplay of said video game; the haptic response is dynamically updated in response to detected changes in the play style of the user; The method, wherein the haptic response is used to notify a player that he or she has missed a particular game task necessary for player progression, and the notification is customized with haptic settings identified using a haptic learning model trained using telemetry data of the player and tuned using telemetry data of a plurality of other players, the telemetry data being collected from a user's gameplay of the video game.

16. 16. The method of claim 15, wherein the input device is a game controller and the haptic response is provided to the user via a control of the game controller used to provide game input to the video game.

17. The method of claim 16 , wherein the haptic response is generated using functionality of the game controller.

18. The method of claim 15 , wherein the haptic response is provided to the user via a head mounted display used to view the gameplay of the video game.

19. 16. The method of claim 15, wherein the haptic response includes spatial cues for orienting the user to the interactive task of the game scenario, the spatial cues being provided using coordinates of the interactive task within a three-dimensional representation of the game scenario of the video game.

20. The method of claim 15 , wherein the haptic response is triggered according to the haptic settings that are customized for the user.

21. The method of claim 20 , wherein the haptic settings are predefined by the user.

22. The method of claim 20 , wherein the haptic settings are dynamically defined based on the user's play style.

23. The method of claim 15 , wherein the haptic response continues until the action required to accomplish the interactive task is performed by the user.

24. 1. A method for providing a haptic response to a user during gameplay of a video game running on a server of a game cloud system, comprising: examining game actions taken by the user during the gameplay of the video game, the context of the game actions being examined in relation to content of a game scenario occurring in the video game to determine a context associated with the game actions; identifying information regarding the haptic responses to be provided to the user based on the examination of the game actions, the information being used to define a haptic setting specific to the user, the defined haptic setting being stored in a user profile for the user, the information regarding the haptic responses being dynamically adjusted and dynamically updated based on the game actions of the user collected during game play, the updates to the information causing corresponding updates to the haptic settings defined in the user profile for the user; generating the haptic response specific to the user in response to detecting an interactive task within the game scenario of the video game requiring the game action from the user, the interactive task being identified to assist the user with progression of the video game and determined by correlating content of the game scenario with a context of the game action taken by the user in the game scenario; the haptic response is generated based on a play style of the user, the haptic response being triggered according to the haptic settings defined in the user profile of the user and being used to provide directional cues to guide the user to the interactive task requiring a game action from the user within the game scenario of the video game; and the haptic response is used to notify a player that he or she has missed a particular game task necessary for the player's progression, and the notification is customized with haptic settings identified using a haptic learning model trained using telemetry data of the player and tuned using telemetry data of a plurality of other players, the telemetry data being collected from a user's gameplay of the video game; generating said haptic response by deactivating, during gameplay, one of the controls of an input device used to provide game input for progressing through said video game, while leaving active a control used to perform said action required to accomplish said interactive task; reactivating the control upon detecting successful completion of the game action by the user, and reactivating the control allows the user to continue playing the video game.

25. the input device is a controller, and the haptic response is provided to the user via the controller in conjunction with controls used to provide game input to the video game; The method of claim 24 , wherein the haptic response is provided to the user using a function of the controller.

26. 25. The method of claim 24, wherein the haptic response is provided to the user via a head mounted display used to view the gameplay of the video game, the haptic response being provided using capabilities of the head mounted display.

27. 25. The method of claim 24, wherein the haptic response is provided to inform the user of a particular action required of the user during gameplay to achieve an objective of the interactive task.

28. The method of claim 24 , wherein the haptic settings for the user are defined based on a play style of the user identified from examining the game actions.

29. The method of claim 2 , wherein the intensity of the haptic response is increased upon detecting that the game object controlled by the user during gameplay is moving in the particular direction to interact with the interactive task.

30. the intensity of the haptic response is increased in the particular direction to indicate to the user a direction in which the interactive task is located relative to a position of the game object controlled by the user during the gameplay in the game scenario; The method of claim 2 , wherein the increase in strength is used to steer the game object controlled by the user during gameplay to move towards the particular direction.

Images