Automated Artificial Intelligence (AI) Personal Assistant Law
By monitoring and analyzing users' gaming performance, and using artificial intelligence to provide personalized game suggestions and task assignments, this technology addresses the shortcomings of existing game assistance technologies and improves the success rate and experience of users and teams.
Patent Information
- Application Number
- CN202210669703.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-03-23
- Filing Date
- 2017-04-27
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2037-04-27
AI Technical Summary
Existing technologies are insufficient to effectively monitor and provide game assistance, and cannot offer personalized game suggestions based on users' game performance and historical data, especially in improving the success rate of team cooperation in multiplayer modes.
By monitoring users' gameplay and using artificial intelligence to analyze their gaming style and skill level, personalized game suggestions and task assignments are provided to improve the user's gaming experience and the success rate of teamwork.
It improves the user's gaming experience in both single-player and multiplayer modes, enhances the effectiveness of game assistance and the user's success rate, and helps users avoid difficult parts of the game and improve efficiency.
Smart Images

Figure CN115089972B_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application with application number 201780047830.1, application date April 27, 2017, and invention name “Automatic Artificial Intelligence (AI) Personal Assistant Method”. Technical Field
[0002] The present disclosure relates to video games. Among other things, the present disclosure describes methods and systems for monitoring game play of users playing a game application, learning various paths available within the game application and how those users are playing along those paths, and providing assistance to the user based on the user's past performance, the user's current performance, and the historical performance of other users playing the game application. Summary of the invention
[0003] Embodiments of the present disclosure relate to systems and methods for providing game assistance to a user playing a game application based on the user's proficiency compared to the game play of other users playing the game application. The user proficiency takes into account the user's performance during the current game play, the user's historical performance in past game plays associated with multiple game applications of general or specific types, and the user's game style associated with multiple game applications of general or specific types. Other embodiments of the present disclosure relate to systems and methods for providing game assistance to a team of members playing a multi-player game application based on a predictive model, the predictive model determining the team's success probability of completing a task set for various combinations of task assignments to team members based on the member's game profile, task set, and current game environment. Specifically, the game profile defines the game proficiency of the corresponding member, and is based on the user's performance during the current game play, the user's historical performance in past game plays associated with multiple game applications of general or specific types, and the user's game style associated with multiple game applications of general or specific types.
[0004] In one embodiment, a method for assisting game play is disclosed. The method includes monitoring the game play of a user playing a game application, wherein the user has a defined task to be completed. The task may be associated with a task type that defines a task of a similar configuration. The method also includes determining a task type proficiency rule based on the results of multiple players undertaking multiple tasks of the task type. The method includes determining a player proficiency score for completing the task based on the task type proficiency rule. The player proficiency score takes into account the user's performance during the current game play, the user's historical performance in past game plays associated with multiple game applications of general or specific types, and the user's game style associated with multiple game applications of general or specific types. The method includes determining a user's predicted success rate for completing the task based on the player proficiency score, the task type proficiency rule, and the task. In some embodiments, the game environment is also taken into account. The method includes determining recommendations for the user based on the user's predicted success rate.
[0005] In another embodiment, a method for providing game assistance is disclosed. The method includes monitoring multiple game sessions of multiple players playing multiple game applications. The multiple game sessions include multiple tasks to be completed, wherein the multiple tasks are associated with multiple task types. The method includes registering multiple team members of a team. The method includes determining multiple ratings associated with multiple task types for multiple team members. This may include accessing user profiles of team members to access game proficiency ratings. The method includes determining, for multiple combinations of task assignments from a related task set, multiple predicted success rates for the team to complete the task set based on the multiple ratings, the task set, and the current conditions of the game environment of the game application. The method includes assigning tasks in the related task set to multiple team members so that the predicted success rate of the team to complete the task set is the highest.
[0006] Other aspects of the disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The present disclosure may be best understood by reference to the following description taken in conjunction with the accompanying drawings, in which:
[0008] Figure 1A A system for providing personal assistance to a user of a locally executed gaming application according to one embodiment of the present disclosure is presented.
[0009] Figure 1B A system for providing personal assistance to a user playing a gaming application executed on a cloud gaming network according to one embodiment of the present disclosure is presented.
[0010] Figure 1CAn embodiment according to the present disclosure is shown Figure 1A Automatic artificial intelligence (AI) personal assistance system.
[0011] Figure 1D An exemplary neural network for training a profiler engine according to one embodiment of the present disclosure is presented.
[0012] Figure 2 A system diagram 200 for providing automatic AI personal assistance to a user playing a gaming application executed on a cloud gaming network according to one embodiment of the present disclosure is shown.
[0013] Figure 3 Tasks and subtasks assigned to team members participating in a multiplayer gaming application are presented according to one embodiment of the present disclosure.
[0014] Figure 4 is a flow chart illustrating steps in a method for assisting game play of a user playing a game application according to one embodiment of the present disclosure.
[0015] Figure 5 is a flowchart illustrating steps in a method for assigning tasks to team members of a team participating in a multiplayer gaming application based on a predictive model that assesses the success of the team in completing a presented task based on past performance of the team members when performing the same type of tasks according to one embodiment of the present disclosure.
[0016] Figure 6 is a diagram showing components of a head mounted display according to an embodiment of the present disclosure.
[0017] Figure 7 is a block diagram of a gaming system according to various embodiments of the present disclosure. The gaming system is configured to provide a video stream to one or more clients via a network. DETAILED DESCRIPTION
[0018] Although the following detailed description contains many specific details for the purpose of illustration, it will be appreciated by those skilled in the art that many changes and modifications to the following details are within the scope of the present disclosure. Therefore, the various aspects of the present disclosure described below are set forth without loss of generality and without imposing limitations on the claims following this specification.
[0019] In general, various embodiments of the present disclosure describe systems and methods for providing assistance to users who are playing game applications in single-player or multi-player configurations. Specifically, the input history of the user's game play and the comparative history of the game play of multiple players are used to rate the proficiency of the user when playing the game application or when playing a type of game application or when completing tasks or subtasks in the game application. In addition, the proficiency of team members of a team participating in a multi-player game application can be used to predict the probability of team success (e.g., winning a competition in a game application implemented in a competitive game arena, completing the game application, and completing tasks and / or subtasks in the game application, etc.), taking into account the conditions of the game environment, which is based on the compatibility of the roles of the team members (e.g., assigned tasks) and team building (e.g., team members). In the described manner, the efficiency and / or proficiency of the user can be quantified for the user who is playing the game application, and the efficiency and / or proficiency can be further used to balance the game play of the user or team member (e.g., recommending the next task, subtask or action), and / or for predicting the satisfaction and / or completion of the tasks, subtasks, and actions of the user and / or team member in the game. Specifically, artificial intelligence can be used to continuously examine the game play of multiple players of multiple game applications of various types (e.g., changing game data) to learn the players' game styles, skills, and habits in order to determine the predicted results of a single user or team member or team success when the game application is played in a single player mode or a multi-player mode. In this way, the prediction model can adapt to changes in the player role of one or more players in a single player mode or a multi-player mode, can adapt to changes in the background of the game environment, and can adapt to changes in the performance changes of the team members of the team. It is in this context that the embodiments of the present disclosure appear.
[0020] Thus, embodiments of the present disclosure provide a better gaming experience for users participating in a gaming application because suggestions are provided in the form of gaming assistance to help guide the user, rather than struggling through difficult portions of the gaming application or exiting the gaming application. Additionally, other embodiments provide the above achievements and also provide more efficient development of gaming applications by using analytics collected from the gaming sessions of users participating in the gaming application. Feedback provided to game developers identifies where users lose interest in the gaming application (e.g., through difficulty, boredom, etc.), and developers can use this feedback to create new gaming applications that avoid these problems.
[0021] Throughout the specification, the term game application describes an application that allows interaction with a user. In one implementation, the game application includes a video game. Embodiments of the present disclosure provide game assistance to users who participate in the game application. Other embodiments of the present disclosure provide assistance to users who interact with any type of application, such as word processing, computer-aided design, etc.
[0022] Throughout the specification, the terms user and player are used interchangeably and are intended to describe a person who participates in a game application or any other type of application. In some cases, embodiments of the present disclosure are described in the context of a "user" playing a game application within the game cloud system 210 and one or more "players" playing one or more game applications separately within the game cloud system 210, where the user may or may not interact with other players (e.g., single player mode or multi-player mode).
[0023] With the above general understanding of various embodiments in mind, exemplary details of various embodiments will now be described with reference to the various figures.
[0024] Figure 1A A system 105A is shown for providing personal assistance to a user playing a game application according to an embodiment of the present disclosure, wherein the game application is executed locally by the corresponding user, and wherein a backend server support (e.g., accessible via a game server 205) can provide analysis of game play data of one or more users for the purpose of providing game assistance to the corresponding user playing the game application.
[0025] like Figure 1AAs shown, multiple users 115 (e.g., user 115A, user 115B, ... user 115N) are playing multiple game applications, each of which is locally executed on the corresponding client device 100 (e.g., game console) of the corresponding user. Each client device 100 can be configured similarly because the local execution of the corresponding game application is executed. For example, user 115A can play a first game application on the corresponding client device 100, where the instance of the first game application is executed by the corresponding game name execution engine 130A. The game logic 117A (e.g., executable code) that implements the first game application is stored on the corresponding client device 100 and is used to execute the first game application. For the purpose of illustration, the game logic 117A can be delivered to the corresponding client device 100 via a portable medium (e.g., a flash drive, an optical disk, etc.) or via a network (e.g., downloaded from a game provider via the Internet 150). In addition, user 115B is playing a second game application on the corresponding client device 100, where the instance of the second game application is executed by the corresponding game name execution engine 130A. The second game application may be the same as or a different game application from the first game application executed for user 115A. The game logic 117B (e.g., executable code) that implements the second game application is stored on the corresponding client device 100 as described above, and is used to execute the second game application. In addition, user 115N is playing the Nth game application on the corresponding client device 100, wherein the instance of the Nth game application is executed by the corresponding game name execution engine 130A. The Nth game application may be the same as the first game application or the second game application, or may be a completely different game application. The game logic 117C (e.g., executable code) that implements the third game application is stored on the corresponding client device 100 as described above, and is used to execute the Nth game application.
[0026] For purposes of clarity and illustration, embodiments of the present disclosure may be described with reference to user 115A, where user 115A is representative of any user in plurality of users 115 .
[0027] The client device 100 may receive input from various types of input devices, such as a game controller, a tablet computer, a keyboard, gestures captured by a video camera, a mouse touchpad, etc. The client device 100 may be any type of computing device having at least a memory and a processor module that is capable of connecting to the game server 205 via the network 150. Some examples of the client device 100 include a personal computer (PC), a game console, a home theater device, a general purpose computer, a mobile computing device, a tablet computer, a phone, or any other type of computing device that can interact with the game server 205 to provide backend functions (such as game assistance or multi-player session management, etc.).
[0028] The corresponding user's client device 100 is configured to generate a rendered image executed by the locally executed game title execution engine 130A, and to display the rendered image on a display. For example, the rendered image may be associated with an instance of a first game application executed on the user 115A's client device 100. Specifically, the corresponding client device 100 is configured to interact with the locally executed instance of the corresponding game application to enable the corresponding user's game play, such as through input commands for driving game play.
[0029] In one embodiment, the client device 100 operates in single player mode for a corresponding user of a particular gaming application being executed locally. As will be described more fully below, backend server support via the game server 205 may provide gaming assistance to the user, such as through the profiler engine 145 .
[0030] In another embodiment, multiple client devices 100 operate in a multi-player mode for corresponding users of each of the gaming applications being played. In this case, multi-player functionality may be provided, such as through a multi-player processing engine 119, via the back-end server support of the gaming server. Specifically, the multi-player processing engine 119 is configured to control a multi-player gaming session for a particular gaming application. For example, the multi-player processing engine 130 communicates with a multi-player session controller 116, which is configured to establish and maintain a communication session with each user and / or player participating in the multi-player gaming session. In this manner, users in the session may communicate with each other under the control of the multi-player session controller 116.
[0031] In addition, the multi-player processing engine 119 communicates with the multi-player logic 118 to enable interaction between users within each user's corresponding game environment. Specifically, the state sharing module 117 is configured to manage the state of each user in the multi-player game session. For example, the state data may include game state data, which defines the state of the game play (of the game application) of the corresponding user at a specific point. For example, the game state data may include game characters, game objects, game object attributes, game attributes, game object states, graphic overlays, etc. In the manner described, the game state data allows the generation of a game environment that exists at a corresponding point in the game application. The game state data may also include the state of each device used to render the game play, such as the state of the CPU, GPU, memory, register values, program counter values, programmable DMA state, DMA buffer data, audio chip state, CD-ROM state, etc. The game state data may also identify which parts of the executable code need to be loaded to execute the video game from the point. The game state data may be stored in Figure 1C and Figure 2in the database 140 and can be accessed by the state sharing module 117.
[0032] In addition, the state data may include user-saved data including information that personalizes the video game for the corresponding player. This includes information associated with a user character so that the video game is rendered using a character that may be unique to the user (e.g., position, shape, appearance, clothing, weapons, etc.). In the manner described, the user-saved data enables the generation of a character for the corresponding user's game play, wherein the character has a state corresponding to a point in the game application currently experienced by the corresponding user. For example, the user-saved data may include game difficulty, game level, character attributes, character position, remaining number of lives, possible total number of available lives, armor, trophies, time counter values, etc. selected by the corresponding user 115 when playing the game. The user-saved data may also include, for example, user profile data that identifies the corresponding user 115. The user-saved data may be stored in the database 140.
[0033] In the manner described, the multiplayer processing engine 119 using the state sharing data 117 and the multiplayer logic 118 is able to overlay / insert objects and characters into each game environment of the users participating in the multiplayer game session. For example, a first user's character is overlaid / inserted into a second user's game environment. This allows interaction between users in the multiplayer game session via each of their respective (e.g., as displayed on a screen) game environments.
[0034] In addition, the back-end server support via the game server 205 can provide game assistance functions such as through the parser engine 145 and the AI personal assistant 120. Specifically, the parser engine 145 in the game cloud system 210 is configured to provide game assistance to one or more users who participate in multiple game applications such as in single player mode or multi-player mode. Typically, artificial intelligence (AI) personal assistant 120 works with the parsing engine 145 to provide game assistance to one or more users. In one embodiment, the AI personal assistant 120 is located to the client device of the corresponding user because it provides an interface to the user for the purpose of requesting and / or receiving personal assistance. In another embodiment, the AI personal assistant is away from the client device of the corresponding user. For example, in one embodiment, when it is determined that the user is inefficiently performing and / or struggling in his or her game, assistance is provided to a single player who is performing the game application in single player mode. When requested or when it is determined that the player needs assistance, assistance can also be provided to the player who is performing in a multi-player game session. In another embodiment, assistance is provided to a team of members playing a gaming application in a multiplayer mode to increase the probability that the team will successfully complete a goal, such as completing a task or set of subtasks or actions, winning a game against other teams in a competitive arena, completing the gaming application, etc.
[0035] Specifically, the AI personal assistant 120 is configured to work closely with the client device 100 of the corresponding user 115 to collect data about the current and past game progress of the corresponding user 115. In addition, the AI personal assistant 120 is configured to provide an interface to the corresponding user 115 so that assistance can be provided. For example, suggestions generated by the profiler engine 145 can be provided through the interface, wherein the suggestions provide assistance during the current game progress of the corresponding user 115. In one embodiment, the AI personal assistant 120 can be configured locally within the client device 100 of the corresponding user 115. In addition, in another embodiment, the located AI personal assistant 120 can be configured to be away from the client device 100 and independent of the game cloud system 210. For example, the AI personal assistant 120 can be accessed through a network 150 (e.g., a local area network, the Internet, etc.). In another embodiment, the AI personal assistant 120 can be configured to be away from the client 100 in the game cloud system 210. In the case, the personal assistant 120 can be directly coupled to the profiler engine 145 within the game cloud system 210.
[0036] As shown in the figure, the profiler engine 145 is configured to provide the back-end server processing of game assistance. Specifically, by the game cloud system 210, the profiler engine 145 can collect and analyze the game of multiple users of multiple game applications when the game cloud system 210 is executing one or more game applications for multiple users 115. For example, when the corresponding user 115A is performing any game application, the profiler engine 145 can collect and analyze user data. Over time, the profiler engine 145 can generally and / or for a specific type of game application to know the game style of the corresponding user 115A. In addition, the profiler engine 145 can be based on user data, the game rules 125 of one or more game applications and the game logic 126 of one or more game applications to rate the proficiency of the corresponding user 115A. In one embodiment, proficiency is given for a specific game application and based on the current game of the user participating in the game application, the past game of the user participating in the game application, and the historical game of the user participating in the game application of the same type (e.g., game style) can be considered. In another embodiment, proficiency is given for one type of gaming application and is based on the game play of users participating in the gaming application, and historical game play of users participating in gaming applications of the same type (e.g., game style) may be considered. In yet another embodiment, proficiency is given for one or more types of gaming applications and is based on the game play of users participating in the gaming application, and historical game play of users participating in gaming applications of the same type (e.g., game style) may be considered. The proficiency rating for each of the plurality of users 115 is included in a corresponding gaming profile stored in the user account 199 database.
[0037] Therefore, the profiler engine 145 can rate the proficiency of the corresponding user based on the game progress of the corresponding user and compare it with the proficiency and game profile of other users, and build a game profile for each of the multiple users 115. When collecting more data from the game progress of multiple users, the profiler engine 145 can learn (for example, by applying deep learning or artificial intelligence technology) how to carry out a specific game application (usually or in the context of a given level, sub-level or given problem) or how successful users and / or other unsuccessful users are carrying out a specific game application (for example, learned from errors). The way in which the game application should be carried out as learned by the analyzer engine 145 will develop over time because more data is collected about the game progress of the user carrying out the game application. In the described manner, in particular, when it is detected that a single user 115A carrying out the game application is stuck in the game progress or inefficiently carrying out the game in his or her game progress, suggestions can be made to the user 115A. In addition, the user proficiency rating can be used to predict the successful completion of the goal by the member team. It may be suggested that tasks be assigned in a manner that maximizes the probability of successfully completing the team's goals and is based on the proficiency rating of each team member, the tasks involved, and the likely gaming environment.
[0038] Figure 1B A system for providing personal assistance to a user of a gaming application executed on a cloud gaming network according to one embodiment of the present disclosure is presented. In some embodiments, the cloud gaming network may be a gaming cloud system 210, the gaming cloud system comprising a plurality of virtual machines (VMs) running on a host's hypervisor, wherein one or more virtual machines are configured to utilize hardware resources available to the host's hypervisor to execute a gaming processor module.
[0039] As shown, the game cloud system 210 includes a game server 205, which provides access to multiple interactive video games or game applications. The game server 205 can be any type of server computing device available in the cloud, and can be configured as one or more virtual machines executed on one or more hosts. For example, the game server 205 can manage a virtual machine that supports a game processor, and the game processor instantiates an instance of a game application for a specific user 115A. Therefore, the multiple game processors of the game server 205 associated with multiple virtual machines are configured to execute multiple instances of game applications associated with the game play of multiple users 115.
[0040] As shown, multiple users 115 access the game cloud system 210 via a network 150, wherein the user 115 accesses the network 150 via a corresponding client device 100. For example, the client device 100 of the corresponding user 115A is configured to request access to a game application via a network 150 such as the Internet, and is used to render an instance of a game application (e.g., a video game) executed by a game server 205 and transmit it to a display device associated with the corresponding user 115A. For example, the user 115A can interact with an instance of a game application executed on a corresponding game processor of the game server 205 via the client device 100. More specifically, the instance of the game application is executed by a game name execution engine 130B. The game logic (e.g., executable code) that implements a specific game application can be stored and accessed by a data storage area 140, and is used to execute the game application. In one embodiment, one or more game name execution engines 130B can be configured to support multiple instances of multiple game applications. Therefore, as shown, the game name execution engine 130B can use multiple game logics 177 to support multiple game applications.
[0041] The client device 100 may receive input from various types of input devices, such as a game controller, a tablet computer, a keyboard, gestures captured by a video camera, a mouse, a touchpad, etc. As previously described, the client device 100 may be any type of computing device having at least a memory and a processor module that is capable of connecting to the game server 205 via the network 150. Some examples of the client device 100 include a personal computer (PC), a game console, a home theater device, a general purpose computer, a mobile computing device, a tablet computer, a phone, or any other type of computing device that can interact with the game server 205 to execute an instance of a game application.
[0042] The client device 100 of the user 115A is configured to receive the rendered image delivered by the game server 205, and to display the rendered image on the display. For example, the rendered image may be associated with an instance of a game application executed on the game server 205 associated with the corresponding user 115A. Specifically, the client device 100 is configured to interact with the instance of the game application associated with the game play of the user 115A, such as through input commands for driving the game play.
[0043] The profiler engine 145 in the game cloud system 210 is configured to provide game assistance to one or more users participating in one or more game applications in single-player mode or multi-player mode. Specifically, the artificial intelligence (AI) personal assistant 120 works with the profiler engine 145 to provide game assistance to one or more users. For example, in one embodiment, when it is determined that the user is inefficiently performing and / or struggling in his or her game, assistance is provided to a single player who is playing a game application in single-player mode or multi-player mode. In another embodiment, assistance is provided to a member team that is playing a game application in multi-player mode to increase the probability of success that the team will complete the goal, such as completing a set of tasks or sub-tasks or actions, winning a competition against other teams in a competitive arena, completing a game application, etc.
[0044] Specifically, the AI personal assistant 120 is configured to work closely with the client device 100 of the corresponding user 115A to collect data about the current and past game progress of the corresponding user 115A. In addition, the AI personal assistant 120 is configured to provide an interface to the corresponding user 115A so that assistance can be provided. For example, suggestions generated by the profiler engine 145 can be provided through the interface, wherein the suggestions provide assistance during the current game progress of the corresponding user 115A. In one embodiment, the AI personal assistant 120 can be configured locally in the client device 100 of the corresponding user 115A. In addition, in another embodiment, the located AI personal assistant 120 can be configured to be away from the client device 100 and independent of the game cloud system 210. For example, the AI personal assistant 120 can be accessed through the network 150 (e.g., a local area network, the Internet, etc.). In another embodiment, the located AI personal assistant 120 can be configured to be away from the client 100 in the game cloud system 210. In the case, the personal assistant 120 can be directly coupled to the profiler engine 145 in the game cloud system 210.
[0045] As shown in the figure, profiler engine 145 is configured to provide the back-end server processing of game assistance. Specifically, by game cloud system 210, profiler engine 145 can collect and analyze the game of multiple users 115 of multiple game applications when game cloud system 210 is executing the instance of game application for user 115 in particular. For example, when corresponding user 115A is performing any game application, profiler engine 145 can collect and analyze user data. Over time, profiler engine 145 can generally and / or for the game application of a specific type to know the game style of corresponding user 115A. In addition, profiler engine 145 can grade the proficiency of corresponding user 115A based on user data, the game rules 125 of one or more game applications and the game logic 126 of one or more game applications. In one embodiment, the proficiency of user 115A is given for a particular game application and is based on the current game play of user 115A participating in the game application, the past game play of user 115A participating in the game application, and the historical game play of user 115A participating in game applications of the same type (e.g., game style) may be considered. In another embodiment, the proficiency of user 115A is given for a type of game application and is based on the game play of user 115A participating in the game application, and the historical game play of user 115A participating in game applications of the same type (e.g., game style) may be considered. In yet another embodiment, the proficiency of 115A is given for one or more types of game applications and is based on the game play of user 115A participating in the game application, and the historical game play of user 115A participating in game applications of one or more types (e.g., evidence of an overall game style) may be considered. The proficiency rating of each of the plurality of users 115 is included in a corresponding game profile stored in the user account 199 database.
[0046] Therefore, the profiler engine 145 can be based on the game progress of the corresponding user 115A and with the proficiency and game profile comparison of other users to rate the proficiency of the corresponding user 115A, and build a game profile for the user 115A. When collecting more data from the game progress of multiple users, the profiler engine 145 can know (for example, by applying deep learning or artificial intelligence technology) how to carry out a specific game application (usually or in the context of a given grade, subgrade or given problem) or how a successful user and / or other unsuccessful users are carrying out a specific game application (for example, knowing from an error). The mode of carrying out the game application as known by the profiler engine 145 will develop over time, because more data is collected about the game progress of the user carrying out the game application. In the described manner, in particular, when it is detected that a single user 115A carrying out the game application is stuck in the game progress or inefficiently carrying out the game in his or her game progress, suggestions can be made to the user 115A. In addition, the user proficiency rating can be used to predict that the member team successfully completes the goal. It may be suggested that tasks be assigned in a manner that maximizes the probability of successfully completing the team's goals and is based on the proficiency rating of each team member, the tasks involved, and the likely gaming environment.
[0047] In another embodiment, the previously described multi-player processing engine 119 provides control for a multi-player game session for a gaming application. Specifically, when the multi-player processing engine 119 manages a multi-player game session, the multi-player session controller 116 is configured to establish and maintain a communication session with each user and / or player in the multi-player session. In this manner, the users in the session can communicate with each other under the control of the multi-player session controller 116.
[0048] In addition, the multi-player processing engine 119 communicates with the multi-player logic 118 so as to enable interaction between users in the corresponding game environment of each user. Specifically, the state sharing module 117 is configured to manage the state of each user in the multi-player game session. For example, as previously described, the state data may include game state data, which defines the state of the game (of the game application) of the corresponding user 115A at a specific point. In addition, as previously described, the state data may include data saved by the user, which includes information for personalizing the video game for the corresponding player. For example, this includes information associated with the user's role, so that the video game is rendered using a role (e.g., shape, appearance, clothing, weapons, etc.) that may be unique to the user. In the described manner, the multi-player processing engine 119 using the state sharing data 117 and the multi-player logic 118 can superimpose / insert objects and roles into each game environment of the user participating in the multi-player game session. This allows the users in the multi-player game session to interact via each of their respective (e.g., as displayed on the screen) game environments.
[0049] Figure 1C An automatic artificial intelligence (AI) personal assistance system 129 according to one embodiment of the present disclosure is shown, which includes Figure 1A-1B The AI personal assistant system 129 is configured to provide game assistance to one or more users playing a game application in a single-player mode or a multi-player mode based on the proficiency of the corresponding one or more users compared to the game play of other players playing the game application. As previously described, as shown in the figure, the AI personal assistant 120 and the profiler engine 145 of the system 129 work together to provide game assistance and can be located together or remotely from each other. In one embodiment, in combination, each user is supported by a separate AI personal assistant system 129, which includes at least a dedicated AI personal assistant 120 supported by a backend profiler engine 145. For example, Figure 1C As shown, user 115S is supported by AI personal assistant 120A, which communicates with user game play profiler 145A of profiler engine 145. Similarly, another user (not shown) may be supported by AI personal assistant 120B, which communicates with user game play profiler 145A of profiler engine 145. Yet another user (not shown) may be supported by AI personal assistant 120C, which communicates with user game play profiler 145A of profiler engine 145.
[0050] As shown, the AI personal assistant system 129 is configured to monitor the game play of a user playing a game application. Specifically, the AI personal assistant 120 is configured to monitor the game play of the user and collect user data that can be used for artificial intelligence (AI) purposes. During the game play, the current user has a defined task 301 to complete. As shown below Figure 3 As further described in, as indicated by the hash mark in task 301, the task may include a single action or subtask or multiple actions or subtasks. In addition, the task may not include an action or subtask. In addition, the task is defined by the task type so that similar tasks can be grouped under a single task type. These tasks may have similar goals, perform similar actions or subtasks to complete the task, or share other similarities. By grouping tasks under the task type, the game play of the user who processes the task type can be analyzed to learn the appropriate actions to be taken, the actions to be avoided, the best style of game play, and the most effective style of game play, for successfully completing those tasks for a specific game application, for a type of game application, or for a general game application.
[0051] In another embodiment, the user's game play is monitored by the AI personal assistant 120. That is, multiple game plays of the user playing multiple game applications are monitored. The game play includes multiple tasks to be processed and / or completed. Each task monitored is defined by a corresponding task type. In this manner, tasks of the same type can be analyzed to determine the user's gaming style, gaming habits, and proficiency, especially when processing the current task. The information helps determine the type of assistance (if any) and degree of assistance that the user needs and / or wants when processing the current task of a particular task type.
[0052] Thus, for each response made by the user 115A of the client device 100 to process a task or subtask or action, the action state 121, action metrics 122, and action results 123 are collected by the corresponding AI personal assistant 120 (locally or remotely located). Responses may be generally defined and include strategic decisions made by the user 115A (e.g., attack now vs. later, collect coins diligently or poorly, etc.), approaches taken during game play (e.g., go to city 1 vs. go to city 2), choices made (equip armor 1 or armor 2). Additionally, responses may be defined to be task specific, such as how the user 115A processes actions or subtasks (e.g., in what order, successfully complete subtasks, etc.).
[0053] Specifically, action state 121 defines an action, subtask or task processed in the background or game environment of a game application. The game environment will help determine which action type describes the task. For example, depending on the game environment, certain actions may be classified differently. When the game environment is relaxed (e.g., usually traversing from one location to another), the action may be classified under a first type, but when the game environment is tense (e.g., fighting a boss), it may be classified under a second type. In addition, action metrics 122 are collected by corresponding AI personal assistants 125. These metrics are associated with the responses taken by user 115A when processing tasks, subtasks or actions. In addition, action results 123 are collected by corresponding AI personal assistants 125. For example, result 123 includes whether user 115A successfully completes a task, subtask or action. In addition, result 123 may define how well user 115A performs in terms of processing tasks, subtasks or actions (e.g., efficiency).
[0054] The collected data (e.g., action states 121, action metrics 122, and action results 123) are passed to the profiler engine 145 for further analysis (e.g., providing recommendations). Specifically, the collected data is passed to the user game play profiler 145A, which compares the data collected for the user 115A with the current and historical performance of the user 115A and the historical performance of other users when playing a game application, a type of game application, or a general game application, and particularly when processing tasks of a specific task type. In the manner described, the user game play profiler 145A is able to build a game profile for each user, as identified by a corresponding player ID stored in a data storage area 143 within the game cloud system 210. The game profile is based on the corresponding user data, the game rules 125 of one or more game applications in one or more game applications, wherein the game rules 125 are stored in the data storage area 140.
[0055] In one embodiment, multiple game plays of multiple users are monitored by one or more AI personal assistant systems 129. Game plays are associated with multiple game applications. As previously described, game plays include multiple tasks to be processed and / or completed, each of which is defined by a corresponding task type. Therefore, as previously described, the collected data can be used to build game profiles for multiple users. In addition, task type proficiency rules and / or player proficiency scores are determined based on the results (e.g., the performance of the players) of multiple players processing multiple tasks of the task type.
[0056] Specifically, the game profiles 131A-N are created by the user game play profiler 145. For illustrative purposes, the game profile may include the historical performance of user A when handling multiple tasks. Each task may be assigned a performance rating. For example, the game profile 131A of user 115A shows a rating of 1 for task / subtask / action 1, a rating of 2 for task / subtask / action 2, a rating of 3 for task / subtask / action 3, and the like. As described below, these ratings may be used to determine the overall proficiency of user 115A when handling any task of any type, or to determine the proficiency of user 115A when handling a particular type of task.
[0057] The data in the game profiles 131A-N may be fed back to the deep learning engine 146 of the user game play profiler 145. The deep learning engine 146 utilizes artificial intelligence, including deep learning algorithms, reinforcement learning, or other artificial intelligence-based algorithms. Figure 1D A discussion about deep learning is provided. In the manner described, analysis of collected data can be continuously performed to provide updated analysis for providing assistance to users. For example, task type proficiency rules can be upgraded to reflect new data.
[0058] In addition, the collected data can be used to determine the proficiency (e.g., user proficiency score) of the user when processing a task of a specific task type. Specifically, the task type is further defined by a task type proficiency rule, wherein the proficiency of the user is determined based on the task type proficiency rule. The rule is determined based on the results of multiple users of multiple tasks of the task type. In other words, the data (e.g., action state 121, action measurement 122, and action result 123) of multiple users of multiple tasks of the task type in one or more game applications are analyzed to generate the task type proficiency rule. For example, the rule can define the average response to the task of the task type, wherein the average response includes the average time spent to complete the task, or the average number of commands required to complete the task, etc. The rule can be adjusted to be more aggressive or less aggressive, so that the more aggressive rule defines a response that is better than the average user and / or player, and the less aggressive rule defines a response that is worse than the average user.
[0059] In one embodiment, a user's proficiency at handling a particular task (e.g., a user proficiency score) may be determined for a particular game application and based on the user's current playthrough of the game application, the user's past playthrough of the game application, and may take into account the user's historical playthrough of game applications of the same type (e.g., play style) or multiple types (e.g., overall play styles). For example, a user proficiency score may be based on the user's past performance when encountering multiple tasks facing the user. In another embodiment, a user proficiency score may be based on the user's current performance when encountering a task, where only the user's playthrough in this particular game application is considered. In one embodiment, a user proficiency score is based on the user's past performance when encountering multiple tasks of the task type, and based on the user's current performance when encountering the current task.
[0060] In another embodiment, proficiency is given for a type of gaming application and is based on the game play of users participating in the gaming application, and historical game play of users participating in gaming applications of the same type (e.g., game style) may be considered. In yet another embodiment, proficiency is given for any task of a task type that handles multiple gaming applications of one or more types and is based on the game play of users participating in the gaming application, and historical game play of users participating in gaming applications of the same type (e.g., game style) may be considered.
[0061] When determining the future proficiency score, the current proficiency score is also included in the user's past performance. That is, the user's current performance is stored and considered when determining the future proficiency score.
[0062] The deep learning engine 146 of the user game progress analyzer 145A can collect data 141 (e.g., Internet data) through social networks to identify problems within the game application, game progress difficulties, etc. For example, a user may post to a game blog asking questions about how to beat a specific part of the game application. The deep learning engine 146 can use the data to identify difficult tasks to build a database, including various methods of tasks performed by multiple players and their relative success and / or failure in handling those tasks. In addition, as previously described, the user game progress analyzer 145A of the game name processing engine 130 is configured to analyze the communications 142 of a specific user or communications between users to identify problems within the game application, game progress difficulties, etc.
[0063] Additionally, the user game play profiler 145A can determine a user's predicted success rate for completing a task based on the user's proficiency score, given task type proficiency rules, and the task presented to the user. The predicted success rate may take into account a given gaming environment that the user experiences.
[0064] Therefore, the suggestion engine 135 is configured to determine suggestions for the user based on the user's predicted success rate for the current task. That is, suggestions are generated based on the user's proficiency, wherein the proficiency may take into account one or more of the following: the user's performance during the current game play, the user's historical performance in past game plays associated with multiple game applications of general or specific types, and the user's game style associated with game applications of general or specific types. In one embodiment, the suggestions include suggested actions or responses for the user that help complete the task. The suggestion engine 135 passes one or more suggestions to the positioned AI personal assistant 125 displayed on the user client device 100. For example, the suggestions may include action suggestions 124.
[0065] In one embodiment, the user game progress analyzer 145A is configured to provide assistance to a member team that is given a set of tasks to be completed. Because the user game progress analyzer 145A is able to determine and / or access the game progress profile of each team member, the information in the profile can be used to determine how to best utilize the team members to successfully complete the task set, and more importantly, assign tasks to team members so that the tasks are completed and / or assigned to team members in the most efficient manner to maximize the success probability of completing related goals (e.g., completing the task set, winning the competition, completing the game application, etc.). In other words, the user game progress analyzer 145A is configured to provide game assistance based on a predictive model that determines the success probability of the team when completing the task set. The success probability is determined for each task combination assigned to the team member, wherein the success probability is determined based on the member's game profile, the task set, and the current game environment.
[0066] Figure 1D An exemplary neural network 190 is shown for training a deep learning engine 146 to support one or more modules of an AI personal assistance system 120 according to one embodiment of the present disclosure. For example, the neural network 190 represents an example of an automated analysis tool for analyzing a data set used by a user game play profiler 145A to determine game play profiles for a plurality of users 115. Specifically, instances of the neural network 190 may be used to train the user game play profiler 145A to determine game play profiles for a plurality of users 115 and / or to provide recommendations during a particular user's game play.
[0067] Different types of neural networks 190 are possible. In an example, the neural network 190 supports deep learning that can be implemented by the deep learning engine 146. Thus, a deep neural network, a convolutional deep neural network, and / or a recurrent neural network using supervised or unsupervised training can be implemented. In another example, the neural network 190 includes a deep learning network that supports reinforcement learning. For example, the neural network 190 is configured as a Markov decision process (MDP) that supports a reinforcement learning algorithm.
[0068] In general, neural network 190 represents a network of interconnected nodes, such as an artificial neural network. Each node learns some information from the data. Knowledge can be exchanged between nodes through interconnections. An input to neural network 190 activates a set of nodes. In turn, this set of nodes activates other nodes, thereby propagating knowledge about the input. The activation process is repeated on other nodes until an output is provided.
[0069] As shown, neural network 190 includes a hierarchical structure of nodes. At the lowest hierarchical level, there is an input layer 191. Input layer 191 includes a set of input nodes. Each of these input nodes is mapped to a user game play profile feature (e.g., a feature specific to a user's interaction with a video game, a user playing a video game, a video game, etc.).
[0070] At the highest hierarchical level, there is an output layer 193. The output layer 193 includes a set of output nodes. For example, the output nodes represent decisions related to one or more components of the user's game play profiler. As previously described, the output nodes can identify problems, difficult tasks, methods for completing specific tasks within the game application, other game play difficulties within the game application, thereby predicting the user's success rate for specific types of tasks or specific tasks, predicting the success rate of a team of members given a set of tasks to complete, determining proficiency scores for users and / or teams, determining recommendations, providing assistance, etc. The nodes in the neural network 190 learn the parameters of the model that can be used to make such decisions.
[0071] The hidden layer 192 exists between the input layer 191 and the output layer 193. The hidden layer 192 includes "N" number of hidden layers, where "N" is an integer greater than or equal to 1. Furthermore, each hidden layer also includes a group of hidden nodes.
[0072] The input nodes are interconnected with the hidden nodes. Likewise, the hidden nodes are interconnected with the output nodes. However, the input nodes are not directly interconnected with the output nodes. If there are multiple hidden layers, then the input nodes are interconnected to the hidden nodes of the lowest hidden layer. In turn, these hidden nodes are interconnected to the hidden nodes of the next hidden layer, and so on. The hidden nodes of the next highest hidden layer are interconnected with the output nodes.
[0073] The interconnect connects two nodes. The interconnect has a numerical weight that can be learned. (e.g., based on the user's game play analyzer, suggested data sets, etc.) so that the neural network 190 is adaptable to the input and can be learned.
[0074] Typically, the hidden layer 192 allows knowledge about the input nodes to be shared between all tasks corresponding to the output nodes. To this end, a transformation f is applied to the input nodes by the hidden layer 192. In the example, the transformation f is nonlinear. Different nonlinear transformations f are available, including, for example, a rectifier function f(x)=max(0,x).
[0075] The neural network 190 also uses a cost function c to find the optimal solution. The cost function measures the deviation between the prediction defined as f(x) output by the neural network 190 for a given input x and the ground truth or target value y. The optimal solution represents the case where there is no solution with a cost lower than the cost of the optimal solution. For such data where ground truth labels are available, an example of a cost function is the mean squared error between the prediction and the ground truth. During the learning process, the neural network 190 can use a back-propagation algorithm to employ different optimization methods to learn the model parameters that minimize the cost function. An example of such an optimization method is stochastic gradient descent.
[0076] In an example, the training data sets for neural network 190 are from the same data domain. For example, neural network 190 is trained for a specific gaming application. In the illustration, the data domain includes game session data collected for a player's interaction with a video game. In another example, the training data sets are from different data domains. For example, neural network 190 is trained for one type of gaming application.
[0077] Thus, neural network 190 may define a prediction model for predicting a user's success rate for a particular type of task or a particular task based on the user's proficiency score and the task type proficiency rules. In addition, neural network 190 may define a prediction model for predicting the success rate of a team of members given a set of tasks to complete. Based on these prediction results, one or more recommendations may be provided to the user or team of members.
[0078] Figure 2A system diagram 200 for enabling access and playing of a video game stored in a game cloud system (GCS) 210 according to an embodiment of the present disclosure is shown. In general, the game cloud system GCS 210 may be a cloud computing system operating on a network 220 to support multiple users. In addition, the GCS 210 is configured to provide assistance to one or more users who are playing a game application in a single player mode or a multi-player mode using a game title processing engine 130 and a positioned AI personal assistant 120. In addition, the game title processing engine 130 and one or more AI personal assistants 120 (local or remote) of the GCS 210 are configured to provide game assistance to a member team, so that tasks or subtasks or actions are assigned to members to increase the probability of successfully completing a goal (e.g., completing a task set, winning a game competition, winning a game application, etc.). Specifically, the system 200 includes a GCS 210, one or more social media providers 240, and a user device 230, all of which are connected via a network 220 (e.g., the Internet). One or more user devices may connect to the network 220 to access services provided by the GCS 210 and the social media provider 240 .
[0079] In one embodiment, the game cloud system 210 includes a game server 205, a video recorder 271, a tag processor 273, and an account manager 274 including a user profile manager, a game selection engine 275, a game session manager 285, a user access logic 280, a network interface 290, and a social media manager 295. The GCS 210 may also include multiple game storage systems, such as a user game profile storage area, a game logic storage area, a game application rule storage area, a state sharing storage area, an Internet data storage area, a game state storage area, a random seed storage area, a user save data storage area, and a snapshot storage area, which can be stored in the data storage area 140. Other game storage systems may include a game code storage area 261, a recorded game storage area 262, a tag data storage area 263, a video game data storage area 264, and a game network user storage area 265. In one embodiment, the GCS 210 is a system that can provide game applications, services, game-related digital content, and interconnections between systems, applications, users, and social networks. GCS 210 can communicate with user device 230 and social media provider 240 through social media manager 295 via network interface 290. Social media manager 295 can be configured to associate one or more friends. In one embodiment, each social media provider 240 includes at least one social graph 245 showing the user's social network connections.
[0080] User U0 can access services provided by GCS 210 via game session manager 285, wherein user U0 may represent user 5 of FIG. 1 . For example, account manager 274 enables authentication and access to GCS 210 by user U0. Account manager 274 stores information about member users. For example, the user profile of each member user may be managed by account manager 274. In one implementation, account manager 274 may be configured to manage the game profiles of member users. For example, member information may be used by account manager 274 for authentication purposes. For example, account manager 274 may be used to update and manage user information associated with member users. In addition, game titles owned by member users may be managed by account manager 274. In the manner described, video games stored in data storage area 264 may be used for any member user who owns those video games.
[0081] In one embodiment, a user (e.g., user U0) can access services provided by GCS 210 and social media provider 240 through a user device 230 via a connection on network 220. User device 230 may include any type of device having a processor and memory, wired or wireless, portable or non-portable. In one embodiment, user device 230 may be in the form of a smartphone, tablet computer, or a hybrid that provides touch screen capabilities in a portable form factor. An exemplary device may include a portable phone device that runs an operating system and is provided with access to various applications (apps) that can be obtained through network 220 and executed on a local portable device (e.g., smartphone, tablet computer, laptop computer, desktop computer, etc.).
[0082] The user device 230 includes a display 232 that acts as an interface for the user U0 to send input commands 236 and display data and / or information 235 received from the GCS 210 and social media providers 240. The display 232 may be configured as a touch screen or a display typically provided by a flat panel display, a cathode ray tube (CRT), or other device capable of rendering a display. Alternatively, similar to a desktop or laptop computer, the user device 230 may have its display 232 separated from the device.
[0083] In one embodiment, the user device 130 is configured to communicate with the GCS 210 to enable the user U0 to play a video game or game application. For example, the user U0 can select a video game available in the video game data storage area 264 via the game selection engine 275 (e.g., by game name, etc.). In the manner described, the selected video game is enabled and loaded for execution on the GCS 210 by the game server 205. In one embodiment, the game play is primarily executed in the GCS 210, so that the user device 230 will receive a stream of game video frames 235 from the GCS 210, and user input commands 236 for driving the game play are sent back to the GCS 210. The received video frames 235 from the streamed game play are displayed in the display 232 of the user device 230.
[0084] In one embodiment, after user U0 selects an available game title to play, user U0 may initiate a game session for the selected game title via the game session manager 285. The game session manager 285 first accesses the game state storage area in the data storage area 140 to retrieve the saved game state (if any) of the last session played by user U0 (for the selected game) so that user U0 can restart the game play from the previous game play stop point. Once the resume or start point is identified, the game session manager 285 may notify the game execution engine in the game processor 210 to execute the game code for the selected game title from the game code storage area 261. After initiating the game session, the game session manager 285 may transmit the game video frames 235 (i.e., streaming video data) to a user device, such as user device 230, via the network interface 290. In another embodiment, the game session manager 285 may be independent of the network interface 290. Figure 1A The multi-player session controller 121 or in combination therewith provides multi-player session management.
[0085] During the game play, the game session manager 285 can communicate with the game processor 210, the recording engine 271 and the tag processor 273 to generate or save a record (e.g., video) of the game play or game play session. In one embodiment, the video record of the game play may include tag content entered or provided during the game play and other game-related metadata. The tag content may also be saved via snapshots. The video record of the game play, along with any game metrics corresponding to the game play, may be saved in the recorded game storage area 262. Any tag content may be saved in the tag data storage area 263.
[0086] During game play, the game session manager 285 can communicate with the game processor 204 to pass and obtain user input commands 236 for affecting the results of the corresponding game play of the video game. The input commands 236 entered by the user U0 can be sent from the user device 230 to the game session manager 285 of the GCS 210. The input commands 236, including input commands for driving game play, can include user interaction inputs, such as including tag content (e.g., text, images, video recording clips, etc.). The game input commands and any user game play metrics (length of time the user plays the game, etc.) can be stored in the game network user storage area 165. Selecting information related to the game play of the video game can be used to make multiple features available to the user. For example, in various embodiments, data related to the proficiency of the corresponding user participating in the type of video game or game application or general game application can be generally collected and stored in the game network user storage area 265. Because the game play is performed by multiple users on the GCS 210, a game profile can be generated for each user. In the manner described, a user's game play may be compared to the game play of multiple users to determine how well the user is playing a video game and whether the user may benefit from suggestions provided by the game title processing engine 130 and the located AI personal assistant 120. Assistance to users in single-player mode or multi-player mode may enhance the user's experience playing a gaming application or video game by facilitating efficient game play and by helping the user through more difficult and / or impossible portions of the gaming application. In the case of user U0, the user device provides an interface that allows user U0 to receive communications from a backend server (e.g., the game title processing engine 130 of the GCS 210 that provides gaming assistance during user U0's game play).
[0087] Figure 3Tasks and subtasks that can be assigned to team members participating in a multiplayer game application according to an embodiment of the present disclosure are shown. As previously described, task assignment is determined based on a predictive model to provide the highest probability of success in completing a set of tasks given to team members. For example, the team consists of three members: user A, user B, and user C. Task A has been assigned to user A, where task A includes N subtasks. Each subtask may include one or more actions. For example, subtask 1 of task A includes three actions (as shown by three hash marks), subtask 2 includes one action, subtask 3 includes 4 actions, ..., and subtask N includes 3 actions. As shown in the figure, subtasks may include one or more actions, but some subtasks may not include actions. In addition, task B has been assigned to user B, where task B includes 2 subtasks: subtask 1 of task B includes 1 action, and subtask 2 of task B includes 5 actions. In addition, task C has been assigned to user C, where task C includes N subtasks. For example, subtask 1 of task C includes 3 actions, subtask 2 includes 1 action, subtask 3 includes 3 actions, ..., and subtask N includes 2 actions.
[0088] although Figure 3 Tasks and subtasks assigned to team members of a team are shown, but tasks assigned to one member of a team may represent tasks presented to users participating in a gaming application, where gaming assistance may be provided to users based on their proficiency as previously described in embodiments of the present disclosure.
[0089] According to one embodiment of the present disclosure, with a detailed description of various modules of a game server and a client device communicating over a network, now with respect to Figure 4 The flowchart 400 of the present invention describes a method for providing gaming assistance. The flowchart 400 shows the process and data flow involved in the operation of the artificial personal assistant system 129 for the purpose of assisting the game play of a user participating in a gaming application.
[0090] At 410, the method includes monitoring the game play of the user who is playing the game application, wherein the user has a defined task to be completed, and wherein the task is defined by a task type. In the described manner, the data related to the performance of the user are during the current game play of the game application. Over time and monitoring of multiple game plays of the user, the data related to the game style of the user associated with multiple game plays of the same game application, general or specific type, and the historical performance of the user during the past game play associated with multiple game plays of the same game application, general or specific type can be collected and analyzed subsequently. For historical game play collection, multiple game plays of the user are monitored, wherein the user participates in a specific game application and / or multiple game applications. The game play includes multiple tasks to be completed, each of which is defined by a task type. Another multiple tasks can be defined, each of which is defined by another task type.
[0091] In one embodiment, the task type is limited to the task itself, which may be further identified as a task type. Thus, the data collected and analyzed is based on the responses of the user and / or other players to a particular task presented during a game play of a game application of the user and other players. In another embodiment, the task type includes tasks encountered during multiple game plays of a game application. In yet another embodiment, the task type includes tasks encountered during multiple game plays of multiple game applications.
[0092] At 420, the method includes determining a task type proficiency rule for a task type based on the results of a plurality of players processing a plurality of tasks of the task type. For example, a game play of a player is monitored, wherein the player is playing a plurality of game applications. The game play includes a plurality of tasks to be completed, each task being defined by a task type. In the manner described, the task type proficiency rule defines a baseline for how a user should act or respond when processing a presented task. The task type proficiency rule may be specific to the presented task, or may be generally applicable to tasks of a task type. The task type proficiency rule may define a player proficiency score or a threshold score.
[0093] At 430, the method includes determining a user proficiency score for completing the task based on the task type proficiency rule. The user proficiency score may be based on the user's past performance when encountering multiple tasks in multiple game applications, where the tasks belong to the task type. In one embodiment, the user proficiency score may be based on the user's current performance when encountering the task, so that only data from the game play of the game application is considered. Additionally, in other embodiments, the user proficiency score may be based on a combination of the user's current performance when encountering the task and the user's past performance when encountering multiple tasks of the same task type in multiple game applications.
[0094] At 440, the method includes determining a predicted success rate for the user to complete the task based on the user proficiency score, the task type proficiency rule, and the task. The gaming environment may also be considered. For example, if the user's proficiency is poor and the task is very difficult, the predicted success rate may be low (e.g., less than 25%); however, if the user's proficiency is high, particularly if the task is of moderate difficulty, the predicted success rate may be high (e.g., greater than 70%).
[0095] At 450, the method includes determining a recommendation for the user based on the user's predicted success rate. Similarly, following the example of a user with poor proficiency being presented with a difficult task above, recommendations may be presented to the user. The recommendations may include opinions on how the user should respond when processing a task, subtask, or action. For example, the recommendations may include suggested responses to a set of commands or actions that facilitate completion of the task. The recommendations are customized according to the user's proficiency. For example, the recommendations given to a user with low proficiency may be very detailed (e.g., step-by-step instructions), while the recommendations given to a user with high proficiency may be very limited or very brief, with the purpose of only giving the user a hint, or may not include recommendations.
[0096] In addition, the suggestions given may be controlled by a sensitivity factor received from the user, wherein the sensitivity factor indicates whether the user is interested in receiving suggestions and may range from extremely enthusiastic, indifferent, mildly opposed, extremely hostile, etc. For example, if the user is highly sensitive, little or no suggestions may be given. On the other hand, if the user is insensitive, indicating a desire to receive suggestions, more suggestions may be given to the requesting user.
[0097] Figure 5 is a flow chart illustrating steps in a method for assigning tasks to team members of a team participating in a multiplayer gaming application based on a predictive model that assesses the success of the team in completing presented tasks based on past performance of team members when performing the same type of tasks, according to one embodiment of the present disclosure. That is, gaming assistance is provided to a team of members playing a gaming application in a multiplayer gaming environment. Assistance is generated and provided based on a predictive model that determines the probability of success of the team in completing a set of tasks for various combinations of task assignments to team members. The predictive model takes into account the members' gaming profiles, the set of tasks, and the current gaming environment.
[0098] At 510, the method includes monitoring a plurality of game sessions of a plurality of players playing a plurality of game applications. The plurality of game sessions include a plurality of tasks to be completed, and the plurality of tasks are associated with a plurality of task types. In the manner described, a game profile for a player may be generated based on the performance of a corresponding player associated with a game application or a plurality of game applications, the historical performance of the corresponding player. In addition, a game style may be determined for the player. Furthermore, a proficiency rating of the player may be determined based on the performance of other players when encountering tasks of a task type or tasks of one or more task types.
[0099] At 520, the method includes registering a plurality of team members of the team. A gaming profile for each team member may be accessed.
[0100] At 530, the method includes determining a plurality of ratings associated with a plurality of task types for a plurality of team members. That is, determining user proficiency ratings for the team members. These ratings may be used to assign tasks to team members based on which member is best suited to complete a task.
[0101] At 540, the method includes assigning tasks from a related task set to a plurality of team members based on a plurality of ratings so that the predicted success rate of the team completing the task set is the highest. Specifically, for a plurality of task assignment combinations from the related task set, a predicted success rate for the team to complete the task set is determined. The predicted rate is determined based on the plurality of ratings, the task set, and the current conditions of the game environment of the game application. Subsequently, the tasks from the related task set may be assigned to the team members in the selected combination so that the corresponding predicted success rate of the team completing the task set is the highest.
[0102] The process may be performed when the set of tasks is changed or updated. For example, the set of related tasks may be redefined as a second set of related tasks to be completed, wherein the set of tasks changes based on a plurality of events (e.g., loss of a team member, passage of time, completion of tasks from the set of tasks, addition of a new team member, etc.). Tasks from the second plurality of tasks may be assigned to a plurality of team members based on a selected combination based on a plurality of ratings such that the predicted success rate of the team in completing the set of tasks is highest.
[0103] Although specific embodiments showing the provision of game assistance to one or more users participating in a game application in a single player mode or a multi-player mode have been provided, these embodiments are described by way of example and not limitation. Those skilled in the art who have read this disclosure will recognize additional embodiments that fall within the spirit and scope of this disclosure.
[0104] It should be noted that access services such as providing access to the game of the current embodiment delivered over a wide geographic area typically use cloud computing. Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as services on the Internet. Users do not need to be experts in the technical infrastructure in the "cloud" that supports them. Cloud computing can be divided into different services, such as infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS). Cloud computing services typically provide online public applications (such as video games) accessed from a web browser, while software and data are stored on servers in the cloud. The term cloud is used as a metaphor for the Internet based on how the Internet is depicted in a computer network diagram, and is an abstract concept of its hidden complex infrastructure.
[0105] A Game Processing Server (GPS) (or simply "game server") is used by game clients to play single-player and multi-player video games. Most video games played over the Internet are run via a connection to a game server. Typically, games use a dedicated server application that collects data from players and distributes the data to other players. This is more efficient and effective than a peer-to-peer arrangement, but it requires a separate server to host the server application. In another embodiment, the GPS establishes communication between players and their respective game playing devices to exchange information without relying on a centralized GPS.
[0106] Dedicated GPS are servers that run independently of the clients. Such servers are usually run on dedicated hardware located in a data center, which provides more bandwidth and dedicated processing power. Dedicated servers are the preferred method of hosting game servers for most PC-based multiplayer games. Massively multiplayer online games are run on dedicated servers, which are usually hosted by the software company that owns the game title, allowing them to control and update the content.
[0107] The user accesses the remote service using a client device, which includes at least a CPU, a display, and an I / O. The client device may be a PC, a mobile phone, a netbook, a PDA, etc. In one embodiment, the network executed on the game server identifies the type of device used by the client and adjusts the communication method used. In other cases, the client device uses a standard communication method (such as html) to access the application on the game server via the Internet.
[0108] Embodiments of the present disclosure may be practiced with various computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. The present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wired or wireless network.
[0109] It should be understood that a given video game may be developed for a specific platform and a specific associated controller device. However, when such a game is made available via a gaming cloud system as presented herein, a user may access the video game with different controller devices. For example, a game may have been developed for a gaming console and its associated controller, while a user may be accessing a cloud-based version of the game from a personal computer using a keyboard and mouse. In this case, an input parameter configuration may define a mapping of inputs that may be generated by a user's available controller devices (in this case, a keyboard and mouse) to acceptable inputs for execution of the video game.
[0110] In another example, a user may access the cloud gaming system via a tablet computing device, a touch screen smart phone, or other touch screen driven device. In this case, the client device and the controller device are integrated together in the same device, where the input is provided by a detected touch screen input / gesture. For such a device, an input parameter configuration may define a specific touch screen input corresponding to a game input of a video game. For example, a button, a directional key, or other type of input element may be displayed or overlaid during the running of a video game to indicate a position on a touch screen that a user may touch to generate a game input. Gestures such as swipes or specific touch actions in a specific direction may also be detected as game inputs. In one embodiment, guidance may be provided to the user, for example, before starting the game play of a video game, indicating how to provide input for the game play via a touch screen to adapt the user to the operation of the controls on the touch screen.
[0111] In some embodiments, the client device serves as a connection point for the controller device. That is, the controller device communicates via a wireless or wired connection with the client device to send input from the controller device to the client device. The client device may then process these inputs and then send the input data to the cloud gaming server via a network (e.g., accessed via a local network device such as a router). However, in other embodiments, the controller itself may be a networked device with the ability to communicate input directly to the cloud gaming server via a network without first communicating such input through a client device. For example, a controller may be connected to a local network device (such as the above-mentioned router) to send to and receive data from a cloud gaming server. Therefore, while the client device may still need to receive video output from a cloud-based video game and render it on a local display, input latency can be reduced by allowing the controller to send input directly to the cloud gaming server over the network, bypassing the client device.
[0112] In one embodiment, the networked controller and client device may be configured to send certain types of input directly from the controller to the cloud gaming server, and to send other types of input via the client device. For example, inputs whose detection does not rely on any additional hardware or processing other than the controller itself may be sent directly from the controller to the cloud gaming server via the network, bypassing the client device. Such inputs may include button inputs, joystick inputs, embedded motion detection inputs (e.g., accelerometers, magnetometers, gyroscopes), etc. However, inputs that utilize additional hardware or need to be processed by the client device may be sent by the client device to the cloud gaming server. These may include captured video or audio from the gaming environment that may be processed by the client device before being sent to the cloud gaming server. In addition, inputs from the motion detection hardware of the controller may be processed by the client device in conjunction with the captured video to detect the position and motion of the controller, which will then be communicated to the cloud gaming server by the client device. It should be understood that the controller device according to various embodiments may also receive data (e.g., feedback data) from the client device or directly from the cloud gaming server.
[0113] It should be understood that the embodiments described herein may be executed on any type of client device.In some embodiments, the client device is a head mounted display (HMD).
[0114] Figure 66 is a diagram illustrating components of a head mounted display 650 shown in accordance with an embodiment of the present disclosure. The head mounted display 650 includes a processor 600 for executing program instructions. A memory 602 is provided for storage purposes and may include both volatile memory and non-volatile memory. A display 604 is included that provides a visual interface that a user may view. A battery 606 is provided as a power source for the head mounted display 650. A motion detection module 608 may include any of a variety of motion sensitive hardware such as a magnetometer 610, an accelerometer 612, and a gyroscope 614.
[0115] An accelerometer is a device used to measure acceleration and the reaction force caused by gravity. Single-axis and multi-axis models can be used to detect the magnitude and direction of acceleration in different directions. Accelerometers are used to sense tilt, vibration, and shock. In one embodiment, three accelerometers 612 are used to provide the direction of gravity, which gives an absolute reference for two angles (world-space pitch and world-space roll).
[0116] The magnetometer measures the strength and direction of the magnetic field near the head mounted display. In one embodiment, three magnetometers 610 are used within the head mounted display to ensure an absolute reference to the natural space yaw angle. In one embodiment, the magnetometer is designed to span the earth's magnetic field of ±80 microtesla. The magnetometer is affected by metal and provides a yaw measurement that is monotonic with the actual yaw. Due to metal in the environment, the magnetic field may bend, which can cause a bend in the yaw measurement. If necessary, information from other sensors (such as a gyroscope or a camera) can be used to calibrate this bend. In one embodiment, an accelerometer 612 is used together with the magnetometer 610 to obtain the tilt and azimuth of the head mounted display 650.
[0117] A gyroscope is a device for measuring or maintaining orientation based on the principle of angular momentum. In one embodiment, three gyroscopes 614 provide information about movement across respective axes (x, y, and z) based on inertial sensing. Gyroscopes help detect rapid rotations. However, gyroscopes can drift over time in the absence of an absolute reference. This requires periodic resetting of the gyroscopes, which can be accomplished using other available information, such as position / orientation determination based on visual tracking of the object, accelerometers, magnetometers, etc.
[0118] A camera 616 is provided to capture images and image streams of the real environment. More than one camera may be included in the head mounted display 650, including a rear-facing camera (facing away from the user when the user is viewing the display of the head mounted display 650) and a front-facing camera (facing toward the user when the user is viewing the display of the head mounted display 650). In addition, a depth camera 618 may be included in the head mounted display 650 for sensing depth information of objects in the real environment.
[0119] In one embodiment, a camera integrated on the front of the HMD can be used to provide warnings about safety. For example, if the user is approaching a wall or object, the user can be warned. In one embodiment, a user can be provided with an outline view of physical objects in the room to warn the user of their presence. For example, the outline can be an overlay in the virtual environment. In some embodiments, the HMD user can be provided with a view of reference markers, such as overlaid in the floor. For example, the markers can provide the user with a reference to the location of the center of the room in which the user is playing a game. For example, this can provide the user with visual information about where the user should move to avoid hitting a wall or other object in the room. Tactile warnings and / or audio warnings can also be provided to the user to provide more safety when the user wears the HMD and plays a game or navigates content with the HMD.
[0120] The head mounted display 650 includes a speaker 620 for providing audio output. In addition, a microphone 622 may be included for capturing audio from a real environment, including sounds from the surrounding environment, speech emitted by the user, etc. The head mounted display 650 includes a tactile feedback module 624 for providing tactile feedback to the user. In one embodiment, the tactile feedback module 624 can cause movement and / or vibration of the head mounted display 650 to provide tactile feedback to the user.
[0121] LED 626 is provided as a visual indicator of the status of the head mounted display 650. For example, the LED may indicate battery charge, power on, etc. A card reader 628 is provided to enable the head mounted display 650 to read information from a memory card and write information to a memory card. A USB interface 630 is included as an example of an interface for enabling connection of peripheral devices or connection with other devices (such as other portable devices, computers, etc.). In various embodiments of the head mounted display 650, any of various types of interfaces may be included to enable greater connectivity of the head mounted display 650.
[0122] A WiFi module 632 is included for connecting to the Internet via wireless network technology. In addition, the head mounted display 650 includes a Bluetooth module 634 for wireless connection with other devices. A communication link 636 may also be included to connect to other devices. In one embodiment, the communication link 636 uses infrared transmission for wireless communication. In other embodiments, the communication link 636 can use any of a variety of wireless or wired transmission protocols to communicate with other devices.
[0123] An input button / sensor 638 is included to provide an input interface for the user. Any of a variety of input interfaces may be included, such as buttons, a touch pad, a joystick, a trackball, etc. An ultrasonic communication module 640 may be included in the head mounted display 650 for facilitating communication with other devices via ultrasonic technology.
[0124] Biosensor 642 is included to enable detection of physiological data from a user. In one embodiment, biosensor 642 includes one or more dry electrodes for detecting bioelectrical signals of a user through the user's skin.
[0125] The foregoing components of the head mounted display 650 have been described merely as exemplary components that may be included in the head mounted display 650. In various embodiments of the present disclosure, the head mounted display 650 may include or may not include some of the various components described above. For the purpose of facilitating the various aspects of the present disclosure described herein, embodiments of the head mounted display 650 may additionally include other components that are not currently described but are known in the art.
[0126] Those skilled in the art will appreciate that in various embodiments of the present disclosure, the aforementioned handheld device may be used in conjunction with an interactive application displayed on a display to provide various interactive functions.The exemplary embodiments described herein are provided by way of example only and not as a limitation.
[0127] Figure 7 7 is a block diagram of a gaming system 700 according to various embodiments of the present disclosure. The gaming system 700 is configured to provide a video stream to one or more clients 710 via a network 715. The gaming system 700 typically includes a video server system 720 and an optional gaming server 725. The video server system 720 is configured to provide a video stream to one or more clients 710 with a minimum quality of service. For example, the video server system 720 may receive a game command that changes a state or viewpoint within a video game, and provide a video stream reflecting an update of the state change to the client 710 with a minimum lag time. The video server system 720 may be configured to provide a video stream in various alternative video formats (including formats that have not yet been defined). In addition, the video stream may include a video frame configured to be presented to a user at various frame rates. Typical frame rates are 30 frames per second, 60 frames per second, and 720 frames per second. However, higher or lower frame rates are included in alternative embodiments of the present disclosure.
[0128] Client 710 (referred to as 710A, 710B, etc. in this article) may include head-mounted display, terminal, personal computer, game console, tablet computer, phone, set-top box, phone booth, wireless device, digital board, stand-alone device, handheld game device, etc. Typically, client 710 is configured to receive encoded (i.e., compressed) video stream, decode video stream, and present the resulting video to a user, such as a player of a game. The process of receiving encoded video stream and / or decoding video stream generally includes storing each video frame in a receiving buffer of the client. The video stream may be presented to the user on a display integrated with client 710 or on a separate device such as a monitor or TV. Client 710 is optionally configured to support more than one game player. For example, a game console may be configured to support two, three, four or more simultaneous players. Each of these players may receive a separate video stream, or a single video stream may include an area of frames generated specifically for each player (e.g., based on each player's viewpoint). Client 710 is optionally geographically dispersed. The number of clients included in the gaming system 700 may vary greatly from one or two to thousands, tens of thousands, or more. As used herein, the term "game player" is used to refer to a person playing a game, and the term "game playing device" is used to refer to a device used to play a game. In some embodiments, a game playing device may refer to a plurality of computing devices that collaborate to deliver a gaming experience to a user. For example, a game console and an HMD may collaborate with a video server system 720 to deliver a game viewed through an HMD. In one embodiment, a game console receives a video stream from the video server system 720, and the game console forwards the video stream or an update of the video stream to the HMD for rendering.
[0129] Client 710 is configured to receive the video stream via network 715. Network 715 can be any type of communication network, including a telephone network, the Internet, a wireless network, a power line network, a local area network, a wide area network, a dedicated network, etc. In a typical embodiment, the video stream is communicated via a standard protocol such as TCP / IP or UDP / IP. Alternatively, the video stream is communicated via a proprietary standard.
[0130] A typical example of a client 710 is a personal computer including a processor, non-volatile memory, a display, decoding logic, network communication capabilities, and an input device. The decoding logic may include hardware, firmware, and / or software stored on a computer-readable medium. Systems for decoding (and encoding) video streams are well known in the art and vary depending on the specific encoding scheme used.
[0131] Client 710 may but need not further include a system configured to modify the received video. For example, the client may be configured to perform further rendering, superimpose one video image on another video image, crop the video image, and the like. For example, client 710 may be configured to receive various types of video frames, such as I frames, P frames, and B frames, and be configured to process these frames into images to display to the user. In some embodiments, members of client 710 are configured to perform further rendering, coloring, conversion to 3-D or similar operations on the video stream. Members of client 710 are optionally configured to receive more than one audio or video stream. The input devices of client 710 may include, for example, a one-handed game controller, a two-handed game controller, a gesture recognition system, a gaze recognition system, a voice recognition system, a keyboard, a joystick, a pointing device, a force feedback device, a motion and / or position sensing device, a mouse, a touch screen, a neural interface, a camera, an input device that has not yet been developed, and the like.
[0132] The video stream (and optionally the audio stream) received by the client 710 is generated and provided by the video server system 720. As further described elsewhere herein, the video stream includes video frames (and the audio stream includes audio frames). The video frames are configured (e.g., they include pixel information in an appropriate data structure) to contribute meaningfully to the image displayed to the user. As used herein, the term "video frame" is used to refer to a frame that primarily includes information that is configured to contribute to, for example, the image displayed to the user. Most of the teachings herein about "video frames" can also be applied to "audio frames."
[0133] Client 710 is generally configured to receive input from a user. These inputs may include game commands that are configured to change the state of a video game or otherwise affect the progress of the game. An input device may be used to receive game commands and / or game commands may be automatically generated by a computing instruction executed on client 710. The received game commands are communicated to video server system 720 and / or game server 725 from client 710 via network 715. For example, in some embodiments, game commands are communicated to game server 725 via video server system 720. In some embodiments, a separate copy of the game command is transmitted from client 710 to game server 725 and video server system 720. The communication of the game command optionally depends on the identity of the command. Game commands are optionally communicated from client 710A by different routes or communication channels for providing audio or video streams to client 710A.
[0134] The game server 725 is optionally operated by an entity different from the video server system 720. For example, the game server 725 may be operated by a publisher of a multiplayer game. In the example, the video server system 720 is optionally viewed as a client by the game server 725, and is optionally configured to appear as a prior art client that executes a prior art game engine from the perspective of the game server 725. The communication between the video server system 720 and the game server 725 may optionally occur via the network 715. Therefore, the game server 725 may be a prior art multiplayer game server that sends game state information to a plurality of clients, one of which is the game server system 720. The video server system 720 may be configured to communicate with a plurality of instances of the game server 725 simultaneously. For example, the video server system 720 may be configured to provide a plurality of different video games to different users. Each of these different video games may be supported by different game servers 725 and / or published by different entities. In some embodiments, several geographically distributed instances of the video server system 720 are configured to provide game videos to a plurality of different users. Each of these instances of the video server system 720 can communicate with the same instance of the game server 725. Communications between the video server system 720 and one or more of the game servers 725 optionally occur via dedicated communication channels. For example, the video server system 720 can be connected to the game server 725 via a high bandwidth channel dedicated to communications between the two systems.
[0135] The video server system 720 includes at least a video source 730, an I / O device 745, a processor 750, and a non-transitory storage device 755. The video server system 720 may include one computing device or be distributed in multiple computing devices. These computing devices are optionally connected via a communication system such as a local area network.
[0136] Video source 730 is configured to provide a video stream, such as a streaming video or a series of video frames forming a dynamic image. In some embodiments, video source 730 includes a video game engine and rendering logic. The video game engine is configured to receive game commands from the player and maintain a copy of the state of the video game based on the received commands. The game state includes the position of objects in the game environment and the usual viewpoint. The game state may also include the properties, images, colors and / or textures of the objects.
[0137] The game state is usually maintained based on game rules and game commands such as moving, turning, attacking, setting focus, interacting, using, etc. The part of the game engine is optionally arranged in the game server 725. The game server 725 can maintain a copy of the game state based on the game commands received from multiple players using geographically dispersed clients. In these cases, the game state is provided to the video source 730 by the game server 725, wherein a copy of the game state is stored and rendering is performed. The game server 725 can directly receive game commands from the client 710 via the network 715, and / or can receive game commands via the video server system 720.
[0138] The video source 730 typically includes rendering logic, such as hardware, firmware, and / or software stored on a computer-readable medium such as a storage device 755. The rendering logic is configured to create video frames of a video stream based on the game state. All or part of the rendering logic is optionally provided within a graphics processing unit (GPU). The rendering logic typically includes processing stages configured to determine three-dimensional spatial relationships between objects and / or to apply appropriate textures, etc. based on the game state and viewing angle. The rendering logic produces raw video, which is then typically encoded before being communicated to the client 710. For example, the rendering logic may be based on Adobe Acrobat. The original video is encoded using the standard, .wav, H.264, H.263, On2, VP6, VC-1, WMA, Huffyu v, Lagarith, MPG-x.Xvid.FFmpeg, x264, VP6-8, realvideo, mp3, etc. The encoding process produces a video stream, which is optionally packaged to be transmitted to a decoder on a remote device. The video stream is characterized by frame size and frame rate. Typical frame sizes include 800x600, 1280x720 (e.g., 720 p), 1024x768, but any other frame size can be used. The frame rate is the number of video frames per second. The video stream may include different types of video frames. For example, the H.264 standard includes "P" frames and "I" frames. I frames include information for refreshing all macroblocks / pixels on a display device, while P frames include information for refreshing its subset. The data size of a P frame is usually smaller than an I frame. As used herein, the term "frame size" means the number of pixels within a frame. The term "frame data size" is used to refer to the number of bytes required to store the frame.
[0139] In an alternative embodiment, video source 730 includes a video recording device such as a camera. The camera can be used to generate delayed or live video that can be included in a video stream of a computer game. The resulting video stream optionally includes both rendered images and images recorded using a still or video camera. Video source 730 may also include a storage device that is configured to store previously recorded video for inclusion in the video stream. Video source 730 may also include a motion or positioning sensing device configured to detect the motion or position of an object (e.g., a person), and logic configured to determine a game state or generate a video based on the detected motion and / or position.
[0140] Video source 730 is optionally configured to provide an overlay configured to be placed on other videos. For example, these overlays may include a command interface, a login instruction, a message to a game player, images of other game players, video feeds of other game players (e.g., webcam videos). In an embodiment where client 710A includes a touch screen interface or a gaze detection interface, the overlay may include a virtual keyboard, a joystick, a touch pad, etc. In an example of an overlay, the player's voice is overlaid on an audio stream. Video source 730 optionally also includes one or more audio sources.
[0141] In embodiments where the video server system 720 is configured to maintain a game state based on input from more than one player, each player may have a different viewpoint including the position and direction of the view. The video source 730 may optionally be configured to provide a separate video stream for each player based on their viewpoint. In addition, the video source 730 may be configured to provide different frame sizes, frame data sizes, and / or encodings to each of the clients 710. The video source 730 may optionally be configured to provide 3-D video.
[0142] I / O devices 745 are configured for the video server system 720 to send and / or receive information such as video, commands, requests for information, game status, gaze information, device motion, device location, user motion, client identity, player identity, game commands, security information, audio, etc. I / O devices 745 typically include communication hardware such as a network card or modem. I / O devices 745 are configured to communicate with game servers 725, network 715, and / or clients 710.
[0143] Processor 750 is configured to execute logic, such as software included in the various components of video server system 720 discussed herein. For example, processor 750 can be programmed with software instructions to perform the functions of video source 730, game server 725, and / or client qualifier 760. Video server system 720 optionally includes more than one instance of processor 750. Processor 750 can also be programmed with software instructions to execute commands received by video server system 720, or to coordinate the operation of various elements of game system 700 discussed herein. Processor 750 can include one or more hardware devices. Processor 750 is an electronic processor.
[0144] Storage device 755 comprises non-temporary analog and / or digital storage device. For example, storage device 755 may comprise an analog storage device configured to store video frames. Storage device 755 may comprise a computer-readable digital storage device, such as a hard disk drive, an optical disk drive, or a solid-state storage device. Storage device 715 is configured to store video frames, artificial frames, video streams including both video frames and artificial frames, audio frames, audio streams, etc. (e.g., through an appropriate data structure or file system). Storage device 755 is optionally distributed in multiple devices. In some embodiments, storage device 755 is configured to store software components of video source 730 discussed elsewhere herein. These components can be stored in a format that is ready to be provided when needed.
[0145] The video server system 720 optionally also includes a client qualifier 760. The client qualifier 760 is configured to remotely determine the capabilities of a client such as client 710A or 710B. These capabilities may include both the capabilities of the client 710A itself and the capabilities of one or more communication channels between the client 710A and the video server system 720. For example, the client qualifier 760 may be configured to test a communication channel through the network 715.
[0146] The client qualifier 760 can manually or automatically determine (e.g., discover) the capabilities of the client 710A. Manual determination includes communicating with the user of the client 710A and asking the user to provide capabilities. For example, in some embodiments, the client qualifier 760 is configured to display images, text, etc. in the browser of the client 710A. In one embodiment, the client 710A is an HMD including a browser. In another embodiment, the client 710A is a game console with a browser that can be displayed on the HMD. The displayed object requests the user to input information of the client 710A, such as an operating system, a processor, a video decoder type, a network connection type, a display resolution, etc. The information input by the user is communicated back to the client qualifier 760.
[0147] For example, an automatic determination may be performed by executing an agent on the client 710A and / or by sending a test video to the client 710A. The agent may include a computing instruction embedded in a web page or installed as an attachment, such as a java script. The agent is optionally provided by the client qualifier 760. In various embodiments, the agent may discover the processing power of the client 710A, the decoding and display capabilities of the client 710A, the latency reliability and bandwidth of the communication channel between the client 710A and the video server system 720, the display type of the client 710A, the firewall present on the client 710A, the hardware of the client 710A, the software executed on the client 710A, the registry entries in the client 710A, etc.
[0148] The client qualifier 760 includes hardware, firmware, and / or software stored on a computer-readable medium. The client qualifier 760 is optionally disposed on a computing device separate from one or more other elements of the video server system 720. For example, in some embodiments, the client qualifier 760 is configured to determine characteristics of a communication channel between a client 710 and more than one instance of the video server system 720. In these embodiments, the information discovered by the client qualifier can be used to determine which instance of the video server system 720 is most suitable for transmitting streaming video to one of the clients 710.
[0149] It should be understood that the various embodiments defined herein may be combined or assembled into specific implementations using the various features disclosed herein. Therefore, the examples provided are only some possible examples and are not limited to various implementations that are possible by combining various elements to define more implementations. In some examples, some implementations may include fewer elements without departing from the spirit of the disclosed or equivalent implementations.
[0150] Embodiments of the present disclosure may be practiced with various computer system configurations including handheld devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a wire-based or wireless network.
[0151] In view of the above embodiments, it should be understood that embodiments of the present disclosure may employ various computer-implemented operations involving data stored in a computer system. These operations are operations that require physical manipulation of physical quantities. Any operation described herein that forms part of an embodiment of the present disclosure is a useful machine operation. Embodiments of the present disclosure also relate to equipment or devices for performing these operations. The device may be specially constructed according to the desired purpose, or the device may be a general-purpose computer selectively activated or configured by a computer program stored in the computer. Specifically, various general-purpose machines may be used with computer programs written according to the teachings of this article, or it may be more convenient to build a more specialized device to perform the desired operations.
[0152] The present disclosure may also be embodied as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of computer-readable media include hard drives, network attached storage (NAS), read-only memory, random access memory, CD-ROM, CD-R, CD-RW, magnetic tape, and other optical and non-optical data storage devices. A computer-readable medium may include a computer-readable tangible medium distributed on a network-connected computer system so that the computer-readable code is stored and executed in a distributed manner.
[0153] Although the method operations are described in a particular order, it should be understood that other housekeeping operations may be performed between the operations, or that the operations may be adjusted so that they occur at slightly different times, or may be distributed in a system that allows processing operations to occur at various time intervals associated with the processing, so long as the processing of the superimposed operations is performed in the desired manner.
[0154] Although the foregoing disclosure has been described in detail for the purpose of clear understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. Therefore, the present embodiments are to be considered illustrative rather than restrictive, and the embodiments of the present disclosure are not limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
Claims
1. A method for assisting game play in a team environment, comprising: monitoring, at a game server, a plurality of game plays of a plurality of users playing a plurality of game applications, wherein the plurality of game plays include a plurality of tasks to be completed, the plurality of tasks being associated with a plurality of task types, wherein the plurality of users are communicatively coupled to the game server via a network through a plurality of client devices; determining, at the game server, a plurality of player proficiency scores of the plurality of users in completing tasks of the plurality of task types, and continuously updating the plurality of player proficiency scores as the plurality of users participate in the plurality of game plays; assigning, at the game server, a first plurality of users to a first team playing a multiplayer game application in a game session, wherein the first team competes with other teams in a competition, wherein in the competition, the first team completes a related set of tasks in the multiplayer game application; determining, at the game server, a first probability of success for the first team to complete the associated set of tasks based on a first plurality of player proficiency scores of the first plurality of users during the game session; as well as During the gaming session, based on the first probability of success, a recommendation is generated and provided to the first team via the network to increase the first probability of success of a goal of winning a competition with the other team.
2. The method according to claim 1, further comprising: for a plurality of combinations of task assignments from the related task set, determining, at the game server, a plurality of predicted success rates for the first team to complete the related task set based on the first plurality of player proficiency scores of the first plurality of users of the first team; as well as Tasks in the set of related tasks are assigned to the first plurality of users associated with a first combination of task assignments having a highest predicted success rate.
3. The method according to claim 1, wherein: Determining the plurality of player proficiency scores includes: For a first user of the first team assigned to a first task of a first task type, a first player proficiency score for the first task type is determined based on corresponding game play.
4. The method according to claim 3, further comprising: A first plurality of game plays of the first user playing a first set of gaming applications is monitored, wherein the first plurality of game plays includes a first plurality of tasks of the first task type.
5. A method for assisting game play in a team environment, comprising: monitoring, at a game server, a plurality of game plays of a plurality of users playing a plurality of game applications, wherein the plurality of game plays include a plurality of tasks to be completed, the plurality of tasks being associated with a plurality of task types, wherein the plurality of users are communicatively coupled to the game server via a network through a plurality of client devices; determining, at the game server, a plurality of player proficiency scores of the plurality of users in completing tasks of the plurality of task types, and continuously updating the plurality of player proficiency scores as the plurality of users participate in the plurality of game plays; assigning, at the game server, a first plurality of users to a first team playing a multiplayer game application in a game session, wherein the first team competes with other teams in a competition, wherein in the competition, the first team completes a first set of related tasks in the multiplayer game application; determining, at the game server, a first probability of success for the first team in completing the first set of related tasks based on a first plurality of player proficiency scores of the first plurality of users during the game session; as well as During the gaming session, based on the first probability of success, a recommendation is generated and provided to the first team via the network to increase the first probability of success of a goal of winning a competition with the other team.
6. The method according to claim 5, further comprising: for first plurality of combinations of task assignments from the first set of related tasks, determining first plurality of predicted success rates for the first team to complete the first set of related tasks based on the first plurality of player proficiency scores of the first plurality of users of the first team; as well as Assigning tasks in the first set of related tasks to the first plurality of users associated with a first combination of task assignments having a first highest predicted success rate.
7. The method according to claim 5, wherein: Determining the plurality of player proficiency scores includes: For a first user of the first team assigned to a first task of a first task type, a first player proficiency score for the first task type is determined based on corresponding game play.
8. The method according to claim 7, further comprising: A first plurality of game plays of the first user playing a first set of gaming applications is monitored, wherein the first plurality of game plays includes a first plurality of tasks of the first task type.
9. A non-transitory computer-readable medium storing a computer program for assisting in playing a game in a team environment, the computer-readable medium comprising: program instructions for monitoring, at a game server, a plurality of game sessions of a plurality of users playing a plurality of game applications, wherein the plurality of game sessions include a plurality of tasks to be completed, the plurality of tasks being associated with a plurality of task types, wherein the plurality of users are communicatively coupled to the game server via a network through a plurality of client devices; program instructions for determining, at the game server, a plurality of player proficiency scores of the plurality of users in completing tasks of the plurality of task types, and continuously updating the plurality of player proficiency scores as the plurality of users participate in the plurality of game plays; program instructions for assigning, at the game server, a first plurality of users to a first team playing a multiplayer game application in a game session, wherein the first team competes with other teams in a competition, wherein in the competition, the first team completes a first set of related tasks in the multiplayer game application; program instructions for determining, at the game server, during the game session, a first probability of success for the first team in completing the first set of related tasks based on a first plurality of player proficiency scores for the first plurality of users; and Program instructions for generating and providing, via the network, recommendations to the first team during the gaming session based on the first probability of success to increase the first probability of success of a goal of winning a competition with the other team.
10. The non-transitory computer-readable medium of claim 9, further comprising: program instructions for determining, for a first plurality of combinations of task assignments from the first set of related tasks, a first plurality of predicted success rates for the first team to complete the first set of related tasks based on the first plurality of player proficiency scores of the first plurality of users of the first team; as well as Program instructions for assigning tasks in the first set of related tasks to the first plurality of users associated with a first combination of task assignments having a first highest predicted success rate.
11. The non-transitory computer readable medium of claim 9, wherein: Determining the plurality of player proficiency scores includes: Program instructions for determining, for a first user of the first team assigned to a first task of a first task type, a first player proficiency score of the first task type based on corresponding game play.
12. The non-transitory computer readable medium of claim 11 , further comprising: Program instructions for monitoring a first plurality of game plays by the first user playing a first set of gaming applications, wherein the first plurality of game plays includes a first plurality of tasks of the first task type.
