Collaborative and Coaching Gameplay

Abstract

A method and system for collaborative or coaching gameplay in a virtual environment is disclosed. The memory may store a content control profile for a set of control inputs associated with actions within a virtual environment of a digital content title. A request may be received from a collection of one or more users associated with different source devices for collaborative gameplay of the digital content title. In response to the request, at least one virtual avatar may be generated for an interactive session of the digital content title. Multiple control inputs may be received from multiple different source devices and combined into a combined set of control inputs. Generating the combined set of control inputs may be based on the content control profile. A virtual action associated with the virtual avatar may be controlled within the virtual environment according to the combined set of control inputs.

Description

[Technical Field]

[0001] The present system relates generally to collaborative or coaching navigation of a virtual environment, and more particularly to collaborative or shared control of individual roles or viewpoints by multiple users within the virtual environment. [Background technology]

[0002] Many different types of individuals with varying levels of experience may engage in the use of entertainment systems to experience virtual environments associated with digital content titles. Video game consoles, handheld devices, computers, and virtual reality systems, as well as the associated digital content accessed and played thereby, may be continually updated or refreshed with new versions that offer new experiences and present learning and adaptation challenges. For some individuals, learning to navigate new types of virtual environments and / or play new video games on new hardware may be a frustrating process due not only to unfamiliarity with the particular game title, but also to unfamiliarity with how to use the hardware to perform intended movements and other game interactions. Furthermore, some types of hardware or hardware inputs (e.g., combos) may be difficult for players with physiological or mental conditions that limit movement, dexterity, memory, etc. Furthermore, even highly experienced, expert players may experience some degree of difficulty or awkwardness during the transition to new or unfamiliar systems, devices, and game titles. The muscle memory and habit forces associated with old or familiar systems and titles may not always transfer, and such users may end up making various input errors that lead to unintended in-game movements and interactions.

[0003] Many entertainment systems and new media content typically provide users with only a cursory introduction, such as a quick setup guide for a new console or an introductory tutorial for a video game. Depending on the user's experience level, ability, and scope of the material, such material (and the experience of learning to navigate a new or unfamiliar virtual environment) may nevertheless be daunting in terms of quantity and complexity for a user who has little interaction with such device or content, and may only serve to further confuse, frustrate, and / or overwhelm the user.

[0004] Therefore, there is a need in the art to provide improved systems and methods for collaborative play and / or coaching play within a virtual environment. Summary of the Invention

[0005] Systems and methods may be provided for collaborative gameplay or coaching gameplay in a virtual environment. A memory may store a content control profile relating to a set of control inputs associated with actions within the virtual environment of the digital content title. Requests may be received from a collection of one or more users associated with different source devices for collaborative gameplay of the digital content title. In response to the requests, at least one virtual avatar may be generated for an interactive session of the digital content title. Multiple control inputs may be received from multiple different source devices and combined into a combined set of control inputs. Generating the combined set of control inputs may be based on the content control profile. Virtual actions associated with the virtual avatar may be controlled within the virtual environment according to the combined set of control inputs. [Brief explanation of the drawings]

[0006] [Figure 1]1 illustrates an exemplary network environment in which collaborative or coaching gameplay may be implemented in a virtual environment. [Figure 2] 1 illustrates an exemplary unified data system (UDS) that may be used to provide data to a system for collaborative gameplay or coaching gameplay in a virtual environment. [Figure 3] 1 is a flowchart illustrating an exemplary method for collaborative or coaching gameplay in a virtual environment. [Figure 4] FIG. 1 illustrates an exemplary implementation of collaborative or coaching gameplay in a virtual environment. [Figure 5] FIG. 1 is a block diagram of an exemplary electronic entertainment system that may be used with embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0007] Embodiments of the present invention include methods and systems for collaborative gameplay or coaching gameplay in a virtual environment. A memory may store a content control profile relating to a set of control inputs associated with actions within a virtual environment of a digital content title. Requests may be received from a collection of one or more users associated with different source devices for collaborative gameplay of the digital content title. In response to the requests, at least one virtual avatar may be generated for an interactive session of the digital content title. Multiple control inputs may be received from multiple different source devices and combined into a combined set of control inputs. Generating the combined set of control inputs may be based on the content control profile. Virtual actions associated with the virtual avatar may be controlled within the virtual environment according to the combined set of control inputs.

[0008] FIG. 1 illustrates an exemplary network environment including a system capable of implementing collaborative or coaching gameplay in a virtual environment. The network environment 100 may include one or more content source servers 110 that provide digital content for distribution (e.g., games, other applications, and services), one or more content provider server application program interfaces (APIs) 120, a content delivery network server 130, a collaboration control server 140, and one or more user devices 150. The devices in the network environment 100 communicate with each other using one or more communication networks, which may include dedicated local networks (e.g., intranets) and / or may be part of a larger wide-area network. A communication network may be a local area network (LAN) that may be communicatively coupled to a wide-area network (WAN) such as the Internet. The Internet is a broad network of interconnected computers and servers that enables the transmission and exchange of Internet Protocol (IP) data between connected users via a network service provider. Examples of network service providers include the public switched telephone network, a cable service provider, a digital subscriber line (DSL) service provider, or a satellite service provider. One or more communication networks enable communication between the various components of network environment 100 .

[0009] The servers described herein may include any type of server known in the art, including standard hardware computing components such as network and media interfaces, non-transitory computer-readable storage (memory), and a processor for executing instructions or accessing information that may be stored in the memory. The functionality of multiple servers may be integrated into a single server. Any of the foregoing servers (or integrated servers) may exhibit characteristics of particular client-side, cache, or proxy servers. These characteristics may depend on the particular network placement of the server or the particular configuration of the server.

[0010] The content source server 110 may maintain and provide a variety of digital content and digital services that are available for distribution over a communications network. The content source server 110 may be associated with any content provider that makes its content available for access over a communications network. Accordingly, the content source server 110 may host a variety of different content titles and / or different related interactive sessions, which may further be associated with object data (e.g., activity information, zone information, character information, player information, other game media information, etc.) about digital or virtual objects that are displayed in a digital or virtual environment during an interactive session. The same content source server 110 or different content source servers 110 may host either or both the content title itself and one or more related interactive sessions, during which a user may navigate, interact, initiate, and control virtual actions (including actions performed by one or more virtual avatars) and / or play within a virtual environment resulting from executing the digital content title.

[0011] Such content may include not only digital video and games, but also other types of digital applications and services. Such applications and services may include any of a variety of different digital content that may be provided to user devices 150, as well as features such as providing and supporting chat and other communication channels. Chat and communication services may include voice-based, text-based, and video-based messages. Thus, user devices 150 may participate in one or more communication sessions and gameplay sessions simultaneously, and gameplay and communication sessions may be hosted on one or more content source servers 110. Such chat and other communication services may be used between user devices 150 (including source devices) to verbally coordinate gameplay or to communicate coaching tips, hints, contextual data, coaching instructions, and the like.

[0012] Content from content source servers 110 may be provided via content provider server API 120, which enables various types of content source servers 110 to communicate with other servers (e.g., user devices 150) in network environment 100. Content provider server API 120 may be specific to the particular operating language, system, platform, protocol, etc. of the content source server 110 providing the content, as well as the user devices 150 and other devices in network environment 100. In network environment 100 including multiple different types of content source servers 110, there may likewise be a corresponding number of content provider server APIs 120, which enable various formats, conversions, and other cross-device and cross-platform communication processes for providing content and other services to different user devices 150, each of which may process such content using different operating systems, protocols, etc. Accordingly, applications and services may be made available in different formats to be compatible with a variety of different user devices 150. In a network environment 100 that includes multiple different types of content source servers 110, content delivery network servers 130, collaboration control servers 140, user devices 150, and databases 160, there may likewise be a corresponding number of APIs managed by the content provider server API 120.

[0013] The content provider server API 120 may further facilitate each of the user devices 150's access to content hosted by or services provided by the content source server 110, either directly or via the content delivery network server 130. Additional information, such as metadata about the accessed content or service, may also be provided to the user device 150 by the content provider server API 120. As described below, the additional information (e.g., object data, metadata) may be available to provide details about the content or service provided to the user device 150. In some embodiments, the services provided by the content source server 110 to the user device 150 via the content provider server API 120 may include supporting services associated with other content or services, such as chat services, ratings, and profiles associated with particular games, teams, communities, etc. In such cases, the content source servers 110 may also communicate with each other via the content provider server API 120.

[0014] The content delivery network servers 130 may include servers that provide resources, files, etc. related to content from the content source servers 110 to the user devices 150, including various content and service configurations. The content delivery network servers 130 may also be invoked by user devices 150 requesting access to particular content or services. The content delivery network servers 130 may include universe management servers, game servers, streaming media servers, servers hosting downloadable content, and other content delivery servers known in the art.

[0015] As shown, the collaboration control server 140 may include any data server known in the art capable of communicating with different content source servers 110, content provider server APIs 120, content delivery network servers 130, user devices 150, and databases 160. Such collaboration control server 140 may be implemented in one or more cloud servers that execute instructions associated with interactive content (e.g., games, activities, videos, podcasts, user-generated content (“UGC”), publisher content, etc.). In some implementations, the functionality of the collaboration control server 140 may be provided as a service according to instructions embodied in any combination of remote, cloud-based, or local applications specifically configured and arranged to provide services to a particular user or set of users.

[0016] User devices 150 may include several different types of computing devices known in the art. User devices 150 may be computing devices that may include controllers associated with any number of different gaming consoles, as well as peripherals, mobile devices, laptops, and desktops. Such user devices 150 may also be configured to access data from other storage media, such as, but not limited to, memory cards or disk drives, which may be appropriate for downloaded services. Such user devices 150 may include standard hardware computing components, such as, but not limited to, network interfaces, media interfaces, non-transitory computer-readable storage (memory), and processors for executing instructions that may be stored in the memory. These user devices 150 may also execute applications using a variety of different operating systems (e.g., iOS®, Android®), applications, or computing languages ​​(e.g., C++®, JavaScript®). An exemplary client device 150 is described in detail herein with respect to FIG. 5. Each of the user devices 150 may be associated with a participant (e.g., an interactive player) or other type of user (e.g., a spectator) involved in one or more interactive sessions within a virtual environment associated with digital content.

[0017] Although depicted separately, database 160 may be stored on the same server, a different server, or any of the servers and devices shown in network environment 100 of any of user devices 150. Such database 160 may store or link to various sources and services used for content control profiles, including specific gameplay or coaching profiles. Database 160 may store a variety of different content profiles associated with different content titles, different hardware (e.g., controllers), different abilities and skills, different gameplay statuses or styles, etc., which may be specific to a particular user, user group or team, user category, game title, game genre, etc. Content control profiles may be used to identify how a user generally plays at different points in the timeline of an interactive session associated with a digital content title, including different sets of control inputs associated with actions within the virtual environment of the digital content title (e.g., including combos corresponding to a predetermined set of control inputs having a particular sequence and timing). Content control profiles may be specific to a user, a group of users, a content title, a content genre, a gameplay style, or other parameters. Thus, one or more content control profiles may be stored in database 160 for each user, user pair or other user group, user-content match, or other combination of parameters. A content control profile may include one or more intent maps that associate particular virtual (e.g., in-game) situations with sets of control inputs that would typically be used to take virtual actions in response to such situations. Such intent maps may be created and refined over time based on ongoing analysis of user gameplay data (e.g., captured by a UGC system, described in more detail with respect to FIG. 2).Thus, by referencing the content control profile and associated intent map, collaborative control server 140 may identify that an incomplete or erroneous set of control inputs is nevertheless associated with a unique virtual action intended by the user in the current interactive session.

[0018] In an exemplary implementation, collaboration control server 140 may receive a request to support collaborative play of a particular digital content title. The request may specify or otherwise indicate a set of one or more source devices (e.g., user devices 150) that may be associated with one or more users (e.g., designated player(s), designated coach(es), including artificial intelligence (AI)-based virtual coach(es). In some implementations, the request may be received from a single user device specifying a need to match with another user device. In response, collaboration control server 140 may identify an appropriate match with another user device that is available for collaborative gameplay. Such match identification may be based on an analysis of user status, abilities and disabilities, skills, experience level, associated hardware and hardware capabilities, historical gameplay data, gameplay style, and other parameters associated with the requesting user and the matched user that have been identified as complementary. In some embodiments, additional user input may be required to identify a match. Such additional user input may be required in the form of a survey, a quiz, a skill assessment and demonstration, or any combination of the above.

[0019] Thus, a player who, for some reason, may not be able to play a game well or otherwise navigate the virtual environment independently may still be provided a successful and enjoyable user experience within the virtual environment based on the matching user(s) identified as available to participate in complementary and collaborative gameplay. In addition to improving accessibility for players of different experience, skills, and abilities, collaborative gameplay may also enable players with different hardware (including hardware specially configured or modified for different abilities and disabilities) to fully participate in modern content titles that may otherwise have strict hardware requirements to perform certain virtual actions. If no other suitable matching players are available or are not explicitly requested, the collaborative control server 140 may match the requesting user with an AI-controlled source device that may be specifically configured to be complementary to the requesting user (e.g., according to content control profile(s) and other user data).

[0020] In communication with one or more hosting content source servers 110, collaborative control server 140 may generate at least one virtual avatar within the virtual environment of a requested content title and configure the virtual avatar for collaborative gameplay control by the source devices. Collaborative gameplay allows a group of source devices to jointly and / or cooperatively provide combined control inputs to simulate single-player control. Thus, control inputs from a group of source devices may be combined to control a single avatar (or other virtual role or perspective typically controlled by a single player) within the virtual environment and to control a set of multiple virtual characters (e.g., an army, fleet, platoon) in content titles that enable single-player control of multiple virtual characters. Collaborative control server 140 may combine control inputs received from a specified set of source devices into a single set of control inputs, identify any title-specific control input sequences and timings from profiles of one or more applicable content profiles, and apply the identified sequences and timings before using the resulting combined set of control inputs to perform one or more actions (e.g., via one or more virtual avatars) in the virtual environment. By referencing the applicable content control profile(s), the collaborative control server 140 can predict which virtual actions correspond to events during a particular scene (e.g., or tier or level) within the timeline of an interactive session. For example, a content control profile may specify that virtual combat actions generally occur in response to a virtual monster or enemy appearing in a scene.

[0021] Accordingly, collaborative control server 140 may analyze the combined control inputs received from one or more source devices in such situations to identify which virtual action may be intended to be performed. One source device may provide a set of control inputs corresponding to one portion of the combo, while another source device may provide a different set of control inputs corresponding to the remainder of the combo. Collaborative control server 140 may not only combine sets of control inputs from different source devices (e.g., hosted by one or more content source servers 110), but may also combine certain control inputs according to a particular sequence and timing before providing the combined set of control inputs to control the virtual action of the interactive session. In some implementations, source devices may further specify how control inputs may be prioritized, e.g., for particular scenes or types of virtual actions, so that certain control responsibilities may be transferred between source devices or control may be handed over entirely between source devices.

[0022] One type of collaborative gameplay may include coaching gameplay, in which one or more of the users (and associated source device(s)) is designated a coach. Different levels of coaching control may be specified, which may enable the coach to exercise different levels of control over the virtual avatar during a session. For example, prioritization input may specify that input from non-coach source devices takes priority in controlling the virtual avatar unless and until a particular trigger or triggering situation occurs, such as a particular combo or type of hardware input. Thus, the coached player may play independently as much as possible, while being able to rely on coach control input as a fallback in response to particularly difficult or otherwise designated situations. Thus, control input from the coach source device may be treated differently from control input from source devices not associated with the designated coach.

[0023] During coaching mode, the collaborative control server 140 may provide the designated coach or the coached player with a preview or prediction of an upcoming event, thereby providing additional time to prepare for performing a particular virtual action. The designated coach source device may provide a set of control inputs associated with the predicted virtual action, which may be used to generate audiovisual (e.g., on-screen, sound, overlay, illuminated button) and / or tactile (e.g., associated with a particular button) notifications for the coached player. The coaching notifications may include automated or coach-provided hints or reminders regarding which control inputs are needed to perform the virtual action or other information that may be helpful to the coached player. Such notifications may be automatically triggered when the coached player is determined to be deviating from the set of control inputs typically required to perform the virtual action as indicated by the content control profile. For example, the coached player may be slow to remember how to complete a particular combo, thereby deviating from the typical timing indicated by the associated content control profile. The collaborative control server 140 may detect the timing deviation and automatically generate a coaching notification. In some examples, the timing of the interactive session may be modified so that events are paused, slowed down, or reversed to allow the coached player to attempt to correctly input control inputs for a combo. Thus, the coached player may be given time to follow one or more prompts provided in the coaching notification before a correction or other coaching action is taken. In some examples, the interactive session may be paused while the player is given a virtual refresher or training session and resumed once the player is ready to try again.

[0024] In some embodiments, one or more content control profiles may be part of or function as a training model for artificial intelligence (AI)-based virtual coaching. The AI-based coaching model may also be customized for a particular user based on the particular user's unique profile data and expressed or identified preferences. As described with respect to FIG. 2, data regarding a user's gameplay under different conditions may be captured and used to generate corresponding metrics. Such metrics may be used by the AI-based virtual coach to adapt to the user, including adjusting when and how to implement different coaching actions (e.g., notification types and timing, difficulty adjustments, timing adjustments, etc.).

[0025] FIG. 2 illustrates an exemplary unified data system (UDS) that may be used to provide data to a system for collaborative gameplay or coaching gameplay in a virtual environment. Based on the data provided by the UDS 200, the collaborative control server 140 may know the current session conditions, such as the digital content title, the virtual (e.g., in-game) objects, entities, activities, and events in which the user is involved and thus supports analysis, and the coordination of collaborative control inputs by the collaborative control server 140 with the current virtual interactive activity and / or in-game activity. Each user interaction in the virtual environment may be associated with metadata, such as the type of virtual interaction, its location within the virtual environment, and its point in time within the virtual world timeline, as well as other players, objects, and entities involved. Thus, metadata may be tracked for any of a variety of user interactions that may occur during the current interactive session, including associated virtual activities, entities, settings, results, actions, effects, locations, character status, and the like. Such data may be further aggregated, applied to a data model, and subjected to analysis. Such a UDS data model may be used to assign contextual information to pieces of information in a uniform manner across digital content and / or game titles.

[0026] For example, various content titles may depict one or more objects with which a user can interact (e.g., engage in in-game activities) and / or UGC (e.g., screenshots, videos, play-by-plays, mashups, etc.) created by peers, the publisher of the media content title, and / or third-party publishers. Such UGC may include metadata for searching such UGC. Such UGC may also include information about the media and / or peers. Such peer information may be derived from data collected during peer interactions with objects in interactive content titles (e.g., video games, interactive books, etc.) and may be “bound” to and stored with the UGC. Such binding extends UGC because the UGC may deep link (e.g., directly launch) to the objects, provide information about the objects and / or peers in the UGC, and / or enable users to interact with the UGC.

[0027] As shown in FIG. 2 , an exemplary console 228 (e.g., user device 150) and exemplary servers 218 (e.g., streaming server 220, activity feed server 224, user-generated content (UGC) server 232, and object server 226) are shown. In one example, the console 228 may be implemented on any of the platform server 120, cloud server, or server 218. In an illustrative example, a content recorder 202 may be implemented on the platform server 120, cloud server, or any of the servers 218. Such a content recorder 202 receives content (e.g., media) from an interactive content title 230 and records it on a content ring buffer 208. Such a ring buffer 208 may store multiple content segments (e.g., v1, v2, and v3), start times for each segment (e.g., V1_START_TS, V2_START_TS, V3_START_TS), and end times for each segment (e.g., V1_END_TS, V2_END_TS, V3_END_TS). Such segments may be stored as media files 212 (e.g., MP4, WebM, etc.) by the console 228. Such media files 212 may be uploaded to the streaming server 220 for storage and subsequent streaming or use, although the media files 212 may be stored on any server, cloud server, any console 228, or any user device 150. Such start and end times for each segment may be stored by the console 228 as a content timestamp file 214. Such content timestamp file 214 may also include a stream ID that matches the stream ID of the media file 212, thereby associating the content timestamp file 214 with the media file 212.Such content timestamp files 214 may be uploaded and stored on the activity feed server 224 and / or the UGC server 232, but the content timestamp files 214 may be stored on any server, cloud server, any console 228, or any user device 150.

[0028] At the same time that the content recorder 202 receives and records content from the interactive content title 230, the object library 204 receives data from the interactive content title 230, and the object recorder 206 tracks the data to determine when an object begins and ends. The object library 204 and the object recorder 206 may be implemented on either the platform server 120, the cloud server, or the server 218. When the object recorder 206 detects the start of an object, it receives object data (e.g., data if the object is an activity, user interaction with the activity, activity ID, activity start time, activity end time, activity result, activity type, etc.) from the object library 204 and records the activity data (e.g., ActivityID1, START_TS, ActivityID2, START_TS, ActivityID3, START_TS) in the object ring buffer 210. Such activity data recorded on the object ring buffer 210 may be stored in the object file 216. Such object files 216 may also include activity start time, activity end time, activity ID, activity result, activity type (e.g., competition, quest, task, etc.), and user or peer data related to the activity. For example, object files 216 may store data regarding items used during the activity. Such object files 216 may be stored on an object server 226, although object files 216 may be stored on any server, cloud server, any console 228, or any user device 150.

[0029] Such object data (e.g., object file 216) can be associated with content data (e.g., media file 212 and / or content timestamp file 214). In one example, UGC server 232 stores content timestamp file 214 and associates it with object file 216 based on a match between the stream ID of content timestamp file 214 and the corresponding activity ID of object file 216. In another example, object server 226 can store object file 216 and receive queries for object file 216 from UGC server 232. Such queries can be performed by searching for the activity ID of object file 216 that matches the stream ID of content timestamp file 214 sent with the query. In yet another example, queries of stored content timestamp file 214 can be performed by matching the start and end times of content timestamp file 214 with the start and end times of the corresponding object file 216 sent with the query. Such object files 216 may also be associated with matching content timestamp files 214 by the UGC server 232, although this association may be performed by any server, cloud server, any console 228, or any user device 150. In another example, the object files 216 and content timestamp files 214 may be associated by the console 228 during creation of each file 216, 214.

[0030] In an exemplary embodiment, media files 212 and activity files 216 may provide information regarding the current session state to collaboration control server 140, which information may also be used to evaluate a particular user (e.g., a user requesting collaborative gameplay), match the user with one or more complementary users or coaches, trigger various coaching actions, and make coaching adjustments (including refining AI-based coaching models for a particular user or similar users). Thus, collaboration control server 140 may use such media files 212 and activity files 216 to identify specific conditions of the current interactive session, including performances by players, characters, and objects at specific locations and during specific events within the virtual environment.

[0031] Based on such files 212 and 216, for example, collaborative control server 140 may identify a relevant content control profile associated with the current interactive session, which may be specific to one or more of a content title, a virtual environment, a virtual scene, or an in-game event (e.g., a major battle with a major boss), a gameplay style, etc., and may be used to identify which intents are associated with possible combinations of incoming control inputs from multiple source devices. Such virtual environment or other conditions may determine how incoming control inputs are grouped and combined with each other into sets having specific sequences or timings, thereby providing joint or shared control of virtual actions by virtual characters normally controlled by a single player.

[0032] As incoming inputs are received, collaboration control server 140 may identify combos or portions of moves that are typically performed at the current virtual location and time where the virtual character is located, or during the current event. Collaboration control server 140 may select sets of incoming inputs to combine into a single set of control inputs having a particular sequence and / or timing that may correspond to the identified combos. Additionally, collaboration control server 140 may sort and select different portions of the incoming control inputs to apply or not apply according to prioritization settings that specify how and under what conditions inconsistent or overlapping control inputs should be applied.

[0033] 3 is a flowchart illustrating an exemplary method for collaborative or coaching gameplay in a virtual environment. The method 300 of FIG. 3 may be embodied as executable instructions on a non-transitory computer-readable storage medium, including, but not limited to, non-volatile memory such as a CD, DVD, or hard drive. The instructions on the storage medium may be executed by a processor (or multiple processors) to cause various hardware components of a computing device that hosts or otherwise accesses the storage medium to perform the method. The steps (and their order) specified in FIG. 3 are exemplary and may include various alternative, equivalent, or derivative forms thereof, including, but not limited to, their order of execution.

[0034] At step 310, content control profiles may be stored in memory (e.g., in database 160). Each content control profile may identify a set of control inputs associated with virtual actions within the virtual environment of the digital content title. The content control profiles may be further customized for a particular player based on historical data in files 212-216 captured by UDS system 200. Such customized control profiles may be further customized for a particular player based on the player's gameplay style, particular hardware, historical or habitual moves or combos performed at particular points during an interactive session of the digital content title, particular hardware, and other relevant historical or habitual actions taken by the player and / or player-associated avatar(s).

[0035] At step 320, a request for collaborative gameplay of a specified digital content title may be received, which may be associated with one of the content control profiles. The request may specify at least one user (associated with one source device) for collaborative gameplay. The request may further specify another user for co-piloted play or coaching play depending on a specified prioritization scheme, while the request may also specify that the requesting user matches another source device (which may be associated with another user or controlled by an AI). In response to the request, collaborative control server 140 may configure at least one avatar of the specified digital content title for joint or shared control by multiple source devices.

[0036] At step 330, control inputs may be received from multiple source devices. Using a content control profile associated with the digital content title and examining the current state, collaborative control server 140 may analyze the combination of incoming control inputs together to identify an intended combo or move, or other virtual action intended to be taken within the virtual environment. Collaborative control server 140 may also apply a particular prioritization scheme to favor or disfavor control inputs from one or more source devices, such as, for example, to favor the gameplay style of a coached player over that of a coach, or to switch primary control between different control devices depending on the type of battle or challenge (e.g., shooting versus close combat).

[0037] At step 340, a combination set of control inputs may be generated based on the received incoming control inputs. While control inputs from different source devices may be out of sync because they were entered by different users / AIs and transmitted over different communication networks and using different source devices, the collaborative control server 140 may nevertheless identify the intended sequence and timing by referencing the content control profile with respect to historical or habitual movements or actions at the current point in time within the timeline of play during the interactive session. Thus, the collaborative control server 140 may apply the identified sequence and timing to the combination set of control inputs before using the combination set of control inputs to control the virtual avatar designated for collaborative gameplay. Generating the combination set of control inputs enables simulation of what other players not involved in collaborative gameplay might normally do to perform virtual actions within the virtual environment.

[0038] At step 350, the virtual avatar configured at step 320 may be controlled according to the generated combined set of control inputs. Collaborative control server 140 may provide the generated combined set of control inputs to the device hosting the interactive session, which may be any of the devices shown in FIG. 1. If the request specifies coaching gameplay, additional coaching content or instructions may also be provided or otherwise communicated to one or more of the source devices for presentation within the virtual environment of the interactive session. In some examples, the designated coach may initiate audiovisual or haptic notifications, implement a training wheel mode to prevent deviation from the in-game path (e.g., fatal errors, injury, and / or death), adjust the timeline to pause, slow down, rewind, or reverse virtual actions or events, and apply adaptive corrections that may differ under different conditions.

[0039] 4 illustrates an exemplary implementation of collaborative or coaching gameplay in a virtual environment. As shown, different user devices 150A-150C (e.g., corresponding to different controllers) may engage in collaborative gameplay that further includes a coaching AI 410. Together, the coaching AI 410 and the collaborative control server 140 may reference one or more applicable content control profiles 420 to assess the current status and state of the current interactive session and identify what movements or actions may be intended by the users of the user devices 150A-150B in such context.

[0040] As further shown, control input data may be provided from user devices 150A-150B and coaching AI 410 to collaborative control server 140, which may interpret it according to an applicable content control profile to generate a combination set of control inputs having a particular sequence and timing (e.g., simulating the input of control inputs by a single player for a particular move or combo). The generated combination set of control inputs may be provided to control virtual avatar 430 in different ways during a virtual avatar timeline 440 of an interactive session. As shown, virtual avatar timeline 440 may include multiple periods 450A-450D during which different user devices 150A-150B may attempt primary or prioritized control over virtual avatar 430. For example, during time period 450A, collaboration control server 140 may identify conditions associated with prioritizing control inputs from user device 150A according to a prioritization scheme, while time period 450B may include conditions associated with prioritizing gesture inputs (associated with either user of user devices 150A-150B) and different controller inputs from user device 150B. Similarly, time period 450C may be associated with prioritizing control inputs from user device 150A, and time period 450D may be associated with prioritizing control inputs from user device 150B.

[0041] The control inputs from the coaching AI 410 may be supplemental or provide a fallback option when neither the user devices 150A-150B nor a combination thereof is able to complete an intended combo move or other virtual action within the virtual environment. The coaching AI 410 may also provide educational content and notifications, and other related coaching content and notifications, to supplement the users of the user devices 150A-150B in providing the correct control inputs. The same or different coaching content may be provided to each of the user devices 150A-150B depending on the needs of each user.

[0042] Figure 5 is a block diagram of an exemplary electronic entertainment system that may be used with embodiments of the present invention. The entertainment system 500 of Figure 5 includes a main memory 505, a central processing unit (CPU) 510, a vector unit 515, a graphics processing unit 520, an input / output (I / O) processor 525, an I / O processor memory 530, a controller interface 535, a memory card 540, a universal serial bus (USB) interface 545, and an IEEE interface 550. The entertainment system 500 may further include an operating system read-only memory (OS ROM) 555, an audio processing unit 560, an optical disc control unit 570, and a hard disk drive 565, which are connected to the I / O processor 525 via a bus 575.

[0043] Entertainment system 500 may be an electronic gaming console. Alternatively, entertainment system 500 may be implemented as a general-purpose computer, a set-top box, a handheld gaming device, a tablet computing device, a mobile computing device, or a mobile phone. Entertainment systems may include more or fewer operating components depending on the particular form factor, purpose, or design.

[0044] The CPU 510, vector unit 515, graphics processing unit 520, and I / O processor 525 of FIG. 5 communicate via a system bus 585. Additionally, the CPU 510 of FIG. 5 may communicate with main memory 505 via a dedicated bus 580, and the vector unit 515 and graphics processing unit 520 may communicate via a dedicated bus 590. The CPU 510 of FIG. 5 executes programs stored in the OS ROM 555 and the main memory 505. The main memory 505 of FIG. 5 may include pre-stored programs and programs transferred via the I / O processor 525 from a CD-ROM, DVD-ROM, or other optical disk (not shown) using the optical disk control unit 570. The I / O processor 525 of FIG. 5 may also enable the introduction of content transferred over wireless or other communication networks (e.g., 4$, LTE, 3G, etc.). The I / O processor 525 of FIG. 5 primarily controls the exchange of data between various devices of the entertainment system 500, including the CPU 510, the vector unit 515, the graphics processing unit 520, and the controller interface 535.

[0045] The graphics processing unit 520 of Figure 5 executes graphics instructions received from the CPU 510 and the vector unit 515 to generate images for display on a display device (not shown). For example, the vector unit 515 of Figure 5 may convert an object from three-dimensional coordinates to two-dimensional coordinates and send the two-dimensional coordinates to the graphics processing unit 520. Additionally, the audio processing unit 560 executes instructions to generate audio signals, which are output to an audio device such as a speaker (not shown). Other devices may be connected to the entertainment system 500 via the USB interface 545 and the IEEE 1394 interface 550, such as a wireless transceiver; these interfaces may also be embedded in the system 500 or as part of some other component, such as a processor.

[0046] 5 provides instructions to CPU 510 via controller interface 535. For example, the user may instruct CPU 510 to store certain game information on memory card 540 or other non-transitory computer-readable storage medium, or may instruct a character in the game to perform some particular action.

[0047] The present invention may be implemented in an application that may be operable by a variety of end-user devices. For example, the end-user device may be a personal computer, a home entertainment system (e.g., Sony Playstation2® or Sony Playstation3® or Sony Playstation4®), a portable gaming device (e.g., Sony PSP® or Sony Vita®), or a home entertainment system from a different, but subordinate, manufacturer. It is fully contemplated that the methods described herein will operate on a variety of devices. The present invention may also be implemented in a cross-title neutral manner, such that embodiments of the inventive system may be utilized across a variety of titles from a variety of publishers.

[0048] The present invention may be implemented in applications that may be operable using a variety of devices. A non-transitory computer-readable storage medium refers to any medium or media that participates in providing instructions to a central processing unit (CPU) for execution. Such media can take many forms, including, but not limited to, non-volatile media such as optical disks, magnetic disks, and volatile media such as dynamic memory. Common forms of non-transitory computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROM disks, digital video disks (DVDs), any other optical media, RAM, PROM, EPROM, FLASHEPROM, and any other memory chip or cartridge.

[0049] Various forms of transmission media may be involved in carrying one or more sequences of one or more instructions to the CPU for execution. A bus carries the data to system RAM, from which the CPU retrieves and executes the instructions. The instructions received by the system RAM may optionally be stored on a fixed disk either before or after being executed by the CPU. Various forms of storage may also be implemented, as well as network interfaces and network topologies necessary to implement them.

[0050] The foregoing detailed description of the technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were selected to best explain the principles of the technology, its practical application, and to enable those skilled in the art to utilize the technology in various embodiments and with various modifications suitable for the particular uses contemplated. It is intended that the scope of the technology be defined by the claims.

Claims

1. 1. A method of AI-based coaching in a virtual environment, comprising: storing an artificial intelligence (AI) based coaching model in a memory, said AI based coaching model being trained to implement one or more coaching actions that modify a set of player inputs under one or more relevant conditions; generating at least one virtual avatar within a virtual environment during a gameplay session, the at least one virtual avatar configured for collaborative gameplay control by a first source device that receives input from a player and a second source device that receives AI-based input according to the AI-based coaching model; identifying, based on one or more current conditions, a set of inputs received from the first source device as associated with an intention to perform a virtual action in the virtual environment, wherein the set of received inputs is incomplete or erroneous with respect to the virtual action; modifying the received set of inputs based on the intended virtual action and a set of AI-based inputs from the second source device; initiating performance of the virtual action by the at least one virtual avatar within the virtual environment according to the modified set of inputs; A method comprising:

2. 10. The method of claim 1, further comprising capturing one or more sets of inputs associated with the virtual action from a plurality of different source devices associated with different users during different gameplay sessions.

3. The method of claim 2 , further comprising identifying a sequence and timing associated with the set of inputs and one or more conditions of the game play session.

4. The method of claim 3 , further comprising tracking metadata about each of the conditions identified for the gameplay session.

5. 4. The method of claim 3, further comprising: storing in a memory a content control profile including the sequence and timing of the set of inputs; and updating the content control profile with one or more new sets of inputs received under similar conditions.

6. The method of claim 5 , further comprising customizing the content control profile for a particular source device based on historical data associated with the particular source device.

7. The method of claim 6 , wherein the historical data includes patterns of inputs associated with the one or more conditions in the virtual environment.

8. The method of claim 5 , wherein modifying the set of received inputs is further based on a combination of one or more inputs from the plurality of different source devices.

9. The method of claim 5 , further comprising supplementing the modified set of inputs based on inputs stored in the content control profile.

10. 2. The method of claim 1, wherein modifying the received set of inputs comprises combining the received set of inputs with the AI-based set of inputs to create a combined set of inputs.

11. The method of claim 10 further comprising identifying an incorrect or incomplete combination of the set of inputs.

12. 2. The method of claim 1, wherein modifying the received set of inputs further comprises using a set of AI-based control inputs from the second source device as a fallback according to a specified control level.

13. The method of claim 1 , further comprising prioritizing the set of received inputs from the first source device.

14. 1. A system for AI-based coaching in a virtual environment, comprising: a memory storing an artificial intelligence (AI)-based coaching model, the AI-based coaching model being trained to implement one or more coaching actions that modify a set of player inputs under one or more relevant conditions; A processor for executing instructions stored in a memory, said processor comprising: generating at least one virtual avatar within a virtual environment during a gameplay session, the at least one virtual avatar configured for cooperative gameplay control by a first source device that receives input from a player and a second source device that receives AI-based input according to the AI-based coaching model; identifying, based on one or more current conditions, a set of control inputs received from the first source device as associated with an intention to perform a virtual action in the virtual environment, wherein the set of received control inputs is incomplete or erroneous with respect to the virtual action; modifying the received set of control inputs based on the intended virtual action and the set of AI-based control inputs from the second source device; initiating performance of the virtual action by the at least one virtual avatar in the virtual environment according to the modified set of control inputs; Executes instructions to execute system.

15. 15. The system of claim 14, wherein the processor executes further instructions for performing the step of capturing one or more sets of inputs associated with the virtual action from a plurality of different source devices associated with different users during different gameplay sessions.

16. 15. The system of claim 14, wherein the processor modifies the received set of control inputs by combining the received set of control inputs with the AI-based set of control inputs to create a combined set of control inputs.

17. 17. The system of claim 16, wherein the processor executes further instructions for performing the step of identifying an incorrect or incomplete combination of the set of control inputs.

18. 15. The system of claim 14, wherein the processor modifies the received set of control inputs by using an AI-based set of control inputs from the second source device as a fallback according to a specified control level.

19. The system of claim 14 , wherein the processor executes further instructions for performing the step of prioritizing the received set of control inputs from the first source device.

20. 1. A non-transitory computer-readable storage medium storing a program executable by a processor to perform an AI-based coaching method in a virtual environment, the method comprising: storing an artificial intelligence (AI) based coaching model in a memory, said AI based coaching model being trained to implement one or more coaching actions that modify a set of player inputs under one or more relevant conditions; generating at least one virtual avatar within a virtual environment during a gameplay session, the at least one virtual avatar configured for collaborative gameplay control by a first source device that receives input from a player and a second source device that receives AI-based input according to the AI-based coaching model; identifying, based on one or more current conditions, a set of control inputs received from the first source device as associated with an intention to perform a virtual action in the virtual environment, wherein the received set of control inputs is incomplete or erroneous with respect to the virtual action; modifying the received set of control inputs based on the intended virtual action and the set of AI-based control inputs from the second source device; initiating performance of the virtual action by the at least one virtual avatar in the virtual environment according to the modified set of control inputs; 1. A computer-readable storage medium comprising:

Images