Motion-Based Navigation

Abstract

To provide an improved system and an improved method for determining navigation and access based on motion in a virtual environment.SOLUTION: One or multiple custom actions are stored in a memory. The respective custom actions are related to physical motion patterns and thresholds. Captured, physical motion data are received by a user's device and are mapped to one identified custom action. When the physical motion data match a related pattern of physical motion within the related threshold, it is determined that a user is permitted to access an identified custom action.SELECTED DRAWING: None

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Patent Application No. 63 / 223,816, filed July 20, 2021, the disclosure of which is incorporated herein by reference.

[0002] The present invention relates generally to motion-based navigation and access, and more particularly to making navigation and access decisions based on user motion in a virtual environment. [Background technology]

[0003] Currently available digital content titles support a variety of scenarios in which players (through their respective avatars or characters) can engage in various types of activities within a virtual environment. Whether playing individually, in teams, and / or against other players, players playing such content titles may be presented with rich virtual environments containing multiple different locations, levels, and other portions. Players can navigate and interact with such virtual environments (and associated virtual objects, entities, and other players' avatars) using controllers or other input devices. Some titles further enable players (and their friends and other social contacts) to create or customize user-generated content related to such interactions within the virtual environment.

[0004] Players typically control their respective avatars or characters using one or more types of game controllers. Currently, there are a variety of controllers (and controller modifications) with different configurations and layouts and with various buttons, touchpads, sensors, microphones, etc. Performing different types of in-game movement, manipulation, or other exercises in in-game actions may require different combinations of user inputs (e.g., sequences or synchronization of button presses, touchpad and other gestures, verbal commands, or other inputs). Because different game titles may include different activities, specific input combinations may result in different in-game movements. Furthermore, incorrectly entering input combinations may result in in-game actions different from those intended by the player. This variety and complexity of game titles, in-game actions, and controllers presents a barrier to entry when players are introduced to new or unfamiliar game titles, controllers, or game console systems.

[0005] Therefore, there is a need in the art for improved systems and methods for making motion-based navigation and access decisions in virtual environments. Summary of the Invention

[0006] Embodiments of the present invention include systems and methods for motion-based navigation. A computing device memory stores one or more custom actions associated with patterns of physical movement and thresholds. A navigation server in communication with the computing device receives physical movement data captured by a user's user device. The navigation server maps the physical movement data to one of the identified custom actions, matching the physical movement data with an associated pattern of physical movement within an associated threshold. The navigation server determines that the user is authorized to access the identified custom action. [Brief explanation of the drawings]

[0007] [Figure 1] 1 illustrates an exemplary network environment in which a system for motion-based navigation may be implemented. [Figure 2] 1 illustrates an exemplary Uniform Data System (UDS) that may be used to provide data to a system for motion-based navigation. [Figure 3] 1 is a flowchart illustrating an example method for motion-based navigation. [Figure 4] FIG. 1 is a block diagram of an example user device that may be used to perform motion-based navigation. [Figure 5] 1 is a block diagram of an exemplary electronic entertainment system that can be used with embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Embodiments of the present invention include systems and methods for motion-based navigation. A computing device memory stores one or more custom actions associated with patterns of physical movement and thresholds. A navigation server in communication with the computing device receives physical movement data captured by a user's user device. The navigation server maps the physical movement data to one of the identified custom actions, matching the physical movement data with an associated pattern of physical movement within an associated threshold. The navigation server determines that the user is authorized to access the identified custom action.

[0009] FIG. 1 illustrates an exemplary network environment 100 in which a system for sharing motion data may be implemented. The network environment 100 may include one or more content source servers 100 providing digital content for distribution (e.g., games, other applications, and services), one or more content provider server application program interfaces (APIs) 120, a content delivery network server 130, a navigation server 140, and one or more user devices 150A-150N. The servers described herein may include any type of server known in the art, including standard hardware computing components such as network and media interfaces, non-transitory computer-readable storage (memory), and a processor for executing instructions or accessing information that may be stored in the memory. The functionality of multiple servers may be integrated into a single server. Any of the aforementioned servers (or integrated servers) may exhibit specific client-side, cache, or proxy server characteristics. These characteristics may depend on the particular network placement of the servers or the particular configuration of the servers.

[0010] Content source server 110 may maintain and provide a variety of digital content available for distribution. Content source server 110 may be associated with any content provider that makes its content available for access over a communications network. Such content may include digital video and games, as well as other types of digital applications and services. Such applications and services may include any of a variety of different digital content and functionality that may be provided to user device 150. An example of content source server 110 may be associated with an internet website that provides downloadable and / or streaming content. The content provided by content source server 110 may include any type of multimedia content, such as movies, games, static / dynamic content, photos, social media content, social media websites, etc. User device 150 may include multiple different types of computing devices. In some embodiments, content data is transmitted from content source server 110 to a computing device, which then renders the content data in a format suitable for use by user device 150.

[0011] The content source server 110 may also include an online server associated with a social media application that may be programmed to provide one or more social media graphs to the user to identify the user's social contacts. These users may not be social contacts or socially associated with the user, but may have played one or more video games with the user. The friend list may include additional information about the user's friends, such as a description of the games each friend owns, an identification of the friend's online status (e.g., online, offline, inactive, etc.), the friend's last login and for how long, and the last game the friend played. The social network includes user data, including data such as the user's social graph, posts, photos, videos, and biographical information.

[0012] Content from content source servers 110 can be provided by content provider server APIs 120, which enable various types of content source servers 110 to communicate with other servers (e.g., user devices 150) in the network environment 100. The content provider server APIs 120 can be specific to the content source servers 110 providing the content and to the particular language, operating system, protocol, etc. of the user devices 150. In a network environment 100 that includes multiple different types of content source servers 110, there can likewise be a corresponding number of content provider server APIs 120, which enable various formats, conversions, and other cross-device and cross-platform communication processes for providing content and other services to different user devices 150, which can process such content using different operating systems, protocols, etc. In this manner, applications and services can be made available in different formats to be compatible with a variety of different user devices 150.

[0013] The content provider server API 120 can further facilitate each user device 150's access to content hosted or services provided by the content source server 110, either directly or through the content delivery network server 130. Additional information, such as metadata about the accessed content or service, can also be provided to the user device 150 by the content provider server API 120. As described below, the additional information (i.e., metadata) may be available to provide details about the content or service provided to the user device 150. In some embodiments, the services provided by the content source server 110 to the user device 150 via the content provider server API 120 can include supporting services associated with other content or services, such as chat services, ratings, and profiles associated with particular games, teams, communities, etc. In such cases, the content source servers 110 can also communicate with each other via the content provider server API 120.

[0014] The content delivery network servers 130 may include servers that provide resources, files, etc. related to content from the content source servers 110, including various content and service configurations with the user devices 150. The content delivery network servers 130 may also be invoked by user devices 150 requesting access to particular content or services. The content delivery network servers 130 may include universe management servers, game servers, streaming media servers, servers hosting downloadable content, and other content delivery servers known in the art.

[0015] Navigation server 140 may include any data server known in the art that is capable of receiving data from user device 150. The content rendered by navigation server 140 may be for essentially any type of computer application and may include one or more types of content, such as games, movies, audio, images, multimedia, etc., among others. In some embodiments, the content, or portions thereof, is generated by navigation server 140. In some embodiments, the content, or portions thereof, is streamed from content source server 110 to computing devices over network 100. In some embodiments, the content, or portions thereof, is streamed to computing devices over network 100 from cloud gaming infrastructure. The infrastructure may direct various types of content to be sent from content source server 110 to computing devices over network 100.

[0016] In an exemplary embodiment, navigation server 140 may store or access a store containing navigation control maps for one or more game titles. Each game title may be associated with a different navigation control map, which may include information about the in-game commands available for the respective game title (e.g., commands for accessing specific user-generated content, navigation or transport between locations in a virtual environment), and information about how the in-game commands correspond to real-world gestures (e.g., custom gestures, gestures selected from a menu). Some game titles may further restrict the types of commands that can be used in various levels, activities, or other portions of the associated in-game environment. Such restrictions may be saved in an activity file (described in further detail in connection with FIG. 2) generated by the UDS system and may be used to identify which in-game commands can be associated with particular real-world gestures.

[0017] Various real-world gestures (which can be customized by the user) can be associated with various in-game commands. For example, various combinations of verbal, key-based, or gestural commands may indicate that the user desires to input or initiate a particular in-game command. Such real-world gestures can be captured and characterized by cameras and / or other sensors (e.g., of user device(s) 150) to capture images and / or other sensor data related to the real-world environment in which the user is located. For example, a camera can capture images of the user performing physical gestures in the real world. A camera (discussed in further detail with respect to FIG. 4 ) can be configured to include multiple image capture devices, such as a stereoscopic pair of cameras, an infrared camera, a depth camera, or a combination thereof. In some embodiments, one or more microphones (discussed in further detail with respect to FIG. 4 ) can also be used to capture sounds from the user's real-world environment. Meanwhile, other sensors (e.g., accelerometers, gyroscopes) associated with the user (e.g., with a controller, mobile device, wearable device, handheld device, etc.) can also capture data related to real-world gestures.

[0018] Such image, audio, and other sensor data may be provided to navigation server 140 for analysis. Using stored maps, navigation server 140 may match received sensor data regarding real-world gestures (within a predetermined accuracy threshold) to in-game commands available in the virtual environment of the associated game title. In some implementations, the received sensor data may indicate multiple available in-game commands, and in such cases, different parameters may be used to filter the available in-game commands. As noted above, some game titles allow different in-game commands to be executed in different parts of the game environment or game session. Such data may be provided in an activity file and may be used to filter the available in-game commands and identify those that may be available in a specified part of the game environment or game session currently associated with the user. Alternatively, or in addition, a menu of multiple available in-game commands may be displayed in a graphical user interface for selection by the user.

[0019] In some embodiments, a user may wish to define custom real-world gestures and link them to specific in-game commands. Using user device 150, the user can add such customizations to a personal navigation map by performing the real-world gesture one or more times to configure and calibrate image or sensor data associated with the real-world gesture. Specific in-game commands can be selected from among the in-game commands available in the virtual environment. Additional parameters or conditions can also be defined, such that the same real-world gesture can have different effects depending on the specific game environment, game title, or other currently applicable parameters / conditions. For example, a user can define a secret handshake to authenticate access to private user-generated content or to identify an avatar associated with the user, a physical signal to send a predefined message to a friend or team member, a gesture to initiate a navigational movement to a designated meeting point within the virtual environment, a dance move to initiate an audiovisual effect, etc.

[0020] Thus, the navigation server 140 can map data about real-world gestures to one or more in-game commands and initiate input or otherwise communicate the same to the content source server 110, the content delivery network server 130, or the user device 150. Thus, a user actively engaged in a virtual environment can provide custom commands by performing gestures rather than the standard input commands supported by default in a game title. A user can further define custom commands that can be applied to a variety of different game environments. For example, a user can define a particular real-world gesture as being associated with a command to send a custom message (e.g., "Meet me at [location]") to a defined group of friend accounts.

[0021] The user device 150 may be a server in the network environment 100 that provides internal services (e.g., to other servers). In such cases, the user device 150 may correspond to one of the content source servers 110 described herein. Alternatively, the user device 150 may be a client device, which may include any number of different game consoles, mobile devices, laptops, and desktops. Such user devices 150 may also be configured to access data from other storage media, such as, but not limited to, memory cards or disk drives, which may be appropriate for downloaded services. Such user devices 150 may include standard hardware computing components, such as, but not limited to, a network interface, a media interface, non-transitory computer-readable storage (memory), and a processor for executing instructions that may be stored in the memory. These user devices 150 may also run a variety of different operating systems (e.g., iOS, Android®), applications, or computing languages ​​(e.g., C++, JavaScript®). An exemplary client device 150 is described in detail herein with respect to FIG. 5. Each user device 150 may be associated with a participant or other type of audience member in a collection of digital content streams.

[0022] In some embodiments, user device 150 is configured to run games locally on the processing hardware of a computing device or game console. The games or content are obtained in any format, such as a physical media format (e.g., digital disc, tape, card, thumb drive, solid-state chip or card, etc.), or by downloading from the Internet over a communications network. In some embodiments, the computing device acts as a client that communicates with a cloud gaming infrastructure over a computer network. The cloud gaming infrastructure can maintain and execute video games being played by user device 150. Inputs received from user device 150, controllers, and cameras (and other sensors) can be transmitted to the cloud gaming infrastructure, which processes the inputs and affects the game state of the running video game.

[0023] Game data from a video game, such as video data, audio data, and haptic feedback data, can be transmitted from content source server 110 to user device 150. The computer system may further process the game data before transmission to the appropriate device, or may transmit the game data directly to the appropriate device. For example, video and audio streams may be transmitted to user device 150.

[0024] User device 150 may also include a module configured to receive inertial sensor data from an inertial sensor within user device 150. The inertial sensor data indicates movement of user device 150 in response to movement of a user with whom user device 150 is associated. The user's movement is based on a virtual reality scene displayed within user device 150. A path of movement of user device 150 can be determined from the inertial sensor data and the rate of movement of user device 150. In some embodiments, the path of movement of the head-mounted display corresponds to one or more user movements within a range of user actions, including forward tilt, backward tilt, left tilt, right tilt, left head turn, right head turn, upward head tilt, downward head tilt, squat, and jump. However, in other embodiments, the path of movement of the head-mounted display may correspond to essentially any user movement within the capabilities of the human body.

[0025] In some embodiments, user device 150 can be used to manipulate, e.g., grab, move, push, pull, etc., virtual objects in, for example, a virtual reality (VR) or augmented reality (AR) scene, where the virtual objects are displayed on user device 150 or another computing device, e.g., a television, computer, etc. The virtual hand in the game moves when the user moves their hand while wearing user device 150. Furthermore, the fingers of the virtual hand in the game move when the user moves their fingers while wearing user device 150. The position and / or orientation of the fingers are determined from image data captured using the cameras described above and in FIG. 5 to generate the finger movements of the virtual hand.

[0026] The display of the user device 150 can generate a three-dimensional scene incorporating the avatar in a virtual space or environment with virtual interactive objects. The navigation server 140 receives visual and / or distance information about the user captured by the user device 150. The navigation server 140 extracts motion information describing the user's movements. The motion information can be encoded to represent the movement of the user's "skeleton," which consists of key points within the user's body.

[0027] A user's avatar can be automatically converted from a single digital image to an animated 3D avatar. Specifically, a user can upload a digital or other image to the user device 150 and, in exchange, receive one or more avatars featuring different facial expressions and / or animations. The avatars are displayed to the user and can be used as still or animated images (e.g., in GIF format). For example, the avatars can be displayed via SMS, MMS, email messages, chat (e.g., Facebook® Chat), instant messengers (e.g., Skype), and other messaging services. They can be sent to other users via social media platforms such as Windows Messenger, Twitter, blogs, forums, or any other means of electronic communication intended to display the user's feelings.

[0028] In some embodiments, the avatar can be realistically based on the user, for example, with the same facial features, clothing, and body type. In other embodiments, the avatar can be intentionally unrealistic, for example, by mapping the user's movements to the facial features of a celebrity, movie star, politician, video game character, or another user. The avatar may be a blend of realistic and intentionally unrealistic elements, for example, the user's face may be used in the avatar, but the avatar may be given different clothing or a different body type. Alternatively, the user's body may be used in the avatar, but the avatar may be given the face of a celebrity, movie star, politician, video game character, or another user.

[0029] The user device 150A transmits information describing the user's movements and / or the movements of the corresponding avatar to the second user device 150B or the navigation server 140, which then transmits this information to the second user device 150, thereby enabling the second user device 150B to generate a scene featuring accurate movements by the avatar. The user device 150A receives information describing the second user's movements and / or the movements of the corresponding second avatar from the second user device 150B or from an intermediate device such as the navigation server 140, and then transmits this information to the first user device 150A upon receiving it from the second user device 150B, thereby enabling the first user device 150A to generate a scene featuring accurate movements by the second avatar.

[0030] The user device 150 or computing device generates a representation of the user's movements as they are captured and causes an avatar to perform the generated representation of the user's movements. The user device 150 also generates a representation of the second user's movements as they are received and causes the second avatar to perform the generated representation of the second user's movements. The user device 150 also updates the virtual space or environment, as well as any virtual interactive objects, as needed.

[0031] When a user wants to access a custom action, such as a locked compartment or a restricted area or portion of a network or electronic storage area or database, the user performs a physical movement using the user device 150. The navigation server 140 can recognize a pattern of physical movement within the stored sequence of movements, find an associated custom action, and initiate that custom action on the user device 150. In an embodiment of the present invention, the user device 150 can be configured to always initiate an additional custom action in response to successful detection of a physical movement pattern. In one embodiment, a response or acknowledgement signal is generated. For example, the user device 150 can emit a sound (such as a short beep) to indicate that it has successfully decoded the rhythm. Another example is that the device can vibrate (using a vibration element, not shown).

[0032] Each custom gesture and command has a different level of security or threshold. For example, a more complex pattern of physical movements may be required to allow a high-level custom action to be entered, while a less complex or simpler pattern of physical movements may be required to allow a low-level custom action to be entered. A simpler pattern may require fewer steps and movements than a complex pattern. Security resource 104 may have multiple levels of security for different services. In that case, when a user is granted access to a protected resource and requests a particular service, a security token module of the security system determines whether the user is authorized to access the requested service according to the security token provided by the user.

[0033] FIG. 2 illustrates an exemplary Uniform Data System (UDS) 200 that can be used to provide data to a system for gesture-based skill search. Based on the data provided by the UDS, the gesture-based search server 140 can recognize in-game objects, entities, activities, and events in which a user has engaged and thus support analysis and filtering related to in-game activity. Each user interaction can be associated with metadata, such as the type of in-game interaction, its location within the in-game environment, and a specific point in time within the in-game timeline, as well as other associated players, objects, and entities. Thus, metadata can be tracked for any of the various user interactions that may occur during a game session, including associated activities, entities, settings, results, actions, effects, locations, character statistics, and the like. Such data can be further aggregated, applied to a data model, and subjected to analysis. Using such a UDS data model, contextual information can be assigned to each piece of information in a uniform manner across games.

[0034] 2, an exemplary console 228 (e.g., user device 150) and exemplary servers 218 (e.g., streaming server 220, activity feed server 224, user-generated content (UGC) server 232, and object server 226) are shown. In one example, console 228 may be implemented on or in association with content source server 110, content delivery network server 130, cloud server, or any combination of server 218 and the servers of FIG. 1. In an exemplary embodiment, content recorder 202 may receive and record content (e.g., media) from interactive content title 230 onto a content ring buffer 208. Such ring buffer 208 may store multiple content segments (e.g., v1, v2, and v3), start times for each segment (e.g., V1_START_TS, V2_START_TS, V3_START_TS), and end times for each segment (e.g., V1_END_TS, V2_END_TS, V3_END_TS). Such segments may be stored as media files 212 (e.g., MP4, WebM, etc.) by the console 228. Such media files 212 may be uploaded to the streaming server 220 for storage and subsequent streaming or use, although the media files 212 may be stored on any server, cloud server, any console 228, or any user device 150. The start and end times for each such segment may be stored by the console 228 as a content timestamp file 214. Such content timestamp file 214 may also include a stream ID that matches the stream ID of the media file 212, thereby associating the content timestamp file 214 with the media file 212.Such content timestamp files 214 may be uploaded and stored on the activity feed server 224 and / or UGC server 232, but the content timestamp files 214 may be stored on any server, cloud server, any console 228, or any user device 150.

[0035] At the same time that the content recorder 202 receives and records content from the interactive content title 230, the object library 204 receives data from the interactive content title 230, and the object recorder 206 tracks the data to determine when an object begins and ends. The object library 204 and the object recorder 206 may be implemented on the platform server 120, a cloud server, or any server 218. When the object recorder 206 detects the start of an object, it receives object data from the object library 204 (e.g., if the object is an activity, the user interaction with the activity, the activity ID, the activity start time, the activity end time, the activity result, the activity type, etc.) and records this activity data in the object ring buffer 210 (e.g., ActivityID1,START_TS; ActivityID2,START_TS; ActivityID3,START_TS). Such activity data recorded in the object ring buffer 210 can be stored in an object file 216. Such object files 216 may also include activity start time, activity end time, activity ID, activity result, activity type (e.g., competitive match, quest, task, etc.), and user or peer data related to the activity. For example, object files 216 may store data regarding items used during the activity. Such object files 216 may be stored on an object server 226, although object files 216 may be stored on any server, cloud server, any console 228, or any user device 130.

[0036] Such object data (e.g., object files 216) can be associated with content data (e.g., media files 212 and / or content timestamp files 214). In one example, the UGC server 232 stores and associates content timestamp files 214 with object files 216 based on a match between the stream ID of the content timestamp file 214 and the corresponding activity ID of the object file 216. In another example, the object server 226 can store object files 216 and receive queries for the object files 216 from the UGC server 232. Such queries can be performed by searching for an activity ID of the object file 216 that matches the stream ID of the content timestamp file 214 sent with the query. In yet another example, queries of stored content timestamp files 214 can be performed by matching the start and end times of the content timestamp file 214 with the start and end times of the corresponding object file 216 sent with the query. Such object files 216 can also be associated with matching content timestamp files 214 by the UGC server 232, although this association can be performed by any server, cloud server, any console 228, or any user device 150. In another example, the object files 216 and content timestamp files 214 can be associated by the console 228 during the creation of each file 216, 214.

[0037] The activity file generated by the UDS 200 can be provided or accessed by the navigation server 140 for use in analyzing, filtering, and matching data related to real-world gestures with in-game commands / actions in one or more maps. For example, the activity file may include gameplay data related to a particular gameplay session of a particular game title by a particular player who was engaged in a particular activity when a real-world gesture was detected to have been performed. The navigation server 140 can use data from the activity file to identify the game title, player, in-game activity, and other conditions under which the real-world gesture was performed. Such identified parameters or conditions under which the real-world gesture was performed can be used by the navigation server 140 to filter search results for relevance.

[0038] 3 is a flowchart illustrating an exemplary method for motion-based navigation. The method 300 of FIG. 3 may be embodied as executable instructions in a non-transitory computer-readable storage medium, including, but not limited to, non-volatile memory such as a CD, DVD, or hard drive. The instructions in the storage medium are executed by a processor(s) to cause various hardware components of a computing device that hosts or otherwise accesses the storage medium to perform the method. The steps (and their order) illustrated in FIG. 3 are exemplary and may include various alternatives, equivalents, or derivations thereof, including, but not limited to, the order of execution thereof.

[0039] In step 310, one or more custom actions are stored in memory. Each custom action is associated with a physical movement pattern and a threshold value. The threshold value corresponds to a level of the custom action. The level of the custom action is determined by the complexity of the physical movement pattern.

[0040] At step 320, physical movement data captured by the user's user device is received. At step 330, the physical movement data is mapped to one of the identified custom actions. The physical movement data is matched with an associated pattern of physical movement within an associated threshold.

[0041] At step 340, it is determined that the user is authorized to access the identified custom action. User device 150 may then be provided with access to the identified custom action. In some embodiments, the custom user gesture may be used as an authentication mechanism to lock and unlock content or actions within the virtual environment. For example, the custom gesture may be stored in association with a custom condition (e.g., matching a movement metric within a specified threshold) as a way to control access to personal or shared (team) content or actions. Thus, performing the custom gesture correctly (within a defined threshold) may result in the execution of a command that unlocks the ability to access specific content or actions / functions.

[0042] In some embodiments, a corresponding movement by an avatar associated with the user in a virtual environment may then be rendered based on the physical movement data, which one or more other users may find via one or more social networks associated with the user, and the corresponding movement by the avatar may be shared with one or more other users.

[0043] FIG. 4 is a block diagram of an exemplary user device that can be used to perform motion-based navigation. It should be understood that more or fewer components than those shown in FIG. 4 may be included or excluded from the user device 400, depending on the available configuration and functionality. The user device 400 may include a processor 402 for executing program instructions. Memory 406 is provided for data storage purposes and may include both volatile and non-volatile memory. A display 416 is included to provide a visual interface viewable by the user. The display 416 may be defined by one single display or in the form of separate display screens for each eye. If two display screens are provided, it is possible to provide video content for the left and right eyes separately. For example, displaying video content separately for each eye improves immersive control of three-dimensional content in a virtual reality scene.

[0044] The motion detection module 418 can include any of various types of motion-sensitive hardware, such as a magnetometer 420, an accelerometer 424, and a gyroscope 426. The user device 400 may include an inertial sensor 422 configured to generate inertial sensor data indicative of the movement of the user device 400 according to the movement of the user with whom the user device 400 is associated. The magnetometer 420 measures the strength and direction of magnetic fields near the user device 400. In some embodiments, three magnetometers 420 are used within the user device 400 to ensure an absolute reference for the world-space yaw angle. In some embodiments, the magnetometer 420 is designed to span the Earth's magnetic field of ±80 microstellar. The magnetometer is affected by metal and provides a yaw measurement that is monotonic with respect to the actual yaw. The magnetic field can be distorted by metal in the environment, which distorts the yaw measurement. If necessary, this distortion can be calibrated using information from other sensors, such as a gyroscope or camera. In some embodiments, an accelerometer 424 is used in conjunction with the magnetometer 420 to obtain the tilt and azimuth angles of the user device 400 .

[0045] In some embodiments, the present invention may also include one or more external inertial sensors positioned on the user's body. The present invention may include operations for comparing the external inertial sensor data with inertial sensor data received from the inertial sensor 422 within the user device 400 to determine a particular movement made by the user.

[0046] The accelerometer 424 is a device that measures acceleration and gravity-induced reaction forces. Single-axis and multi-axis (e.g., six-axis) models can detect the magnitude and direction of acceleration in different directions. The accelerometer 524 is used to sense tilt, vibration, and shock. In one embodiment, three accelerometers 524 are used to provide the direction of gravity, which provides an absolute reference for two angles: world-space pitch and world-space roll.

[0047] Gyroscope 426 is a device that measures or maintains orientation based on the principles of angular momentum. In one embodiment, three gyroscopes 426 provide information about movement across their respective coordinate axes (x, y, and z) based on inertial sensing. Gyroscopes 426 are useful for detecting high-speed rotations. However, gyroscopes 426 can drift over time if no absolute reference is present. This requires periodically resetting gyroscope 426, which can be done using other available information, such as visual tracking of objects, determining position / orientation based on accelerometers, magnetometers, etc.

[0048] A camera 404 is provided to capture images and image streams of the real-world environment to which the user device 400 is exposed. Multiple cameras 404 can (optionally) be included in the user device 400, including a rear-facing camera 404 (pointed away from the user when the user is looking at the display of the user device 400) and a front-facing camera 404 (pointed towards the user when the user is looking at the display of the user device 400). In some embodiments, the camera 404 can be included in the user device 400 to sense depth information of objects in the real-world environment to which the user device 400 is exposed.

[0049] The user device 400 includes a speaker 412 for providing audio output and may also include a microphone 414 for capturing audio from the real-world environment, including sounds from the surrounding environment, speech made by the user, etc.

[0050] A Wi-Fi module 410 may be included to enable connection of the user device 400 to the Internet via wireless network technology. The user device 400 may also include a Bluetooth module 408 to enable wireless connection to other devices.

[0051] It should be understood that the components of user device 400 as shown in Figure 4 are examples of components that may be included in user device 400 and do not represent all possible components that may be included in user device 400. For example, in various embodiments, user device 400 may or may not include some of the components shown in Figure 4. In some embodiments, user device 400 may include additional components not shown in Figure 4.

[0052] In an exemplary embodiment, a user can define custom real-world user gestures 430 that are associated with specific in-game commands. As shown in FIG. 4 , user gestures 430 can be full-body movements and / or any fine motor skill movements of one or more body parts for which data can be captured by a camera or sensor. Once defined, information about user gestures 430 can be stored in memory and made accessible to navigation server 140 for calibration and matching against movements and gestures later performed in the real-world environment monitored by the camera or sensor.

[0053] The navigation server 140 may identify that the user gesture 430 characterized by the sensor data is associated with one or more available in-game commands. The available in-game commands may be further filtered based on one or more activity files associated with the conditions under which the gesture was performed. For example, the gesture may have been performed during a particular gameplay session. Accordingly, a filtered subset of in-game commands may be identified as being associated with the game title of the gameplay session. Thus, the navigation server 140 may determine that the identified or selected in-game movement is mapped to a particular in-game command. The navigation server 140 may then execute the command within the in-game environment, resulting in navigation, providing access to particular content, player-to-player communication, audiovisual effects, and the like, being available within the associated content title.

[0054] 5 is a block diagram of an exemplary electronic entertainment system that can be used in embodiments of the present invention. Entertainment system 500 can be an electronic game console. Alternatively, entertainment system 500 can be implemented as a general-purpose computer, a set-top box, a handheld gaming device, a tablet computing device, a mobile computing device, or a mobile phone. Entertainment systems may include more or fewer operating components depending on their particular form factor, purpose, or design.

[0055] The CPU 510, vector unit 515, graphics processing unit 520, and I / O processor 525 of FIG. 5 communicate via a system bus 585. Additionally, the CPU 510 of FIG. 5 may communicate with main memory 505 via a dedicated bus 580, and the vector unit 515 and graphics processing unit 520 may communicate via a dedicated bus 590. The CPU 510 of FIG. 5 executes programs stored in the OS ROM 555 and the main memory 505. The main memory 505 of FIG. 5 may include pre-stored programs and programs transferred via the I / O processor 525 from a CD-ROM, DVD-ROM, or other optical disk (not shown) using the optical disk control unit 570. The I / O processor 525 of FIG. 5 may also enable the introduction of content transferred via wireless or other communication networks (e.g., 4G, LTE, 3G, etc.). The I / O processor 525 of FIG. 5 primarily controls the exchange of data between various devices of the entertainment system 500, including the CPU 510, the vector unit 515, the graphics processing unit 520, and the controller interface 535.

[0056] The graphics processing unit 520 of Figure 5 executes graphics instructions received from the CPU 510 and the vector unit 515 to generate images for display on a display device (not shown). For example, the vector unit 515 of Figure 5 may convert an object from three-dimensional coordinates to two-dimensional coordinates and send the two-dimensional coordinates to the graphics processing unit 520. Additionally, the audio processing unit 560 executes instructions to generate audio signals, which are output to an audio device such as a speaker (not shown). Other devices may be connected to the entertainment system 500 via the USB interface 545 and the IEEE 1394 interface 550, such as a wireless transceiver, which may be embedded within the system 500 or as part of some other component, such as a processor.

[0057] 5 provides instructions to CPU 510 via controller interface 535. For example, the user may instruct CPU 510 to store particular game information on memory card 540 or other non-transitory computer-readable storage medium, or may instruct a character in a game to perform a particular action.

[0058] The present invention may be implemented in an application that may be operable by a variety of end-user devices. For example, the end-user device may be a personal computer, a home entertainment system (e.g., Sony PlayStation2® or Sony PlayStation3® or Sony PlayStation4®), a portable gaming device (e.g., Sony PSP® or Sony Vita®), or a home entertainment system from a different, but subordinate, manufacturer. It is fully intended that the methods described herein be operable on a variety of devices. The present invention may also be practiced in a cross-title neutral manner, whereby embodiments of the system may be utilized across a variety of titles from a variety of publishers.

[0059] The present invention may be implemented in applications that may be operable using a variety of devices. A non-transitory computer-readable storage medium refers to any medium or media that participates in providing instructions to a central processing unit (CPU) for execution. Such media can take many forms, including but not limited to non-volatile and volatile media, such as optical or magnetic disks and dynamic memory, respectively. Common forms of non-transitory computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, any other magnetic media, CD-ROM disks, digital video disks (DVDs), any other optical media, RAM, PROM, EPROM, FLASHEPROM, and any other memory chips or cartridges.

[0060] Various forms of transmission media may be involved in carrying one or more sequences of one or more instructions to the CPU for execution. A bus carries the data to system RAM, and the CPU retrieves and executes the instructions from the system RAM. The instructions received by the system RAM may optionally be stored on a fixed disk either before or after execution by the CPU. Various forms of storage may be implemented, along with network interfaces and network topologies necessary to implement the storage.

[0061] The above detailed description of the present technology has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present technology to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were selected to best explain the principles of the technology, its practical application, and to enable those skilled in the art to utilize the technology in various embodiments and with various modifications suitable for the particular uses contemplated. It is intended that the scope of the present technology be defined by the claims.

Claims

1. 1. A method for motion-based navigation, comprising: storing one or more custom actions in a memory, each of the custom actions being associated with a physical movement pattern corresponding to a set of predetermined metrics and an associated set of thresholds for the set of predetermined metrics, the thresholds being determined by the complexity of the physical movement pattern; receiving physical movement data captured by a user device of a user, the physical movement data including a plurality of metrics; mapping metrics of the received physical movement data to an identified set of pre-defined metrics corresponding to one of the custom actions, wherein the metrics of the received physical movement data match one or more of the set of pre-defined metrics corresponding to the associated pattern of physical movement; and determining whether the user is allowed access to the identified custom action based on a set of thresholds associated with matching the set of predetermined metrics, wherein if the captured physical movement is performed within the associated thresholds, the identified custom action can be executed within a virtual environment currently associated with the user of the user device, and if the captured physical movement is performed beyond at least one of the thresholds in the set, the identified custom action is restricted from execution within the virtual environment.

2. initially receiving user input defining a real-world gesture associated with the identified custom action, the physical movement pattern being associated with the real-world gesture defined by the user input; 10. The method of claim 1, further comprising: analyzing the received user input to identify the corresponding set of predetermined metrics, the corresponding set of predetermined metrics being stored in association with the identified custom action.

3. 10. The method of claim 1, further comprising: receiving a request from the user device to share the identified custom action with one or more contacts; and sending a notification regarding the identified custom action to a user device associated with the contact.

4. The method of claim 1 , wherein the identified custom action comprises navigating an avatar associated with the user to a specified location within the virtual environment.

5. The method of claim 1 , wherein the identified custom action is used to lock or unlock specified content within the virtual environment.

6. The method of claim 1 , wherein the identified custom action is associated with one or more designated contacts of the user of the user device.

7. The method of claim 1 , wherein the identified custom action comprises generating a communication within the virtual environment.

8. The method of claim 1 , wherein the identified custom action is associated with one or more audiovisual effects within the virtual environment.

9. 2. The method of claim 1, wherein the stored map of identified custom actions is specific to at least one of a game title, a set of game titles, a game genre, a game developer, a game console, a game controller, a set of controller modifications, a game environment, and an in-game activity.

10. 1. A system for motion-based navigation, comprising: a memory storing one or more custom actions, each of the custom actions being associated with a physical movement pattern corresponding to a predetermined set of metrics and an associated set of thresholds for the predetermined set of metrics, the thresholds being determined by the complexity of the physical movement pattern; a communication interface for receiving physical movement data captured by a user device of a user and transmitted over a communication network, the physical movement data including a plurality of metrics; A processor that executes instructions stored in a memory, the processor executing the instructions to: mapping metrics of the received physical movement data to an identified set of pre-defined metrics corresponding to one of the custom actions, wherein the metrics of the received physical movement data match one or more of the set of pre-defined metrics corresponding to the associated pattern of physical movement; determining whether the user is permitted access to the identified custom action based on the set of thresholds associated with matching the set of predetermined metrics, wherein if the captured physical movement is performed within the associated thresholds, the identified custom action is permitted to execute within a virtual environment currently associated with the user of the user device, and if the captured physical movement is performed beyond at least one of the thresholds in the set, the identified custom action is restricted from execution within the virtual environment. the processor.

11. the communication interface further comprises initially receiving user input defining a real-world gesture associated with the identified custom action, the pattern of physical movement being associated with the real-world gesture defined by the user input; 11. The system of claim 10, wherein the processor executes instructions to further analyze the received user input to identify the corresponding set of predetermined metrics, the corresponding set of predetermined metrics being stored in association with the identified custom action.

12. 11. The system of claim 10, wherein the communication interface further receives a request from the user device to share the identified custom action with one or more contacts and sends a notification regarding the identified custom action to a user device associated with the contact.

13. The system of claim 10 , wherein the identified custom action comprises navigating an avatar associated with the user to a specified location within the virtual environment.

14. The system of claim 10 , wherein the identified custom action is used to lock or unlock specified content within the virtual environment.

15. The system of claim 10 , wherein the identified custom action is associated with one or more designated contacts of the user of the user device.

16. The system of claim 10 , wherein the identified custom action comprises generating a communication within the virtual environment.

17. The system of claim 10 , wherein the identified custom action is associated with one or more audiovisual effects within the virtual environment.

18. 11. The system of claim 10, wherein the stored map of identified custom actions is specific to at least one of a game title, a set of game titles, a game genre, a game developer, a game console, a game controller, a set of controller modifications, a game environment, and an in-game activity.

19. 1. A non-transitory computer-readable storage medium having an embodied program executable by a processor to perform a method for motion-based navigation, the method comprising: storing one or more custom actions in a memory, each of the custom actions being associated with a physical movement pattern corresponding to a set of predetermined metrics and an associated set of thresholds for the set of predetermined metrics, the thresholds being determined by the complexity of the physical movement pattern; receiving physical movement data captured by a user device of a user, the physical movement data including a plurality of metrics; mapping metrics of the received physical movement data to an identified set of pre-defined metrics corresponding to one of the custom actions, wherein the metrics of the received physical movement data match one or more of the set of pre-defined metrics corresponding to the associated pattern of physical movement; and determining whether the user is allowed access to the identified custom action based on a set of thresholds associated with matching the set of predetermined metrics, wherein if the captured physical movement is performed within the associated thresholds, the identified custom action is allowed to execute in a virtual environment currently associated with the user of the user device, and if the captured physical movement is performed beyond at least one of the thresholds in the set, the identified custom action is restricted from execution in the virtual environment.

Images