Virtual reality notification sync

Abstract

In a method for synchronously presenting notifications created in a VR environment in a real environment at other times, the notifications may be mapped across all associated environments and associated with conditions that, when triggered, cause the notification to be presented to a user. The computer-implemented method may include receiving a natural language request at a computing device in a first environment at a first time, parsing the natural language request into semantic components including an alert component and a content component, determining one or more conditions based on the alert component and the content component, mapping the semantic components to the multiple environments, and executing, by the computing device, the notification, where the notification renders the alert component and the content component in a second environment of the multiple environments when the one or more conditions are satisfied at a second time, subsequent to the first time.

Description

[Technical Field]

[0001] The present invention relates generally to the field of virtual reality, and more particularly to synchronizing notifications across virtual reality and real-world environments. [Background technology]

[0002] Interaction within a virtual reality (VR) environment can be achieved using head-mounted displays and other interface devices and can be used in a wide variety of applications. Wearable VR devices allow users to visually immerse themselves in VR worlds or environments that are completely virtual or a combination of virtual and physical reality. VR devices and environments have been used for work, shopping, gaming, virtual travel, and other forms of entertainment. While using a VR device to access a particular VR environment, users are partially or completely disconnected from the real world. In most cases, users are also completely disconnected from other VR environments that may be accessible at other times. As users navigate within a VR environment, various VR features may or may not be available. For example, a user may perform tasks in one VR environment that cannot be performed in another. This presents a drawback for users who want to perform tasks in one environment and translate those tasks into actions or events in another environment. Another drawback is that currently, while a user may have various ways to create notifications (e.g., alerts) specific to the user's current environment (e.g., virtual or real), the user cannot create or modify notifications that take effect in other environments that the user may access or visit at other times. Disjointed VR environments pose challenges for users who wish to synchronize multiple VR environments with tasks and events created in one VR environment. Summary of the Invention

[0003]

[0006] Embodiments of the present invention disclose a computer-implemented method, computer program product, and system for synchronizing notifications across virtual reality and real-world environments. The computer-implemented method for synchronizing notifications across virtual reality and real-world environments may include one or more processors configured to: receive a natural language request at a computing device operating in a first environment of a plurality of environments at a first time, the plurality of environments each including one or more parameters, the plurality of environments being accessible by a user associated with the computing device; parse the natural language request into semantic components including an alert component and a content component; determine one or more conditions based on the alert component and the content component; map the semantic components to the plurality of environments; and execute a notification by the computing device, the notification rendering the alert component and the content component in a second environment of the plurality of environments if the one or more conditions are met at a second time, after the first time.

[0004] A solution provides an advantage to the challenges faced in the prior art by synchronizing multiple VR environments so that an alert or notification created in one environment is triggered in other VR environments when certain conditions are met. Such an advantage allows a user to be immersed in the full realm of VR scenery, create an event or notification in one VR environment, and then move to other VR environments where the event or notification will similarly apply. Because the event or notification is synchronized in all applicable VR environments, a condition associated with the event or notification can be triggered in the appropriate VR environment, requiring the user to perform an associated action or receive an associated set of information, as determined when the event or notification was created.

[0005] As a further advantage, the computer-implemented method may further include converting the natural language request into text data using semantic analysis. Preferably, the computer-implemented method may further include identifying an alert component within the natural language request as a device command instructing the computing device to generate a notification. As another advantage, the computer-implemented method may further include identifying a content component within the natural language request as a user command instructing the user to perform a task. Preferably, the one or more conditions may include one or more of a geographic condition corresponding to a location, an activity condition corresponding to a user event, and a user condition determined based on the content component. Advantageously, the first environment may be a virtual reality environment and the second environment may be a real environment, or vice versa. Preferably, the notification may be one or more of an auditory alert and a visual alert representing the content component, and is presented to the user via VR goggles communicatively coupled to the computing device. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a functional block diagram illustrating a distributed data processing environment for synchronizing notifications across virtual reality and real-world environments, according to one embodiment of the present invention. [Figure 2] 1 is a flowchart of a system for synchronizing notifications across virtual reality and real-world environments, according to one embodiment of the present invention. [Figure 3] 1 is a flowchart of a method for synchronizing notifications across virtual reality and real world environments, according to one embodiment of the present invention. [Figure 4] 2 is a block diagram of a computing device in the distributed data processing environment of FIG. 1 in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0007] The embodiments described herein provide computer-implemented methods, computer systems, and computer program products that allow a user to create alerts or notifications in the real world or multiple VR environments (e.g., for work, shopping, gaming, or other forms of human activity). Embodiments of the present invention recognize that an alert entered in one environment can be highly useful when presented in another environment, whether real or virtual. Alerts can be synchronized between two or more of the environments, including the real world and the VR world. Alerts can be presented based on conditions specified at the time of alert creation or thereafter. The user or other users can be notified of the alert during activities in one of the environments, depending on the conditions specified as part of the alert properties.

[0008] While using a VR device to access a particular VR environment, a user is partially or completely disconnected from the real world and, in most cases, completely disconnected from other VR environments the user may be using at other times. A user may use various methods to create notifications (e.g., alerts) specific to the user's current environment (e.g., a real environment or a virtual environment), but may be unable to create or modify notifications that are valid in environments the user visits at other times. For example, at a first time, a user may be using a VR device in a VR environment and the user creates a notification to remind the user to perform a task when one or more conditions are met (e.g., at a particular time, in a particular location, upon observing a particular item), but the user may be unable to create that notification in other environments because the user is not in or does not have access to those environments. Thus, a notification created by a user in a VR environment may be presented to the user in other environments when one or more conditions established when the notification was created are met.

[0009] In one embodiment, a computer-implemented method may include one or more processors configured to receive user input via a user interface of a user device associated with a user, where the user input may be processed to create or modify notifications (e.g., alerts) in either the real environment or the virtual environment in which the user device resides. The notifications (e.g., alerts) may then be synchronized across all environments and applied according to conditions specified or determined when the notifications were created or thereafter.

[0010] Synchronizing multiple VR environments so that an alert or notification created in one environment is triggered in other VR environments when certain conditions are met provides advantages over challenges faced in the prior art. Such advantages allow a user to be immersed in the full range of VR scenery, allowing the user to create an event or notification in one VR environment and then move to other VR environments where the event or notification will similarly apply. Because the event or notification is synchronized in all applicable VR environments, the condition associated with the event or notification may be triggered in the appropriate VR environment, in which case the user may be invited to perform an associated action or receive an associated set of information, as determined when the event or notification was created.

[0011] In one embodiment, a computer-implemented method may include one or more processors configured to detect a user device associated with a user interacting within a VR environment. A user interface may be generated and configured to receive user input during interaction within the VR environment to create one or more notifications (e.g., alerts) within the VR environment based on one or more selected conditions. The one or more processors may be configured to apply the notifications according to conditions specified or determined when or after the notifications are created while the user is accessing another VR environment or in a real-world environment.

[0012] In one embodiment, a computer-implemented method may include one or more processors configured to receive user input for creating one or more notifications (e.g., alerts) based on one or more selected conditions within a real-world environment, wherein the notifications may be presented for detection by the user when the user is detected within one or more particular VR environments and the one or more selected conditions are met while in one or more of the particular VR environments.

[0013] In one embodiment, the user input may correspond to instructions to create a notification to be received by the user or another user. In other words, the one or more processors may be configured to receive user input from a first user to create one or more notifications to be presented to a second user when one or more conditions are met while the second user is detected within a particular VR environment or a particular real-world environment. The user input to create the notifications may be via text entry, a graphical user interface (GUI), speech-to-text, text-to-speech, or any other method of user data entry known to one skilled in the art.

[0014] In one embodiment, the one or more processors may be configured to consider multiple factors when creating a notification and associating specified conditions for triggering the notification. For example, the location of the environment (e.g., VR or real), geofencing area, time, contextual circumstances, etc. may be considered when mapping or synchronizing the notification and associated conditions to one or more environments. Other VR and real environment parameters may be considered or incorporated into one or more environments and used when determining whether to present the notification to the intended user. In one embodiment, notification parameters (e.g., user location, user behavior, or timing information, or a combination thereof) may be continuously considered and compared to the notification parameters to determine whether conditions for presenting the notification to the intended user are met. In one embodiment, the notification parameters may also include the location and / or behavior of other users (e.g., users other than the user responsible for creating the notification) while in an environment (e.g., a VR environment or a real environment or both) when determining when to present the notification to the intended user.

[0015] By considering multiple factors when creating notifications and associating specified conditions for triggering the notifications, the embodiments described herein allow for a great deal of customization to be applied to each notification. Furthermore, it is advantageous to synchronize notifications between each applicable VR environment so that a user can move freely throughout the domain and still be presented with appropriate notifications when conditions are met.

[0016] In one embodiment, the one or more processors may be configured to receive input for creating a notification from a user by receiving parameter input corresponding to a condition for presenting the notification. For example, the condition for presenting the notification may include determining whether the user is in an environment corresponding to a certain store type or a particular geographic location before presenting the notification to the user. Another example of a condition for presenting the notification may include determining whether a certain amount of time has passed before presenting the notification to the user. Multiple conditions may be established to trigger the presentation of a notification to the user.

[0017] In one embodiment, the notification may be in any form perceptible to the user. For example, the notification may include an audio alert, a visual alert, a tactile alert, a message alert, or any combination thereof. The notification may include details corresponding to multiple actions performed by one or more of the user devices associated with the user. Additionally, the notification may include details corresponding to multiple actions performed by one or more user devices not associated with the user (e.g., a second user device associated with the first user device).

[0018] In one embodiment, a first user may set a conditional notification on a first user device, which, if met, may include a second user device presenting the notification to the second user. For example, a notification set by a first user may trigger a notification to make a phone call or send a text message, which is presented to the second user by the second user device. The notification may also include parameters or instructions regarding the environment in which the notification should be presented. For example, a user may provide a condition in which the notification should be presented to the user or other users only if the user or other users are in a specific environment determined by detecting the intended user's geographic location. The notification parameters may also include information regarding the specific users who should receive the notification based on the user's permissions, the environment in which the notification should be received, and timing parameters for when the notification should be presented.

[0019] In one embodiment, the computer-implemented method may include one or more processors configured to receive user input for creating one or more notifications (e.g., alerts) based on one or more selected conditions when not within a real-world environment, wherein the notifications may be presented when the one or more selected conditions are met while the user is not or is detected within a particular VR environment and is not or is within the particular VR environment. The multiple environments may each include one or more parameters, and the multiple environments may be accessible by a user associated with the computing device.

[0020] In one embodiment, a computer-implemented method may include one or more processors configured to synchronize (e.g., map) one or more notifications within one or more environments (e.g., VR environments, real-world environments) based on one or more conditions specified by a user or determined by the one or more processors.

[0021] Once one or more notifications are synchronized or successfully mapped to one or more environments, the one or more processors may be configured to determine whether a user is detected in the one or more environments and, if one or more conditions associated with the one or more notifications are satisfied, present the one or more notifications to the intended user.

[0022] In one embodiment, a user associated with one or more user devices (e.g., a smartphone, VR goggles, etc.) may be browsing items in a store while wearing the one or more user devices (e.g., VR goggles coupled to a smartphone). As the user browses the store, the system may be configured to detect that the user purchases a first item and views a second item. The one or more processors may be configured to analyze the user's shopping behavior (e.g., purchasing the first item and viewing the second item) and generate a notification to alert the user about the second item based on one or more environmental parameters corresponding to the current user environment and / or a subsequent user environment, and based on one or more conditions that may be set by the user or determined by the one or more processors.

[0023] In other embodiments, a computer-implemented method may include one or more processors configured to execute instructions on a computing device (e.g., a user device) that maintain a mapping between a real environment and a virtual environment by analyzing contextual parameters and conditions in the real environment and the VR environment. For example, while a user wears the VR user device and moves through the VR environment, the VR user device may be configured to analyze VR content present in the VR environment to identify the user's virtual geographic location (e.g., virtual coordinates). Furthermore, while a user wears the VR user device and moves through the VR environment, the VR device may be configured to identify the user's virtual contextual location, such as a shopping center, train station, or other user-accessible point of interest. Furthermore, while a user wears the VR user device and moves through the VR environment, the VR device may be configured to identify or detect the presence of other users present in the real environment or the VR environment.

[0024] In examples in which the VR device is configured to detect the presence of other users in the VR environment, the one or more processors may be configured to process the presence of the other users to determine whether a condition exists to trigger an alert. For example, an alert may be created while the user is in a first environment to "remind me to say happy birthday to Trayvon." Subsequently, when the VR device detects Trayvon's presence in the other environment using sensors in the VR device, the one or more processors may be configured to present a notification to the user conveying the message "Say happy birthday to Trayvon." The notification may be presented as an audio message played from speakers connected to the VR device, as a text message visibly displayed to the user, or as a unique haptic notification indicating an alert to be seen or received.

[0025] In one embodiment, one or more processors may be configured to identify a virtual geofencing area using a contextual analysis of VR content present in the VR environment. When performing the contextual analysis of the VR content, the one or more processors may be configured to process data corresponding to time, weather, user activity, user response, and environmental parameters (e.g., VR environment parameters, real-world environment parameters, or both), such as geographic location, timing, etc. When performing the contextual analysis to generate a notification, multiple factors (e.g., time, weather, user activity, user response, and / or environmental parameters) from the real-world and VR environments may be combined. For example, a VR user device may be activated and in use by a user to participate in a tour having one or more tour paths, in which case the VR user device is configured to execute instructions corresponding to a tour guide operator for a particular path. During the tour of the first path, the VR user device may determine that it is at a first location (e.g., a first photo stop on a path in a national park) based on the location data. Further, the VR user device may be configured to determine weather conditions (e.g., visibility, precipitation, or temperature, or a combination thereof) based on the weather data. Further, the VR user device may be configured to determine time data (e.g., time of day) and use the time data to determine a suitable time to participate in the tour. Furthermore, the VR user device may determine that the selected time is not suitable for participating in the tour and provide a first notification, detected by the user, to detour from the first tour path to a second tour path. Multiple notifications, detected by the user of the VR user device, may be provided. For example, in addition to the first notification, the VR user device may be configured to provide a second notification to provide information that may be relevant to the user. The second notification may include information telling the user, "Enjoy the view!" Further, the notification may include an optional message for the user to receive and make a decision (e.g., whether to detour to another path or continue on the current path).

[0026] Implementation of embodiments of the present invention may take a variety of forms, and details of exemplary implementations are discussed below with reference to the figures.

[0027] 1 illustrates a functional block diagram of a distributed data processing environment for synchronizing notifications across virtual and real-world environments, generally designated 100, in accordance with one embodiment of the present invention. As used herein, the term "distributed" refers to a computer system that includes multiple physically separate devices that operate together as a single computer system.

[0028] 1 is intended to provide an illustration of one embodiment of the present invention and does not imply any limitation with regard to the environments in which different embodiments may be implemented. As shown in FIG. 1, a distributed data processing environment 100 for synchronizing notifications across virtual reality and real-world environments includes a network 110 configured to facilitate communication between a database 124, a server 125, a user device 130, and a virtual reality headset 132.

[0029] Network 110 operates as a computing network that may be, for example, a local area network (LAN), a wide area network (WAN), or a combination of the two, and may include wired, wireless, or fiber optic connections. In general, network 110 may be any combination of connections and protocols that support communication between database 124, server 125, user device 130, and virtual reality headset 132. It is further understood that in some embodiments, network 110 is optional and distributed data processing environment for synchronizing notifications across virtual and real environments 100 may operate as a standalone system, while in other embodiments, network 110 may be configured to allow user device 130, virtual reality headset 132, or both, to share a collaborative database using network 110.

[0030] The user device 130 may be an electronic device configured to be carried by a user. The user device 130 may be a personal electronic device, such as a mobile communication device, a smartphone, a tablet, a personal digital assistant, a smart wearable device, a personal laptop computer, a desktop computer, or any other electronic device configured to interact with a user, collect user information, and generate a user profile. In the illustrated embodiment, the user device 130 includes a user interface 122 and sensor(s) (not shown). The user device 130 may include components described in further detail in FIG. 4.

[0031] The user interface 122 operates as a local user interface on the user device 130 through which one or more users of the user device 130 interact with the user device 130. In some embodiments, the user interface 122 is a local app interface for a program (e.g., software configured to perform the inventive steps described herein) on the user device 130 or virtual reality headset 132. In some embodiments, the user interface 122 is a graphical user interface (GUI), web user interface (WUI), or voice user interface (VUI), or combination thereof, that can display (i.e., visually), present (i.e., audibly), or allow a user to input or receive information (i.e., graphics, text, or sound, or a combination thereof) about or from the program over the network 110. In one embodiment, the user interface 122 allows a user to send and receive data (i.e., to and from the program, respectively, over the network 110). In one embodiment, the user interface 122 allows a user to opt in to the program, enter user-related data, and receive alerts for completing tasks or activities.

[0032] The virtual reality headset 132 may be an electronic device configured to be a component configured to provide access to a VR environment. The electronic device may include wireless sensors, software, actuators, and computing devices. The virtual reality headset 132 may be controlled from a remote control system over the network 110, from a local control system over a local network, or a combination of both. Additionally, the virtual reality headset 132 may be configured to be controlled through software applications installed and executed on the virtual reality headset 132 or the user device 130. When connected to a network, the virtual reality headset 132 may communicate usage data and other types of data corresponding to itself or other devices connected via the network 110, which may provide useful insights within the scope of the designed application. The virtual reality headset 132 may be configured with a processor, memory, and peripherals (not shown) for receiving and processing data. The virtual reality headset 132 may include components described in further detail in FIG. 4.

[0033] In some embodiments, a user may wear special gloves and / or utilize handheld controllers to perform actions in the VR environment in conjunction with a virtual reality headset 132. Optionally, one or more wearable sensors may be utilized to obtain various data about the user as the user performs actions in the virtual environment. This data may include, but is not limited to, location data, biometric data, and / or ambient environmental data.

[0034] Database 124 may act as a repository for data related to server 125, user device 130, virtual reality headset 132, and other data transmitted within network 110. A database is an organized collection of data. Database 124 may be implemented using any type of storage device, such as a database server, hard disk drive, or flash memory, capable of storing data and configuration files that can be accessed and utilized by server 125, user device 130, or virtual reality headset 132, or a combination thereof. In one embodiment, database 124 may be accessed by server 125, user device 130, or virtual reality headset 132, or a combination thereof, to store data related to user device 130 or virtual reality headset 132. In other embodiments, database 124 may be accessed by user device 130 or virtual reality headset 132 to access data as described herein. In one embodiment, database 124 may exist independently of network 110. In other embodiments, database 124 may reside elsewhere within distributed data processing environment 100 , provided that database 124 is accessible to network 110 .

[0035] In the illustrated embodiment, server 125 may include a program (i.e., software) configured to perform the inventive steps described herein. In some embodiments, server 125 may be a standalone computing device(s), management server(s), web server(s), mobile computing device(s), or any other electronic device(s) or computing system(s) capable of receiving, transmitting, and processing data. In some embodiments, server 125 may be a laptop computer, tablet computer, netbook computer, personal computer (PC), desktop computer, smartphone, or any programmable electronic device capable of communicating with user device 130 and virtual reality headset 132 over network 110. In other embodiments, server 125 represents a server computing system that utilizes multiple computers as a server system, such as a cloud computing environment. In yet another embodiment, server 125 represents a computing system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that function as a single pool of seamless resources when accessed within distributed data processing environment 100. Server 125 may include components described in further detail in FIG.

[0036] FIG. 2 illustrates a flow chart of a system 200 for synchronizing notifications across virtual and real-world environments, according to one embodiment of the present invention.

[0037] In one embodiment, system 200 may include one or more processors configured to receive user input 202 via a user interface of a user device (e.g., smartphone 230a) associated with a user, and the user input 202 may be processed to create or modify notifications 203 (e.g., alerts) in either the virtual environment (e.g., virtual kitchen 240) or real environment (e.g., grocery store 250) in which the user device (e.g., smartphone 230a) is present. The notifications 203 may then be synchronized across all environments (e.g., virtual kitchen 240 and grocery store 250) and applied according to conditions specified or determined at or after the time the user input 202 was used to create the notification 203.

[0038] In one embodiment, the system 200 may include one or more processors configured to detect one or more user devices (e.g., smartphone 230a and VR glasses 230b) associated with a user interacting within the VR environment. While interacting within the VR environment, a user interface may be generated or made available and configured to receive user input 202 for creating one or more notifications 203 based on one or more selected conditions within the VR environment. The one or more processors may be configured to apply the notification 203 according to conditions specified or determined when or after the user input 202 is received and processed to create the notification 203 while the user is accessing another VR environment or in a real-world environment. Once the notification 203 is applied, the one or more processors may be configured to present the notification 203 for recognition by the user according to conditions associated with the notification 203.

[0039] The VR glasses 230b may be an electronic computing device that provides virtual reality capabilities. This may include a virtual reality headset, a mobile computing device (e.g., a smartphone, tablet computer, or other suitable computing device), a laptop computer, a desktop computer, or a wearable computing device, or a combination thereof. In an embodiment, a user uses the VR glasses 230b to navigate within a virtual environment (world) and perform virtual actions associated with tasks that correspond to similar tasks in the real world. During task performance, various biometric and / or environmental data is acquired. This data facilitates product evaluation in the virtual environment and allows for estimation of product sales in similar real-world environments.

[0040] In one embodiment, system 200 may include one or more processors configured to receive user input 202 for creating one or more notifications 203 based on one or more selected conditions within a real-world environment, and notification 203 may be presented for detection by the user when the user is detected within one or more particular environments and one or more selected conditions are met while in one or more of the particular environments.

[0041] In one embodiment, the user input 202 may correspond to instructions to create a notification 203 to be received by the user or received by another user. In other words, the one or more processors may be configured to receive user input 202 from a first user for creating one or more notifications 203 to be presented to a second user when one or more conditions are met while the second user is detected within a particular VR environment or a particular real-world environment. Receiving user input 202 for creating a notification 203 may include input via text input, a graphical user interface (GUI), speech-to-text, text-to-speech, or any other method of user data input known to one of skill in the art.

[0042] In one embodiment, the system 200 may include one or more processors configured to consider multiple factors when creating the notification 203 and associating the conditions specified for triggering the notification 203. For example, the location of the environment, geofencing area, time, contextual conditions, etc. may be considered when mapping or synchronizing the notification 203 and associated conditions to one or more environments (e.g., a VR environment, a real-world environment, or both). Other VR and real-world environment parameters may be considered or incorporated into one or more environments and used in determining whether to present the notification to the intended user. In one embodiment, notification parameters (e.g., user location, user behavior, or timing information, or a combination thereof) may be continuously considered and compared to the notification parameters to determine whether the conditions for presenting the notification 203 to the intended user are met. In one embodiment, the notification parameters may also include the location and / or behavior of other users (e.g., users other than the user responsible for creating the notification) while in the environment (e.g., a VR environment, a real-world environment, or both) when determining when to present the notification to the intended user.

[0043] In one embodiment, the system 200 may include one or more processors configured to receive input 202 for creating the notification 203 from a user by receiving parameter input corresponding to a condition for presenting the notification 203. For example, the condition for presenting the notification 203 may include determining whether the user is in an environment corresponding to a certain store type or a particular geographic location before presenting the notification 203 to the user. Another example of a condition for presenting the notification 203 may include determining whether a certain amount of time has passed before presenting the notification 203 to the user. Multiple conditions may be established to trigger the presentation of the notification 203 to the user.

[0044] In one embodiment, system 200 may include one or more processors configured to convert user input 202 from speech to text using information extraction techniques and semantic analysis to process user input 202. User input 202 may include instructions to perform a task, and the task may be to purchase a particular item 201. In one embodiment, item 201 may be a garlic string of multiple garlic bulbs, and item 201 may be identified via VR glasses 230b while a user browses through virtual kitchen 240. For example, if user input 202 includes the text "Don't forget to buy garlic," one or more processors may be configured to identify the action to be performed as "buy," identify the object to be bought or purchased as "garlic," and identify the location to buy the garlic as "grocery store." The actions, objects, and locations determined by performing information extraction and semantic analysis may be stored in notification database 224 for further processing and transmission.

[0045] In one embodiment, system 200 may include one or more processors configured to send data corresponding to notification 203 to smartphone 230a for presentation to a user of smartphone 230a when a condition established for presenting notification 203 is met. Additionally, one or more processors may be configured to compare location data and other parameters with the data corresponding to notification 203 to determine whether and when notification 203 should be presented to the user. For example, if the condition associated with notification 203 corresponds to the user being near a real-world environment corresponding to grocery store 250, one or more processors may be configured to present notification 203 to the user when smartphone 230a determines that the user is near a real-world environment corresponding to grocery store 250.

[0046] In one embodiment, notification 203 may be in any form perceptible to the user. For example, notification 203 may include an audio alert, a visual alert, a tactile alert, a message alert, or any combination thereof. Notification 203 may include details corresponding to multiple actions performed by one or more of the user devices associated with the user (e.g., smartphone 230a, VR glasses 230b). Additionally, notification 203 may include details corresponding to multiple actions performed by one or more user devices not associated with the user (e.g., a second user device associated with a first user device, a second user device not associated with the first user).

[0047] In one embodiment, a first user may provide user input 202 via a first user device (e.g., smartphone 230a, VR glasses 230b) to create a conditional notification 203, which, if met, may include one or more processors configured to present the notification 203 to a second user associated with a second user device (not shown). For example, user input 202 for creating a notification 203 by a first user may trigger the notification 203 to make a phone call or send a text message, which is presented to the second user by the second user device (not shown). The notification may also be created to include parameters or instructions regarding the environment in which the notification should be presented. For example, a user may provide a condition to present the notification to the user or other users only if the user or other users are in a specific environment determined by detecting the intended user's geographic location. The notification parameters may also include information regarding specific users who should receive the notification based on the user's permissions, the environment in which the notification should be received, and timing parameters for when the notification should be presented.

[0048] In one embodiment, system 200 may include one or more processors configured to receive user input 202 for creating one or more notifications 203 based on one or more selected conditions when not within a real environment, and notification 203 may be presented to the user when the one or more selected conditions are met even while the user is not detected and not within a particular VR environment.

[0049] In one embodiment, system 200 may include one or more processors configured to synchronize (e.g., map) one or more notifications 203 with one or more environments (e.g., VR environments and / or real environments) based on one or more conditions specified by a user or determined by one or more processors.

[0050] Once one or more notifications 203 are synchronized or successfully mapped to one or more environments, the one or more processors may be configured to determine whether a user is detected in the one or more environments and, if one or more conditions associated with the one or more notifications 203 are met, present the one or more notifications 203 to the intended user.

[0051] In one embodiment, a user associated with one or more user devices (e.g., smartphone 230a, VR glasses 230b) may be browsing items in a store while wearing VR glasses 230b coupled to smartphone 230a. As the user browses the store, system 200 may be configured to detect, via the one or more user devices, that the user is purchasing a first item and viewing a second item. One or more processors may be configured to analyze the user's shopping behavior (e.g., purchasing a first item and viewing a second item) and generate notification 203 to alert the user about the second item based on one or more conditions, which may be set by the user or determined by the one or more processors, based on one or more environmental parameters corresponding to the current user environment and / or a subsequent user environment.

[0052] In other embodiments, system 200 may include one or more processors configured to execute instructions on a computing device (e.g., smartphone 230a, VR glasses 230b) that maintain a mapping between the real environment and the virtual environment by analyzing contextual parameters and conditions in the real environment and the VR environment. For example, while a user wears a VR user device (e.g., VR glasses 230b) and moves through a VR environment (e.g., virtual kitchen 240), the VR user device may be configured to analyze VR content present in the VR environment to identify the user's virtual geographic location (e.g., virtual coordinates). Furthermore, while a user wears a VR user device (e.g., VR glasses 230b) and moves through a VR environment (e.g., virtual kitchen 240), the VR device (e.g., VR glasses 230b) may be configured to identify the user's virtual context location, such as a shopping center, train station, or other user-accessible point of interest. Furthermore, while a user wears a VR user device and moves through a VR environment, the VR device may be configured to identify or detect the presence of other users present in the real environment or the VR environment.

[0053] FIG. 3 illustrates a flowchart of a computer-implemented method 300 for synchronizing notifications across virtual and real-world environments, according to one embodiment of the present invention.

[0054] In one embodiment, computer-implemented method 300 may include one or more processors configured to receive 302 a natural language request at a computing device operating within a first environment of a plurality of environments at a first time, at least one of the plurality of environments being a VR environment. For example, the natural language request may correspond to user input in the form of text data, audio input, or other user data intended as user input to the computing device. The audio input may include a voice utterance that may be converted into audio data via one or more processors. The audio data may be analyzed for phonemes and converted into text for further processing. In some embodiments, natural language processing may be performed on-board a user device (e.g., user device 130) and server 125 of FIG. 1 . In other embodiments, all or a portion of the natural language processing may be performed on a remote computer.

[0055] In one embodiment, audio data may be analyzed using a natural language processing (NLP) system, such as a machine learning system. Additionally, machine learning systems may be used to categorize and classify input data, including biometric data acquired from sensors, image data, scenes, object recognition and / or object classification, person recognition, natural language processing (NLP), sentiment analysis, and / or other classification processes. The machine learning system may include one or more neural networks, convolutional neural networks (CNNs), or other deep learning techniques, or a combination thereof. The machine learning system may include regression algorithms, classification algorithms, clustering techniques, anomaly detection techniques, Bayesian filtering, or other suitable techniques, or a combination thereof, to analyze information acquired by one or more processors to assist in categorizing the information.

[0056] In one embodiment, the computer-implemented method 300 may include one or more processors configured to parse 304 the natural language request into semantic components, including an alert component and a content component. For example, parsing 304 the natural language request into semantic components may include converting the natural language request into text data using semantic analysis. Further, parsing 304 the natural language request may include identifying text corresponding to an action as the alert component and identifying text corresponding to an object, location, or both as the content component.

[0057] In one embodiment, the computer-implemented method 300 may include one or more processors configured to identify an alert component in the natural language request as a device command or action that directs a computing device to generate a notification.

[0058] In one embodiment, the computer-implemented method 300 may include one or more processors configured to identify content components within the natural language request as user commands that instruct a user to perform a task.

[0059] In one embodiment, the computer-implemented method 300 may include one or more processors configured to determine 306 one or more conditions based on the alert component and the content component. In one embodiment, the one or more conditions may include one or more of a geographic condition corresponding to a location, an activity condition corresponding to a user event, and a user condition determined based on the content component. A user event may include a user activity corresponding to a natural human action performed at a particular time and / or place. For example, a user event may include a user walking into a grocery store, driving past a retail store, or jogging past a clothing store. A user event may be determined by one or more processors processing data collected by sensors in communication with a computing device associated with the user.

[0060] In one embodiment, the computer-implemented method 300 may include one or more processors configured to map 308 semantic components to multiple environments.

[0061] In one embodiment, the computer-implemented method 300 may include one or more processors configured to perform 310, by a computing device, a notification, the notification expressing an alert component and a content component in a second environment of the plurality of environments when one or more conditions are met at a second time, the second time being after the first time.

[0062] In one embodiment, the first environment may include a virtual reality environment and the second environment may include a real environment.

[0063] In one embodiment, the notification may be one or more of an audible and a visual alert representing the content component.

[0064] 4 illustrates a block diagram of a computing device 400 in the distributed data processing environment 100 of FIG. 1, such as a server 125, a user device 130, or a virtual reality headset 132, or a combination thereof, in accordance with one embodiment of the present invention. It should be understood that FIG. 4 is intended as an illustration of one implementation and is not intended to imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

[0065] Computer 400 includes a communications fabric 402 that provides communication between cache 416, memory 406, persistent storage 408, communications unit 410, and input / output (I / O) interface(s) 412. Communications fabric 402 can be implemented with any architecture designed to pass data and / or control information between processors (e.g., microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components in the system. For example, communications fabric 402 can be implemented with one or more buses or crossbar switches.

[0066] Memory 406 and persistent storage 408 are computer-readable storage media. In this embodiment, memory 406 includes random access memory (RAM). Generally, memory 406 may include any suitable volatile or non-volatile computer-readable storage medium. Cache 416 is high-speed memory that enhances the performance of computer processor(s) 404 by holding recently accessed data and data near recently accessed data from memory 406.

[0067] Software and data 414 may be stored in persistent storage 408 and memory 406 via cache 416 for execution and / or access by one or more of the respective computer processors 404. In one embodiment, persistent storage 408 includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage 408 may include a solid-state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0068] The media used by persistent storage 408 may also be removable. For example, a removable hard drive may be used for persistent storage 408. Other examples include optical and magnetic disks, thumb drives, and smart cards, which are inserted into a drive and transferred onto other computer-readable storage media that are also part of persistent storage 408.

[0069] The communications unit 410, in these examples, provides for communication with other data processing systems or devices. In these examples, the communications unit 410 includes one or more network interface cards. The communications unit 410 may provide communications using one or both physical and wireless communications links. Software and data 414 may be downloaded to persistent storage 408 via the communications unit 410.

[0070] The I / O interface(s) 412 allow for the input and output of data with other devices that may be connected to the database 124, the server 125, the user device 130, or the virtual reality headset 132, or a combination thereof. For example, the I / O interface 412 may provide a connection to an external device 418, such as a keyboard, a keypad, a touch screen, or any other suitable input device, or a combination thereof. The external device 418 may also include a portable computer-readable storage medium, such as a thumb drive, a portable optical or magnetic disk, and a memory card. Software and data 414 used to practice embodiments of the present invention may be stored on such a portable computer-readable storage medium and loaded into the persistent storage 408 via the I / O interface(s) 412. The I / O interface(s) 412 are also connected to a display 420.

[0071] Display 420 provides a mechanism for displaying data to a user and may be, for example, a computer monitor.

[0072] The programs described herein are identified based on the applications for which they are implemented in specific embodiments of the invention. However, it should be understood that any specific program names herein are used for convenience only, and therefore the invention should not be limited to any particular application identified and / or implied by such names.

[0073] The present invention may be a system, a computer-implemented method, or a computer program product, or any combination thereof. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0074] A computer-readable storage medium may be any tangible device capable of retaining and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Memory Stick®, floppy disk, mechanically encoded devices such as punch cards or grooved ridge structures having instructions recorded thereon, and any suitable combination thereof. Computer-readable storage medium, as used herein, should not be construed as a transitory signal per se, such as, for example, radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over wires.

[0075] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, fiber optic transmission cables, wireless transmission cables, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0076] The computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the last scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or a connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may be personalized by utilizing state information of the computer-readable program instructions to execute the computer-readable program instructions to perform aspects of the present invention.

[0077] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0078] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when the instructions are executed by the processor of the computer or other programmable data processing apparatus, means are generated for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored constitutes an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0079] Furthermore, computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device and caused to perform a series of operational steps on the computer, other programmable apparatus, or other device to generate a computer-implemented process that, when executed on the computer, other programmable apparatus, or other device, implements the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0080] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order depicted. For example, depending on the functionality involved, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may possibly be executed in the reverse order. It will also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions.

[0081] The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present invention. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements over commercially available technologies, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A method for synchronizing notifications across a virtual reality environment and a real world environment, comprising: receiving, by one or more processors, a natural language request at a computing device operating within a virtual reality environment of a plurality of environments at a first time, the plurality of environments each including one or more parameters, the plurality of environments being accessible by a user associated with the computing device; parsing, by the one or more processors, the natural language request into semantic components including an alert component and a content component; determining, by the one or more processors, one or more conditions based on the alert component and the content component; mapping, by the one or more processors, the semantic components to the plurality of environments; executing, by the one or more processors, a notification with the computing device, the notification representing the alert component and the content component in a real environment of the plurality of environments when the one or more conditions are satisfied at a second time that is later than the first time; maintaining, by the one or more processors, a mapping between the real environment and the virtual reality environment by analyzing contextual parameters and conditions in the real environment and the virtual reality environment; A method comprising:

2. converting, by the one or more processors, the natural language request into text data using semantic analysis. The method of claim 1 further comprising:

3. identifying, by the one or more processors, the alert component in the natural language request as a device command instructing the computing device to generate the notification.

3. The method of claim 1 or 2, further comprising:

4. identifying, by the one or more processors, the content components in the natural language request as user commands directing a user to perform a task. The method of any one of claims 1 to 3, further comprising:

5. 5. The method of claim 1, wherein the one or more conditions include one or more of a geographic condition corresponding to a location, an activity condition corresponding to a user event, and a user condition determined based on the content component.

6. 6. The method of claim 1, wherein maintaining a mapping between the real environment and the virtual reality environment includes analyzing virtual reality content present in the virtual reality environment to identify, by the computing device, the user's virtual geographic location, the user's virtual contextual location, or other users present in the virtual reality environment or the real environment.

7. 7. The method of claim 1, wherein the notification is one or more of an auditory and visual alert representative of the content component and is presented to the user via VR goggles communicatively coupled to the computing device.

8. A computer program product for causing a computer to carry out the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having the computer program according to claim 8 recorded thereon.

10. 1. A computer system for synchronizing notifications across a virtual reality environment and a real world environment, comprising: one or more computer processors; one or more computer-readable storage media; program instructions collectively stored on the one or more computer-readable storage media for execution by at least one of the one or more computer processors; wherein the stored program instructions include: program instructions for receiving, at a first time, a natural language request at a computing device operating within a virtual reality environment of a plurality of environments, each of the plurality of environments including one or more parameters, the plurality of environments being accessible by a user associated with the computing device; program instructions for parsing the natural language request into semantic components, including an alert component and a content component; program instructions for determining one or more conditions based on the alert component and the content component; program instructions for mapping the semantic components to the plurality of environments; program instructions for executing, by the computing device, a notification, the notification representing the alert component and the content component in a real environment of the plurality of environments when the one or more conditions are satisfied at a second time that is later than the first time; program instructions for maintaining a mapping between the real environment and the virtual reality environment by analyzing contextual parameters and conditions in the real environment and the virtual reality environment; 1. A computer system comprising:

11. program instructions that convert said natural language request into text data using semantic analysis; 11. The computer system of claim 10, further comprising:

12. program instructions that identify the alert component in the natural language request as a device command that instructs the computing device to generate the notification; program instructions that identify the content components in the natural language request as user commands that instruct a user to perform a task; 12. The computer system of claim 10 or 11, further comprising:

13. 13. The computer system of claim 10, wherein the one or more conditions include one or more of a geographic condition corresponding to a location, an activity condition corresponding to a user event, and a user condition determined based on the content component.

14. 14. The computer system of claim 10, wherein maintaining a mapping between the real environment and the virtual reality environment includes analyzing virtual reality content present in the virtual reality environment to identify, by the computing device, the user's virtual geographic location, the user's virtual contextual location, or other users present in the virtual reality environment or the real environment.

15. 15. The computer system of claim 10, wherein the notification is one or more of an auditory and visual alert representative of the content component and is presented to the user via VR goggles communicatively coupled to the computing device.

Images