Augmented reality interactions

Abstract

A computer-implemented method for allowing a first user 200 to interact with a second user 210 in an augmented or virtual reality environment comprises receiving a signal associated with the first use

Description

Technical Field The present disclosure relates to a computer-implemented method for allowing a first user to interact with a second user in an augmented or virtual reality environment, a data processing system for carrying out said method, and a computer program product comprising instructions which causes a computer to carry out said method. Background In mixed-reality, the physical or real world and the virtual world work together to create an augmented or virtual reality environment. It is desirable to create a seamless transition between the virtual and real-world parts of the environment that a user is experiencing. Images are displayed virtually, which may overlap or coincide with the physical or real-world environment. This can include, for example, displaying virtual objects (e.g. three dimensional interactive images) on tabletops, walls, chairs and floors of the real-world environment. Summary Aspects of the invention are set out in the independent claims, further features according to embodiments are set out in the dependent claims. In a first aspect of the invention, a computer-implemented method for allowing a first user to interact with a second user in an augmented or virtual reality environment, the method comprising: receiving a signal associated with the first user; determining that the first user can interact with the second user based on the signal associated with the first user generating a user interface menu for display to the first user comprising one or more interaction options for interaction with the second user; determining an optimum display location, scale and / or position within the augmented or virtual reality environment for displaying the user interface menu to the first user based on one or more inputs; displaying the user interface menu to the first user in the optimum position for a selection of a desired interaction option by the first user; and initiating the desired interaction option between the first user and the second user based on the selection of the first user from the user interface menu. In a second aspect of the invention, a data processing system comprising means for carrying out the method of the first aspect. In a third aspect of the invention, a computer program product comprising instructions which, when the program is executed by a computer, causes the computer to carry out the method of the first aspect. The first aspect (and thus the second and third aspects) of the present invention thus advantageously provides a method by which two users may interact with each other in an augmented or virtual reality environment based on signals received from the first user that interaction is desired. To facilitate the interaction, the method determines the optimum location (e.g. where in augmented or virtual reality environment), scale (e.g. the size, e.g. the relative size with respect to another item, object or the user) and / or position (e.g. where with respect to the user) to display the user interface menu. This advantageously allows the user interface menu to be displayed such that the first user can select the desired interaction with minimum impact on their virtual experience within the augmented or virtual reality environment. For example, the optimum location for the user interface menu may be next to the location of the second user with the optimum position being at eye level with the first and / or second user such that the first user does not have to search the augmented or virtual reality environment to find the user interface menu. This advantageously allows the system to adapt dynamically to the experience context and the users’s needs. For example, the system can adapt based on the accessibility requirements of the user, thus improving the users experience. The user experience is also improved by enhancing user collaboration, a sense of virtual community and thus an enhanced mixed-reality environment and experience. Furthermore, providing the user with the means to interact with the second user allows both users to cooperate and provide a more realistic user experience within the augmented or virtual reality environment. It will be appreciated that the method steps of the first aspect may be performed in any desired order and the order in which the method steps are presented is not limiting. However, optionally, the method steps may be performed in the order in which they are presented above. For example, the method comprises first, receiving the signal associated with the first user; second, determining that the first user can interact with the second user based on the signal associated with the first user; third, generating a user interface menu for display to the first user comprising one or more interaction options for interaction with the avatar of the second user; fourth, determining an optimum display location, scale and / or position within the augmented or virtual reality environment for displaying the user interface menu to the first user based on one or more inputs; fifth, displaying the user interface menu to the first user in the optimum position for a selection of a desired interaction option by the first user; and sixth, initiating the desired interaction option between the first user and the avatar of the second user based on the selection of the first user from the user interface menu. Brief Description of Drawings A more complete understanding of the subject matter may be derived by referring to the detailed description and claims when considered in conjunction with the following figures, wherein like reference numbers refer to similar elements throughout the figures. Figure 1 shows a flow diagram of a method according to an embodiment of the disclosure; Figure 2 shows a schematic representation of part of the method in accordance with an embodiment of the present disclosure; Figure 3 shows a schematic representation of part of the method in accordance with an embodiment of the present disclosure; Figure 4 shows a schematic representation of part of the method in accordance with an embodiment of the present disclosure; Figure 5 shows an example system arranged to implement the present disclosure; and Figure 6 shows a block diagram of one example implementation of a computing device. Detailed Description The following detailed description is merely illustrative in nature and is not intended to limit the embodiments of the subject matter or the application and uses of such embodiments. As used herein, the words “exemplary” and “example” mean “serving as an example, instance, or illustration.” Any implementation described herein as exemplary or an example is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, or the following detailed description. Optionally, the augmented reality environment comprises, at least in part, the physical environment of a user and a virtual part of the environment of the user. As such, the augmented or virtual reality environment may allow further users (such as the second user), who are not physically present in the same physical environment as the first user, to share the augmented or virtual reality environment of the first user. For example, an avatar associated with a second user may be located on the sofa next to the first user in the augmented or virtual reality environment (even though the second user is not physically present on the sofa next to the first user in the real world). In alternative examples, the second user may be physically present on the sofa next to the first user and may experience a similar augmented or virtual reality environment to the first user. Alternatively, the environment may be a completely virtual environment in which avatars of both the first and second users are present. For example, the virtual reality environment may mimic a holiday destination or landmark destination corresponding to a real-world environment or a fantastical environment. Optionally, the augmented or virtual reality environment presented to the first user and the second user is the same environment (i.e. both users see an augmented or virtual reality environment that mimics the first users physical environment). Optionally, the augmented or virtual reality environment experienced by the first and second user is different. For example, the first user may see an augmented or virtual reality environment based on their physical environment whilst the second user sees an augmented or virtual reality environment based on their different physical environment. As such, an avatar associated with the first user may be located within the second users augmented or virtual reality environment whilst an avatar associated with the second user may be located within the first users augmented or virtual reality environment. As such, the reality of the augmented reality experience is improved for both users. In a virtual environment, both users will be represented by avatars. Optionally, the interaction between the first user and the second user occurs via an avatar associated with the second user. For example, the method allows the first user to interact with an avatar associated with (e.g. the avatar of) the second user in the augmented or virtual reality environment. For example, the desired interaction option is initiated between the first user and an avatar associated with the second user. Optionally, the interaction between the first user and the second user occurs via avatars associated with each of the first user and the second user. For example, the method allows an avatar associated with (e.g. the avatar of) the first user to interact with an avatar associated with (e.g. the avatar of) the second user. For example, the desired interaction option is initiated between an avatar associated with the first user and an avatar associated with the second user. Optionally, the interaction between the first user and the second user occurs via an avatar associated with the first user. For example, the method allows an avatar associated with (e.g. the avatar of) the first user to interact with the second user. For example, the desired interaction option is initiated between an avatar associated with the first user and the second user. Optionally, the signal associated with the first user comprises one or more biometric signals. Optionally, the one or more biometric signals comprise data representative of one or more of the first users: eye-movement, gaze location or direction, heart rate, facial expression, brain activity (e.g. indicative of emotional state), electrodermal activity and / or pupil dilation. Optionally, the step of receiving the signal associated with the first user comprises: receiving biometric data from one or more biometric sensors associated with the first user; wherein the one or more biometric sensors may comprise a heart rate monitor; an optical sensor, a thermometer; a camera; or electroencephalogram (EEG) sensors. Optionally, the biometric sensors may be integrated within gaming devices used by the first user in the real world or physical environment as part of engaging in the augmented or virtual reality environment. For example, the gaming device includes an augmented reality (AR) headset or a virtual reality (VR) headset worn by the users, wherein the AR and VR headsets comprises a display that presents the augmented or virtual reality environment to the users. For example, the AR / VR headset may further comprise a camera for detecting the user’s eye movement dynamics and / or facial expressions and / or pupil dilation. The AR / VR headset may further comprise detectors for determining a position and rotation of the users head. The gaming device may also include a gaming controller (i.e. a handheld controller) which is used to control movement (e.g. of an avatar) and / or selection of activities within the augmented or virtual reality environment. For example, the gaming controller may further comprise a heart rate monitor, thermometer, positional and rotational detectors (such as a compass) and / or an accelerometer. Optionally, the biometric sensors may be stand alone sensors (i.e. sensors separate from any other component). Optionally, the method further comprises: receiving a signal associated with the second user; and determining that the second user or their avatar is available for interaction with the first user. Optionally, the signal associated with the second user comprises one or more biometric signals of the second user. Optionally, the one or more biometric signals comprise data representative of one or more of the first users: eye-movement, gaze direction, heart rate, facial expression, brain activity (e.g. indicative of emotional state), electrodermal activity and / or pupil dilation. Optionally, the step of receiving the signal associated with the second user comprises: receiving biometric data from one or more biometric sensors associated with the second user; wherein the one or more biometric sensors may comprise a heart rate monitor; an optical sensor, a thermometer; a camera; or an electroencephalogram (EEG) sensor. Optionally, the signal associated with the second user comprises a status of the second user. For example, the status of the second user may include indications that the second user is busy and / or engaged in an activity, and is therefore unable to immediately interact with the first user. Interaction may be undesirable for the second user as an interaction would disrupt the second user’s current activity. For example, the status of the second user may indicate that the second user is engaged with another user (i.e. not the first user) or game characters (i.e. allies or antagonists that are not tied to a user, e.g. a boss), or in the middle of a game activity . In another example, the status of the second user may indicate or include a progress stage of a game that the second user is engaged in. The status of the second user may indicate whether the second user has completed a particular activity (e.g. a particular stage in a game). As such, the status may indicate that the second user is more or less open to an interaction with the first user (e.g. a celebratory indication to consolidate finishing a level or game activity). Optionally, the step of determining that the second user is available for interaction with the first user may comprise comparing the status with a second users pre-set preferences for interacting with users. For example, the second user may have preset preferences including that they never wish to be interrupted during a game activity, or that they don’t wish to be interrupted during activity of a particular level of difficulty ,and / or when the user is located in a particular part (e.g. room) of the augmented or virtual reality environment. This thus improves the quality of the second users experience within the augmented or virtual reality environment as they are not distracted at important moments. Optionally, when it is determined that one or more (e.g. a threshold number, e.g. all of) the pre-set preferences are not met, the method comprises determining that the second user is not available for interaction. Displaying the user interface menu to the first and second users may be based on a positive determination that the biometric data does not fall outside of the biometric data threshold ranges. As such, the first user is not able to interact with the second user and thus the quality of the second user’s experience is preserved. Optionally, the step of determining that the second user is available for interaction with the first user comprises comparing one or more features of the second user, such as each of the one or more biometric signals, to a threshold range and / or comparing each of the second user’s status to their pre-set preferences for interacting with users. Features may relate to the second users playing of the game, for example. Optionally, the method comprises determining that the second user is not available for interaction based on a comparison of the biometric data to the pre-set preferences. Optionally, when it is determined that a threshold number of the pre-set preferences are not met (e.g. biometric signals being outside of the threshold range and / or the status requirements are not met), the method comprises determining that the second user is not available for interaction. Optionally, each of the biometric signals and status are assigned an importance factor. For example, a biometric signal or status being assigned “high” importance means that a measurement outside of a threshold range or the status not being met results in the determination that the second user is not available for interaction regardless of the other biometric signals or status received. For example, a status or biometric signal being assigned a “moderate” importance means that a status condition being met or a measurement outside of the threshold range only results in the determination that a second user is not available for interaction if there is at least one (e.g. at least two, e.g. at least three, e.g. at least a pre-determined number of) other failed result (e.g. one other status or measurement of a biometric signal outside of a threshold range). For example a status or biometric signal being assigned a “low” importance means that a status condition being met or a measurement outside of the threshold range only results in the determination that a second user is not available for interaction if all of the other biometric signals are also outside of their associated threshold ranges. As such, different biometric signals may be provided different degrees of significance corresponding to the precision of the sensor measurement and / or the error associated with said measurements. Optionally, each of the pre-set preferences (e.g. biometric signals or status) may be assigned an importance value which are then used to indicate an interaction score for the user. For example, the importance value may be 1 for high, 2 for medium and 3 for low importance and the interaction score is an average of the pre-set preferences that are measured. It may thus be determined that a user is not available for interaction when the interaction score falls below a threshold. Alternatively, if the importance value is 3 for high, 2 for medium and 1 for low, the interaction score may indicate the user is not available for interaction when the interactions score exceeds a threshold. It will be appreciated that any suitable and / or desirable value system may be used. It will be appreciated that the threshold range for each biometric signal may be predetermined based on a user’s unique biometric makeup. For example, the threshold range may be set based on a set of calibration measurements associated with the user at rest. The extreme values of the threshold range may be set based on the error associated with the sensors used to make the measurements, calibration data associated with the user in different conditions (such as high stress and / or excitement) and standard values. Optionally, the threshold range for each biometric signal may be dynamically updated using machine learning to refine the meaning of different biometric readings. For example, if it is identified that a second user regularly declines and / or ignores an interaction with a first user at a specific biometric signal value (even though the biometric signal falls within the threshold range), the method may update the threshold range associated with that signal accordingly to exclude that signal. Optionally, the step of displaying the user interface menu is based on a positive determination that one or more (e.g. two or more, e.g. three or more, e.g. all) of the one or more biometric signals falls within their associated threshold range. Optionally, when it is determined that the second user is not available for interaction, the method further comprises monitoring the signal associated with the second user until it is determined that the biometric signal is within the threshold range and / or the pre-set preferences are met. Additionally or alternatively, the method may further comprise alerting the first user that the second user is unavailable for interaction. Optionally, when it is determined that the second user is not available for interaction, the method further comprises alerting the first user that the second user is not available and presenting the user with one or more options. The options may include: the option to wait for the second user to be available; the option to be notified when the second user is available; and / or the option to abandon the interaction with the second user. Optionally, the interaction options (e.g. for interaction with the second user and / or their associated avatar) comprise one or more of: a gesture, presenting a gift (e.g. to the second user), initiating a conversation (e.g. with the second user) and / or sending a message (e.g. to the second user). Optionally, the gesture may include one or more of: a wave, a high-five, and / or a hand-shake. Optionally, one of the one or more inputs (e.g. for determining the optimum display location) is a relative position of the second user with respect to first user. Optionally, the relative position comprises: a relative distance between the first user and the second user; and / or a relative angular displacement between the first user and the second user. For example, it may be appreciated that, if a first user (or their avatar) is relatively far away from the second user (or their avatar) within the augmented or virtual reality environment, presenting the display next to the second user (or their avatar) may result in the user interface menu being too small for the first user to read and / or engage with. As such, it may be determined that the user interface menu should be located closer to the first user and / or that the scale of the user interface menu should be increased to improve the usability of the menu. Optionally, one of the one or more inputs is a relative size of the second user with respect to the first user. Optionally, one of the one or more inputs is a relative size of an avatar associated with the second user in the augmented or virtual reality environment with respect to the fist user. Optionally, the one or more inputs may be classified (i.e. comprise classified information). For example, the one or more inputs may comprise inputs from the real-life environment of the user or inputs from the first or second user. Optionally, one of the one or more inputs is contextual information relating to the second user and / or the first user. Optionally, the contextual information may comprise the augmented or virtual reality environment, such as the type of game environment (e.g. of the first user or their associated avatar, e.g. of the second user or their associated avatar). Optionally, the contextual information may comprise a location of the first user in the augmented or virtual reality environment. For example, whether the first user (or their associated avatars) are in a reception room (e.g. a waiting room) or a activity room. Optionally, the contextual information may comprise a location of the second user (or their associated avatar) in the augmented or virtual reality environment. For example, whether the first user (or their associated avatars) are in a reception room (e.g. a waiting room) or an activity room. Optionally, the contextual information may comprise a relative location of the first user with respect to the second user. For example, whether the first user and the second user are in the same part (e.g. room) of the augmented or virtual reality environment or different parts (e.g. rooms). Optionally, the user interface menu is displayed in a location that, from the perspective of the first user, is proximate to the second user in the augmented or virtual reality environment. Optionally, the step of initiating the desired interaction option comprises providing the first user and / or the second user with a prompt associated with the interaction. Optionally, the prompt may be one of an audio or visual count-down timer indicating the timing of the interaction and / or a visual target indicating the location of the intended interaction. Optionally, the step of initiating the desired interaction option comprises providing a feedback signal associated with the interaction. Optionally, the feedback signal may be one or more of a haptic signal, a visual signal, or an audio signal. Figure 1 shows a flow diagram of a computer-implemented method 100 according to an embodiment of the disclosure. In a first step 110, a signal associated with a first user is received. The signal may be a biometric signal, i.e. a signal that represents a physical and / or biological characteristic or response of the human body. The biometric signal may comprise data that represents a characteristic associated with the first users eyes and / or direction of vision. For example, the biometric signal may comprise or consist of data that represents the first users eye-movement dynamics and / or pupil dilation and / or the direction in which the user is looking. Thus, in some embodiments the first user may be associated (e.g. wearing) a sensor (e.g. a camera, e.g. an inertial sensor) configured to detect characteristics of the eyes and / or movement of the head. For example, the first user may be associated with a wearable device (e.g. headset, e.g. glasses) which comprises the sensor (e.g. a camera, e.g. an inertial sensor). For example, the first user may be holding a gaming device (e.g. controller) comprising a sensor (e.g. a camera). In some embodiments the first user may be wearing an AR / VR headset that comprises a screen configured to display the augmented or virtual reality environment to the first user and a camera configured to detect characteristics associated with the first users eye movement. In some embodiments, the sensor (e.g. camera) may transmit data to a processor that is configured to perform eye-tracking analysis and / or measure the extent of dilation of the first users pupils. The biometric signal may comprise data that represents a characteristic associated with the first users heart rate. For example, the biometric signal may comprise or consist of data that represents the heart rate, heart rhythm and / or electrical activity. Thus, in some embodiments the first user may be associated (e.g. wearing) a sensor (e.g. a electrocardiogram electrode, e.g. a heart rate monitor) configured to detect characteristics of the heart. For example, the first user may be holding a gaming device (e.g. controller) comprising a sensor (e.g. a camera). For example, the first user may be associated with (e.g. wearing) a device (e.g. a smartwatch, e.g. a heart rate monitor, e.g. a body suit) which comprises a sensor for measuring heart rate characterises. The biometric signal may comprise data that represents a characteristic associated with the first users electrodermal activity. Thus, in some embodiments the first user may be associated (e.g. wearing) a sensor (e.g. a electrodermal sensor or electrode, e.g. a thermometer) configured to detect characteristics of the first users skin. For example, the first user may be associated with (e.g. wearing) a device (e.g. a glove, watch or body suit) which comprises a sensor for measuring electrodermal activity. For example, the first user may be holding a gaming device (e.g. controller) comprising a sensor (e.g. a camera). The biometric signal may comprise data that represents a characteristic associated with the first users facial expressions. For example, the biometric signal may comprise or consist of data that represents the movement dynamics or expression of a users face. Thus, in some embodiments the first user may be associated (e.g. wearing) a sensor (e.g. a camera, a RGB sensor, a depth sensor, an EEG sensor, a thermal sensor and / or an inertial sensor) configured to detect characteristics of the eyes. For example, the first user may be associated with a wearable device (e.g. a headset, watch, item of clothing, body suit) which comprises the sensor(s). For example, the first user may be holding a gaming device (e.g. controller) comprising a sensor (e.g. a camera). In some embodiments the first user may be wearing an AR / VR headset that comprises a screen configured to display the augmented or virtual reality environment to the first user and a camera configured to detect facial expressions. In some embodiments, the sensor(s) may transmit data to a processor that is configured to perform facial recognition and / or expression analysis. The biometric signal may comprise data that represents a characteristic associated with the first users facial expressions. Thus, in some embodiments the first user may be associated with (e.g. wearing) a sensor (e.g. a electrodermal sensor or electrode) configured to detect characteristics of the first users skin. For example, the first user may be associated with (e.g. wearing) a device (e.g. a glove, watch or body suit) which comprises a sensor for measuring electrodermal activity. For example, as shown in Figure 2, the (e.g. biometric) signal associated with the first user may represent the fact that the first user 200 has turned to look towards the second user 210, either in the augmented or virtual reality environment or the physical environment. For example, the user 200 shown may represent the physical first user, or may represent the virtual avatar associated with the first user 200. Similarly, the second user 210 shown in Figure 2 may be represent the physical user or the virtual avatar associated with the second user. For example, Figure 2 may represent the view of the first user 200 in the augmented or virtual reality environment such that we are seeing the second users 210 avatar and there is no second user 210 physically sat next to the first user 200 in the physical environment. Alternatively, the second user 210 may be sat next to the first user 200 in the physical environment and thus the signal represents the first user turning towards the second user in the physical environment (and not necessarily in the augmented or virtual reality environment). In some examples, the first user may be offered an opportunity to interact with a second user at a specific point in a game, for example when the first user has completed a game stage, or defeated an enemy. In a second step 120, the method 100 comprises determining that the first user 200 can interact with the second user 210. In some examples, the method 100 may further comprise receiving a signal (e.g. a biometric signal) from the second user 210 and using that signal to determine that the second user is available for interaction with the first user 200. As with the signal associated with the first user 200 (described above for method step 110), the signal may be a biometric signal, i.e. a signal that represents a physical and / or biological characteristic or response of the human body. As such, the embodiments and / or configurations described above in relation to the first user apply equally to the second user. To determine whether the second user 210 is available for interaction, the second user may have input a set of pre-determined preferences (e.g. contained within a user profiled stored within a computer readable memory unit) which define when the second user does (or more likely, does not) wish to engage or interact with other users. Optionally, these pre-set preferences may be based on biometric signals. For example, the pre-set preferences may include a threshold heart rate at which the second user should be considered to be engaged in a stressful and / or exciting activity and thus unwilling to be interrupted. For example, it can be anticipated that when the second user is at a critical point in a game (e.g. near the end of a time limit, fighting a boss, or doing something difficult), biometric characteristics associated with the second user may deviate from “normal” (e.g. unstressed) levels. For example, the second user’s heart rate may increase, their electrodermal activity may indicate increased sweating (associated with stress), their facial muscles may tense (indicating concentration) and / or their temperature may rise. The pre-set preferences may therefore represent a threshold range for these characteristics whereby, falling outside of (e.g. above or below) this range indicates the second user is not available for interaction. Optionally, the pre-set preferences may include game scenarios or characteristics. For example, the second user may pre-set that they do not wish to interact with other users when inside certain environments or rooms within the augmented or virtual reality environment. For example, the second user may pre-set that they do not wish to interact with another user if they are already engaged in an activity (e.g. combat, or another interaction). Optionally, the pre-set preferences may be periodically updated based on machine learning algorithms. For example, the thresholds may be changed if it is determined that the second user declines interactions frequently, even though the measurement is within a threshold range. It will thus be appreciated that the pre-set preferences may include a complex relationship of characteristics which indicate when the second user is available for interaction. For example, the second users heart rate may not be very high (e.g. it is within the threshold range indicating that the second user is available for interaction - and thus the preference is met) but the second user may also be within a game environment (e.g. a combat scenario) whereby the second user has pre-set they do not wish to interact with other users. The method thus may further include a means by which a ranking system may apply to determine an interaction score associated with the second user. Thus, it may be determined that a second user is not available for interaction when a threshold number of pre-set preferences are not met and / or the interaction score determined by the pre-set preferences comparison exceeds a threshold condition. Alternatively, there may be some pre-set preferences that result in a determination that the second user is not available regardless of the other pre-set preferences. For example, if the status of the second user is determined to not meet one of the preferences it may be determined that the second user is not available for interaction even if the other pre-set preferences are met (e.g. the second users heart rate is not elevated and thus falls within the threshold range). In a third step 130, when it is determined that the second user is available for interaction, the method comprises generating a user interface menu. As shown in Figure 3, the user interface menu 300 includes a plurality of interaction options represented pictorially. These interaction options include a high five gesture 310, gifting a present 320, sending a message 330 or starting a conversation 340. The user interface menu 300 is not limited to these interaction options shown and thus may comprise other interaction options such as a handshake, a wave or any other suitable and / or desirable interaction options. In a fourth 140 and fifth 150 step, the method 100 includes determining 140 the best display location, scale and / or position for the user interface menu 300 and then displaying the user interface menu 300 to the first user 210. To determine the best display location, scale and / or position, the is based on one or more inputs which may include a relative position of the second user 210 with respect to first user 200. For example, as shown in Figure 3, the first user 200 is next to the second user 210 such that it is determined that the best position for the user interface menu 300 is within the first users 200 eyeline to the second user 210 at an appropriate size for the user to see the user interface 300 without having to move their head. Alternatively, if the first user 200 was further away from the second user 210 the size of the user interface menu 300 shown in Figure 3 may be too small if it was positioned near to the second user 210. As such, the optimum display position may be selected such that the size of the user interface menu is increased and / or located closer to the first user 210 to allow the first user 210 to easily read the menu 300. Similarly, the one or more inputs may be contextual information relating to the first 200 and / or second 210 user. The contextual information may comprise the augmented or virtual reality environment, such as the type of game environment (e.g. of the first user or their associated avatar, e.g. of the second user or their associated avatar). In a fifth step 150, the method comprises initiating the interaction with the second user 210 or their avatar 210a. As shown in Figure 4, this may include displaying a prompt 400 associated with the interaction. The interaction displayed in Figure 4 is a high five and thus the associated prompt 400 is a target for the first user 200 and the second user 210 represented virtually by their avatar 210a to aim for. Other prompts may alternatively or additionally include an audio or visual count down timer. Upon completion of the interaction, the method may further include providing a feedback signal to the first and / or second user such as a haptic feedback, a sound or an image. Figure 5 illustrates an example system 500 arranged to implement the present disclosure. The system 500 comprises modules suitable for mapping and classifying the local environment and rendering an image to be displayed to a user. The system comprises sensor(s) 502, an AR / VR headset, a games console 506, a display unit 508 and a processing unit 510. The processing unit 510 comprises a SLAM module 512, an image generation and rendering module 514, a data store 516 (e.g. a memory), an object log data cache 518, an object recognition module 520 and an API module 522. Sensor(s) 502 may comprise one or more of the biometric sensors and / or sensors used to create the AR environment. For example, the sensors may comprise a camera, an electrode, an accelerometer, a gyroscope, a global positioning sensor, a light sensor and / or a depth sensor, each configured to capture data from the environment. In some examples, the sensor(s) 502 may be part of the VR / AR headset 504 worn by a user. The AR / VR headset 504 may also include the display unit 508 in some examples. The sensor(s) 502, AR / VR headset 504, games console 506 and display unit 508 are in communication with the processing unit 510. The processing unit 510 may be located in the cloud or be a part of the games console 506 or other computing device. Any suitable communications network can be used to transmit data between the sensor(s) 502, AR / VR headset 504, games console 506, display unit 508 and processing unit 510. The processing unit 510 comprises a SLAM module. The SU\M module 512 provides an estimate of the surrounding environment and comprises an internal front end which performs the method 100 described in Figure 1 and / or the feature extraction required to generate the AR environment (e.g. with data received from sensor(s) 502). In some embodiments, the SLAM module may be used to identify a location for the interaction menu or a visual representation of the interaction to be displayed. The SLAM module 512 further comprises an internal back end which uses the data output from the internal front end to determine the relationships between different frames, localization of a camera, geometrically reconstructing the surrounding environment as captured by the sensor(s) 502. An internal output module provides output data indicative of the surrounding environment, including data for the physical objects in the surrounding environment, the locations of those objects and landmark data corresponding to landmarks on those objects. The SLAM module 512 may feed directly into the object recognition module 520, which recognises planes including surfaces relating to, for example, tabletops, floors, walls etc which can be used to construct the augmented reality environment. Data concerning the objects which are identified in the environment may be stored at the data store 516 and / or object log data cache 518, for example their location or type of object. The image generation and rendering module 514 may create image content to be displayed at the AR / VR headset 504 and / or display unit 508 to be seen by a user. This can be based on data received from the sensor(s) 502 and / or one or more outputs from the SLAM module 512 and / or object recognition module 520 in response to detecting a plane or object in an environment. Image data may be created dynamically (or statically) at the image generation and rendering module 514 and provided as an image stream (or image) to a display output of the system. The image provides the virtual-part of the mixed-reality environment whilst the user sees the real-world part of the environment through one or more lenses of the AR / VR headset. The API module 522 may create or run one or more applications to be presented on a display device such as display unit 508. The system 500 is implemented using any suitable processing resource, which may be hardware and / or software implemented. Figure 6 shows a block diagram of one example implementation of a computing device 600 that can be used for implementing the steps indicated in Figure 1. The computing device is associated with executable instructions for causing the computing device to perform any one or more of the methodologies discussed herein. The computing device 600 may operate in the capacity of the data model or one or more computing resources for implementing the data model for carrying out the methods of the present disclosure. In alternative implementations, the computing device 600 may be connected (e.g., networked) to other machines in a Local Area Network (LAN), an intranet, an extranet, or the Internet. The computing device may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The computing device may be a personal computer (PC), a tablet computer, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single computing device is illustrated, the term “computing device” shall also be taken to include any collection of machines (e.g., computers) that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein. The example computing device 600 includes a processing device 602, a main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic randomaccess memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 606 (e.g., flash memory, static random-access memory (SRAM), etc.), and a secondary memory (e.g., a data storage device 718), which communicate with each other via a bus 630. Processing device 602 represents one or more general-purpose processors such as a microprocessor, central processing unit, or the like. More particularly, the processing device 602 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 602 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. Processing device 602 is configured to execute the processing logic (instructions 622) for performing the operations and steps discussed herein. The computing device 600 may further include a network interface device 608. The computing device 600 also may include a video display unit 610 (e.g., a light emitting diode (LED) display, a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 612 (e.g., a keyboard or touchscreen), a cursor control device 614 (e.g., a mouse or touchscreen), and an audio device 616 (e.g., a speaker). The data storage device 618 may include one or more machine-readable storage media (or more specifically one or more non-transitory computer-readable storage media) 628 on which is stored one or more sets of instructions 622 embodying any one or more of the methodologies or functions described herein. The instructions 622 may also reside, completely or at least partially, within the main memory 604 and / or within the processing device 602 during execution thereof by the computer system 600, the main memory 604 and the processing device 602 also constituting computer-readable storage media. The various methods described above may be implemented by a computer program. The computer program may include computer code arranged to instruct a computer to perform the functions of one or more of the various methods described above. The computer program and / or the code for performing such methods may be provided to an apparatus, such as a computer, on one or more computer readable media or, more generally, a computer program product. The computer readable media may be transitory or non-transitory. The one or more computer readable media could be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a propagation medium for data transmission, for example for downloading the code over the Internet. Alternatively, the one or more computer readable media could take the form of one or more physical computer readable media such as semiconductor or solid-state memory, magnetic tape, a removable computer diskette, a random-access memory (RAM), a read-only memory (ROM), a rigid magnetic disc, and an optical disk, such as a CD-ROM, CD-R / W or DVD. In an implementation, the modules, components and other features described herein can be implemented as discrete components or integrated in the functionality of hardware components such as ASICS, FPGAs, DSPs or similar devices. A “hardware component” is a tangible (e.g., non-transitory) physical component (e.g., a set of one or more processors) capable of performing certain operations and may be configured or arranged in a certain physical manner. A hardware component may include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component may be or include a special-purpose processor, such as a field programmable gate array (FPGA) or an ASIC. A hardware component may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. Accordingly, the phrase “hardware component” should be understood to encompass a tangible entity that may be physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. In addition, the modules and components can be implemented as firmware or functional circuitry within hardware devices. Further, the modules and components can be implemented in any combination of hardware devices and software components, or only in software (e.g., code stored or otherwise embodied in a machine-readable medium or in a transmission medium). Unless specifically stated otherwise, as apparent from the following discussion, it is appreciated that throughout the description, discussions utilising terms such as “providing”, “calculating”, “computing,” “identifying”, “detecting ”, “establishing” , “training”, “determining”, “storing”, “generating” / ’checking”, “obtaining” or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other implementations will be apparent to those of skill in the art upon reading and understanding the above description. Although the disclosure has been described with reference to specific example implementations, it will be recognised that the disclosure is not limited to the implementations described but can be practiced with modification and alteration within the scope of the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense. The scope of the disclosure should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. A computer-implemented method for allowing a first user to interact with a second user in an augmented or virtual reality environment, the method comprising:receiving a signal associated with the first user;determining that the first user can interact with the second user based on the signal associated with the first user;generating a user interface menu for display to the first user comprising one or more interaction options for interaction with the second user;determining an optimum display location, scale and / or position within the augmented or virtual reality environment for displaying the user interface menu to the first user based on one or more inputs;displaying the user interface menu to the first user in the optimum position for a selection of a desired interaction option by the first user; andinitiating the desired interaction option between the first user and the second user based on the selection of the first user from the user interface menu.

2. The computer-implemented method of claim 1, wherein the desired interaction option is initiated between an avatar associated with the first user and the avatar associated with the second user.

3. The computer-implemented method of claim 1 or claim 2, wherein the signal associated with the first user comprises one or more biometric signals.

4. The computer-implemented method of any preceding claim, wherein the method further comprises:receiving a signal associated with the second user; anddetermining that the avatar of the second user is available for interaction with the first user.

5. The computer-implemented method of claim 4, wherein the signal associated with the second user comprises one or more biometric signals.

6. The computer-implemented method of claim 3 or 5, wherein the one or more biometric signals are data representative of one or more of the first or second users:eye-movement dynamics, heart rate, facial expression, electrodermal activity and / or pupil dilation.

7. The computer-implemented method of any one of claims 3, 5 or 6, wherein the step of receiving the signal associated with the first user and / or receiving the signal associated with the second user comprises:receiving biometric data from one or more biometric sensors associated with the first and / or second user;wherein the one or more biometric sensors may comprise a heart rate monitor; an optical sensor, a thermometer, a camera, and / or or an electroencephalogram sensor.

8. The computer-implemented method of any one of claims 5-7, wherein the step of determining that the second user is available for interaction with the first user comprises:comparing each of the one or more biometric signals to a threshold range; andwhen it is determined that at least one of the biometric signals is outside of the threshold range, determining that the second user is not available for interaction; andwherein the step of displaying the user interface menu is based on a positive determination that all of the one or more biometric signals falls within their associated threshold range.

9. The computer-implemented method of any one of claims 4 to 8, wherein the signal associated with the second user comprises a status of the second user.

10. The computer-implemented method of claim 9, wherein the step of determining that the second user is available for interaction with the first user comprises:comparing the status to the second users pre-set preferences for interacting with users within the augmented or virtual reality environment; andwhen it is determined that at least one of the pre-set preferences are not met, determining that the second user is not available for interaction,wherein the step of displaying the user interface menu is based on a positive determination that the biometric data does not exceed one or more of the biometric data thresholds.

11. The computer-implemented method of claims 8 or 10, wherein, when it is determined that the second user is not available for interaction, the method further comprises:monitoring the signal associated with the second user until it is determined that the biometric signal is within the threshold range and / or the pre-set preferences are met; and / oralerting the first user that the second user is unavailable for interaction.

12. The computer-implemented method of claim 11, wherein the step of alerting the first user further comprises presenting the user with one or more of the following options:to wait for the second user to be available;to be notified when the second user is available; orto abandon the interaction with the second user.

13. The computer-implemented method of any preceding claim, wherein the interaction options comprise one or more of: a gesture, presenting a gift, initiating a conversation and / or sending a message.

14. The computer-implemented method of claim 13, wherein the gesture includes one or more of: a wave, a high-five, and / or a hand-shake.

15. The computer-implemented method of any preceding claim, wherein one of the one or more inputs is a relative position of the second user with respect to first user; wherein the relative position comprises:a relative distance between the first user and the second user; and / ora relative angular displacement between the first user and the second user in the augmented or virtual reality environment.

16. The computer-implemented method of any preceding claim, wherein one of the one or more inputs is a relative size of the second user with respect to the first user.

17. The computer-implemented method of any preceding claim, wherein one of the one or more inputs is contextual information relating to the second user and / or the first user, wherein the contextual information comprises one or more of:(i) the augmented or virtual reality environment;(ii) a location of the first user in the augmented reality environment; and / or(iii) a location of the second user in the augmented reality environment18. The computer-implemented method of any preceding claim, wherein the user interface menu is displayed in a location that, from the perspective of the first user, is proximate to the second user in the augmented reality environment.

19. The computer-implemented method of any preceding claim, wherein the step of initiating the desired interaction option comprises:providing the first user and / or the second user with a prompt associated with the interaction; and / orproviding a feedback signal associated with the interaction.

20. The computer-implemented method of claim 19, wherein the prompt may be one of an audio or visual count-down timer indicating the timing of the interaction and / or a visual target indicating the location of the intended interaction.

21. The computer-implemented method of claim 19 or 20, wherein the feedback signal may be one or more of a haptic signal, a visual signal, or an audio signal.

22. A data processing system comprising means for carrying out the method of any one of claims 1 to 21.

23. A computer program product comprising instructions which, when the program is executed by a computer, causes the computer to carry out the method of any one of claims 1 to 21.

Images