An availability indicator in an extended reality device

Abstract

There is provided a method performed by an extended reality (XR) device capable of rendering an XR scene. The method comprises obtaining information related to a cognitive state of a user of the XR device and based at least on the obtained information, generating an availability indicator indicating an availability of the user of the XR device. The method further comprises displaying the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

Description

AN AVAILABILITY INDICATOR IN AN EXTENDED REALITY DEVICETECHNICAL FIELD

[0001] This disclosure relates to an availability indicator in an extended reality (XR) device.BACKGROUND

[0002] Extended reality (XR) devices are devices for rendering XR scenes, thereby allowing users of the XR devices to experience the XR scenes. Examples of XR devices include virtual reality (VR) devices, augmented reality (AR) devices, and mixed reality (MR) devices.

[0003] XR devices can be provided in different forms. For example, an XR device may be provided in the form of eyeglasses, a goggle, or a helmet. Like normal prescription eyeglasses, a ski goggle, or a motorcycle helmet, an XR device may be worn over the eyes of the user of the XR device.

[0004] In case an XR device is an AR device, the XR device may include a transparent surface via which the user of the XR device can see a real-world environment surrounding the user. The AR device may also include parts for generating and / or overlaying computergenerated content over the view of the real-world environment seen by the user at his / her point of view, in order to enhance the user’s perception of the real-world environment. The computergenerated content may include but are not limited to text, images, videos, animations, and / or three-dimensional (3D) holograms.

[0005] In case an XR device is a MR device, the XR device may be capable of mixing a virtual reality view and a view of a real-world environment. Instead of allowing a user of the XR device to view the real-world environment via a transparent surface of the XR device, the XR device may utilize camera(s) to capture a view of the real-world environment, which may overlay the virtual reality view The MR device may also include an opaque surface preventing the user of the MR device from directly seeing the real-world environment surrounding the MR device.

[0006] Like the MR device, an VR device may include an opaque surface preventing the user of the VR device from directly seeing a real-world environment surrounding the user of the VR device, and it may include parts for generating and / or displaying computer-generated contentcorresponding to a VR environment.SUMMARY

[0007] Certain challenges presently exist.

[0008] A user of a mobile phone can convey to people in the vicinity of the user whether he or she is paying attention to the mobile phone. This is possible because the eyes of the user are not obscured, and thus the people in the vicinity of the user can tell whether the user is paying attention to them or to the mobile phone by observing the user’s line of sight.

[0009] On the other hand, a user wearing and interacting with an XR device (e.g., which has the shape of eyeglasses, a goggle, or a helmet) may have difficulties in conveying to people in the vicinity of the user where his / her current attention is because the eyes of the user can be obscured by the XR device. More specifically, the people in the vicinity of the user may not be able to tell whether the user of the XR device is paying attention to them or to the XR device.

[0010] Also, even in case where the user of the XR device is paying attention to the XR device, there may be a scenario where the user of the XR device doesn’t mind being disturbed. For example, while the user of the XR device is casually browsing internet, the user may be available for talking to another person. On the other hand, while the user of the XR device is in a video conference, the user may not be available for talking to another person.

[0011] Therefore, there is a need for an XR device capable of determining and conveying the availability of the user of the XR device to people in the vicinity of the user while the user is using the XR device.

[0012] Accordingly, in one aspect of some embodiments of this disclosure, there is provided a method performed by an extended reality, XR, device capable of rendering an XR scene. The method comprises obtaining information related to a cognitive state of a user of the XR device; based at least on the obtained information, generating an availability indicator indicating an availability of the user of the XR device; and displaying the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

[0013] In another aspect, there is provided a computer program comprising instructions which when executed by processing circuitry cause the processing circuitry to perform the methodof any one of the embodiments described above.

[0014] In a different aspect, there is provided a carrier containing the computer program of the above embodiment, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

[0015] In a different aspect, there is provided an extended reality, XR, device capable of rendering an XR scene. The XR device is configured to obtain information related to a cognitive state of a user of the XR device; based at least on the obtained information, generate an availability indicator indicating an availability of the user of the XR device; and display the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

[0016] In a different aspect, there is provided an apparatus comprising: a memory; and processing circuitry coupled to the memory, wherein the apparatus is configured to perform the method of any one of the above embodiments.

[0017] As discussed above, some embodiments of this disclosure allow an XR device to determine and convey the availability of the user of the XR device to people in the vicinity of the user while the user is using the XR device.BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0019] FIG. 1 shows an exemplary scenario where some embodiments of this disclosure can be applied.

[0020] FIG. 2A is a block diagram of an apparatus according to some embodiments.

[0021] FIG. 2B shows a function of an apparatus according to some embodiments.

[0022] FIG. 3 A shows an exemplary view of a real- world environment.

[0023] FIG. 3B shows an exemplary extended reality scene.

[0024] FIG. 4 shows a process according to some embodiments.

[0025] FIG. 5A shows an order of performing the process shown in FIG. 4.

[0026] FIG. 5B shows an order of performing the process shown in FIG. 4.

[0027] FIG. 6 shows a process according to some embodiments.

[0028] FIG. 7 shows a process according to some embodiments.

[0029] FIG. 8 shows an apparatus according to some embodiments.DETAILED DESCRIPTION

[0030] FIG. 1 shows an exemplary scenario 100 where some embodiments of this disclosure can be applied. In the scenario 100, a user 102 is wearing and using an XR device 104. Even though, in FIG. 1, the XR device 104 has the form of a goggle, in some embodiments, the XR device 104 may be in a different form. F or example, the XR device 104 may have the form of an eyeglasses or any head mounted device (HMD).

[0031] FIG. 2A is a block diagram of the XR device 104 according to some embodiments. The block diagram shows some components included in the XR device 104. As shown in FIG. 2A, the XR device 104 may include a first display area 202, a second display area 204, and a third display area 206.

[0032] The first and second display areas 202 and 204 are inward display areas, and the third display area 206 is an outward display area. As shown in FIG. 2B, the inward display areas are directed towards the eyes of the user 102, and are configured to display content to the user 102. As further shown in FIG. 2B, the outward display area is directed towards outside of the XR device 104, and is configured to display content towards the outside of the XR device 104. The first display area 202 may be configured to display content for a left eye of the user 102 while the second display area 204 may be configured to display content for a right eye of the user 102.

[0033] Each of the first, second, and third display areas 202-206 may be formed of a separate display panel as shown in FIG. 2B. Alternatively, two or more of the first, second, and third display areas 202-206 may be formed of a single display panel. In case the third display area 206 and one or more of the first and second display areas 202 and 204 are formed of a single display panel, the display panel may be a folded panel. The first, second, and / or third display areas 202-206 may be provided by any electronic display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a microLED display, etc.). In this disclosure,the electronic displays 202 and 204 may also be referred as adaptive Near Eye Display (NED) layer as the displays 202 and 204 are worn on the face of the user 102 and are capable of producing a virtual image.

[0034] As shown in FIG. 2A, the XR device 104 may also include one or more inward cameras 208 capable of capturing a view of the face of the user 102 and / or a view of the eyes of the user 102, and one or more outward cameras 210 capable of capturing a view of a real-world environment surrounding the XR device 104. The captured view of the real-world environment can be displayed to the user 102 via the inward display areas 202 and 204.

[0035] FIG. 3 A shows an example of a view 300 of a real-world environment, and FIG. 3B shows a view 302 of the real-world environment reproduced by the inward display areas 202 and 204. As shown in FIG. 3B, the inward display areas 202 and 204 may also be configured to display an overlay 304 on top of the captured view 302 of the real-world environment 300. Even though, in FIG. 3B, the view 300 of the real- world environment is reproduced by the inward display areas 202 and 204, as mentioned above, in other embodiments, the user 102 can see the view 300 of the real- world environment via one or more transparent surfaces.

[0036] Referring back to FIG. 2A, the XR device 104 may also include one or more sensors212 for determining a distance between the XR device 104 and an object outside the XR device 104, and one or more sensors 214 for detecting a movement of the XR device 104. One example of the sensors 212 is a LiDAR sensor, and examples of the sensors 214 are a gyroscope and an accelerometer.

[0037] The XR device 104 may also include one or more processors 216 (e.g., a central processing unit, a graphic processing unit, etc.) and one or more storage mediums 218 (e.g., memories) capable of storing codes executable by at least one of the processors 216. Using the codes, the XR device 104 is capable of running various applications such as a video conferencing app, a video chatting app, a video streaming app, a photo viewer, an internet browser, etc.

[0038] Referring back to FIG. 1, in the scenario 100, while the user 102 is using the XR device 104, a person 106 is approaching the user 102 and trying to initiate a conversation with the user 102. But since the eyes of the user 102 are obscured by the XR device 104, the person 106 does not know whether the user 102 is paying attention to the content displayed on the inward displays 202 and 204 of the XR device 104 or whether the user 102 is looking at the person 106. Even if the eyes of the user 102 are not obscured, in case the content displayed onthe inward displays 202 and 204 are aligned with the external view, the person 106 wouldn’t know.

[0039] In order to solve the above described problem, in some embodiments, there is provided a method 400 shown in FIG. 4. The method 400 may be performed by the XR device 104. The method 400 may comprise one or more of steps s402, s404, s406, and / or s408. The order of performing the steps s402-s408 is explained with respect to FIGS. 5A and 5B.

[0040] The step s402 comprises determining whether the user 102 of the XR device 104 is passively or actively engaged with the XR device 104. For example, in the step s402, the XR device 104 may determine whether the user 102 is passively or actively engaged with the content displayed on the inward display areas 202 and / or 204.

[0041] In some embodiments, the step s402 may comprise sub-steps s402a, s402b, and s402c.

[0042] The step s402a comprises obtaining information related to a cognitive state of the user 102 of the XR device 104. The XR device 104 may obtain the information related to the cognitive state of the user 102 by monitoring device data of the XR device 104, app data of the XR device 104, and / or context information (e.g., a location, time of day, etc.). More specifically, in some embodiments, the XR device 104 may obtain the information related to the cognitive state of the user 102 by determining one or more of the followings:

[0043] The step s402b comprises analyzing the information related to the cognitive state of the user 102, which is obtained in the step s402a, and, using the information, determine a value of a cognitive state indicator indicating the cognitive state of the user 102. There are various ways of determining the value of the cognitive state indicator. In some embodiments, a numerical value may be given to each of some or all of the above listed information 1 through 14 and the value of the cognitive state indicator may be calculated as a weighted sum of the numerical values. In other embodiments, each of some or all of the above listed information 1 through 14 may be provided to a machine learning (ML) model as inputs, and the ML model may be configured to output the value of the cognitive state indicator.

[0044] The step s402c comprises comparing the value of the cognitive state indicator to a threshold value, and determine whether the user 102 of the XR device 104 is passively or actively engaged with the XR device 104.

[0045] In one example, in case the value of the cognitive state indicator is greater than or equal to the threshold value, it may be determined that the user 102 of the XR device 104 is actively engaged with the XR device 104 while in case the value of the cognitive state indicator is less than the threshold value, it may be determined that the user 102 is passively engaged with the XR device 104.

[0046] In another example, in case the value of the cognitive state indicator is less than the threshold value, it may be determined that the user 102 of the XR device 104 is passively engaged with the XR device 104 while in case the value of the cognitive state indicator is greaterthan or equal to the threshold value, it may be determined that the user 102 is actively engaged with the XR device 104.

[0047] In some embodiments, instead of performing the step s402c — i.e., comparing the value of the cognitive state indicator to the threshold value — , and determining whether the user 102 is actively or passively engaged with the XR device 104, there may be provided a mapping table or a mapping function for mapping a specific value or a specific range of the cognitive state indicator to a certain state of whether the user 102 is actively or passively engaged with the XR device 104. The table below illustrates the concept of this mapping.

[0048] In some embodiments, the above mapping or the threshold value used in the step s402c may be dynamically adjusted. For example, the above mapping or the threshold value used in the step s402c may be adjusted such that, between 9am and 12pm, the XR device 104 is more inclined to determine the user’s cognitive state as being active engagement while, between 9 pm and 6 am, the XR device 104 is more inclined to determine the user’s cognitive state as being passive engagement.

[0049] The step s404 comprises determining whether one or more persons are nearby the XR device 104. In some embodiments, the step s404 may comprise one or more of sub-steps s404a, s404b, s404c, s404d, and / or s404e.

[0050] The step s404a comprises activating the camera(s) 210 of the XR device 104 and capturing a view of a part of the real-world environment surrounding the XR device 104 (e.g., capturing the front facing view of the real-world environment).

[0051] The step s404b comprises analyzing the captured view and the step s404c comprises determining whether any person is in the captured view based on the analysis. By determining whether any person is in the captured view or not, the XR device 104 may determine whether any person is nearby the XR device 104.

[0052] In some embodiments, after performing the step s404c, additional steps s404d and s404e may be performed to determine whether any person is nearby the XR device 104. The step s404d comprises determining a distance between the person that is determined to be present in the captured view in step s404c and the step s404e comprises comparing the distance to athreshold distance value. In these embodiments, a person is determined to be nearby the XR device 104 only if the distance obtained in the step s404d is less than the threshold distance value. In some embodiments, instead of or in addition to determining whether any person is nearby the XR device 104, the XR device 104 may determine whether any person is approaching the XR device 104, and the next step (e.g., s406 in FIG. 5A and / or s402 in FIG. 5B) may be performed only if it is determined that any person is approaching the XR device 104. In further embodiments, instead of or in addition to determining whether any person is nearby the XR device 104 and / or determining whether any person is approaching the XR device 104, the XR device 104 may determine if the person’s face in the captured view is directed towards the user 102, and the next step (e.g., s406 in FIG. 5 A and / or s402 in FIG. 5B) may be performed only if it is determined that the person’s face is directed towards the user 102.

[0053] The step s406 comprises generating and displaying an availability indicator that indicates an availability of the user 102 of the XR device 104. The indicator may be provided in any one or more of the various forms such as text, image, brightness of light, pattern, etc. For example, the XR device 104 may display text - “I am in a video call,” “Busy,” “Wait 15 sec,” “I will call you,” etc.

[0054] FIG. 5 A shows an order 500 of performing the steps included in the process 400 according to some embodiments. In the order 500, the step s402 is performed first. In other words, the XR device 104 may first determine whether the user 102 of the XR device 104 is passively or actively engaged with the XR device 104. The step s402 may be triggered by the XR device 104 detecting an occurrence of a certain triggering event. Examples of the triggering event include elapse of a certain time interval, executing any app, executing a certain app, displaying certain content in the inward display areas, displaying any content in the inward display areas, detecting any user input, detecting any particular user input, etc.

[0055] In case it is determined that the user 102 of the XR device 104 is actively engaged with the XR device 104, the process 400 may proceed to the step s404 which comprises determining whether any one or more persons is nearby the user 102 (and / or whether any one or more person is approaching the user 102). In case it is determined that any person is nearby the user 102 (and / or any person is approaching the XR device 104), the process 400 may proceed to step s406 which comprises displaying, on the outward display area 206, the availability indicator indicating that the user 102 is not available.

[0056] On the other hand, in case it is determined, in the step s404, that no one is nearby the XR device 104, the process 400 may return to the step s402, and the step s402 may be performed again — re-determining whether the user 102 of the XR device 104 is passively or actively engaged with the XR device 104. Additionally or alternatively, in case it is determined, in the step s404, that no one is nearby the XR device 104, the step s404 may be performed again after a certain time period.

[0057] Referring back to the step s402, in case it is determined that the user 102 of the XR device 102 is passively engaged with the XR device 104, the process 400 may not proceed to the step s404. Instead, the XR device 104 may re-perform the step s402 after a certain time period or after re-detecting the occurrence of the triggering event. Alternatively, the process 400 may still proceed to the step s404, and in case it is determined that one or more person is nearby the XR device 104, in the step s406, the XR device 104 may display, on the outward display area 206, the availability indicator indicating that the user 102 is currently available.

[0058] In some embodiments, after displaying the availability indicator indicating that the user 102 is not available, the XR device 104 may re-perform the step s404 and determine that no one is nearby the XR device 104. In such scenario, the process 400 may proceed to the step s408, and the XR device 104 may clear the availability indicator displayed on the outward display area 206.

[0059] FIG. 5B shows a different order 550 of performing the steps included in the process 400 according to some embodiments. In the order 550, the step s404 is performed first. In other words, the XR device 104 may first determine whether any person is nearby the XR device 104. The XR device 104 may be triggered to perform the step s404 upon detecting an occurrence of a triggering event. An example of the triggering event includes elapse of a certain time interval.

[0060] In case it is determined that no one is nearby the XR device 104, the XR device 104 may re-perform the step s404 after re-detecting the occurrence of the triggering event. On the other hand, in case it is determined that one or more persons is nearby the XR device 104, the process may proceed to the step s402, and the XR device may determine whether the user 102 of the XR device 104 is passively or actively engaged with the XR device 104. After performing the step s402, the process 400 may proceed to the step s406.

[0061] In case it is determined that the user 102 of the XR device 104 is actively engaged with the XR device 104, the XR device 104 may display, on the outward display area 206, theavailability indicator indicating that the user 102 of the XR device 104 is not currently available. On the other hand, in case it is determined that the user 102 of the XR device 104 is passively engaged with the XR device 104, the XR device 104 may display, on the outward display area 206, the availability indicator indicating that the user 102 of the XR device 104 is currently available.

[0062] After displaying the indicator, the process 400 may return to the step s404 and redetermine whether any one is nearby the XR device 104. In case no one is nearby, the process 400 may proceed to step s408 and the XR device 104 may clear the displayed indicator.

[0063] In some embodiments, in the process 400 shown in FIGS. 5 A and 5B, instead of performing the step s404, a combination of the step s404 and steps s602 and / or s604 shown in FIG. 6 may be performed.

[0064] The step s602 comprises determining whether the person that is determined to be nearby the XR device 104 (and / or determined to be approaching the XR device 104) is facing towards the XR device 104. More specifically, in some embodiments, the outward cameras 210 of the XR device 104 may be configured to capture a view of the person, and the XR device 104 may be configured to analyze the captured view of the person to determine whether the person is facing towards the XR device 104. Additionally or alternatively, the XR device 104 may be configured to analyze the captured view of the person to determine the gaze direction of the person.

[0065] In those embodiments, the process 400 may proceed to the step s406 in the order 500 or to the step s402 in the order 550 only if it is determined that the person is facing towards the XR device 104 (e.g., for at least a required time period) and / or if is determined that the person is looking at the user 102 of the XR device 104 (e.g., for at least a required time period).

[0066] The step s604 comprises determining the identity of the person determined to be nearby the XR device 104 (and / or determined to be approaching the XR device 104). For example, the XR device 104 may perform a facial recognition and / or a voice recognition of the person using the outward cameras 210 and / or a microphone (not shown). In these embodiments, the process 400 may proceed to the step s406 in the order 500 or to the step s402 in the order 550 only if it is determined that the person has a certain identity that satisfies a condition (e.g., registered in a database (e.g., an address book, a list of social contact, etc.) stored in the XR device 104).

[0067] There may be a scenario where the person who is nearby the XR device 104 and / oris approaching the XR device 104 is also using an XR device (hereinafter, “the secondary XR device”). In this scenario, according to some embodiments, the two XR devices may be configured to communicate with each other.

[0068] For example, in case the XR device 104 determines that the user 102 of the XR device 104 is actively engaged with the XR device 104 while the secondary XR device determines that the user of the secondary XR device is passively engaged with the secondary XR device. In such case, the secondary XR device may provide to the XR device 104 an indication (e.g., by displaying a message towards to the XR device 104, or by transmitting a wireless signal containing the indication to the XR device 104) indicating that the user of the secondary XR device is passively engaged with the secondary XR device, and, in response to receiving such indication, the XR device 104 may be configured to display the availability indicator indicating that the user 102 of the XR device 104 is not currently available.

[0069] On the other hand, in case both the XR device 104 and the secondary XR device determine that their users are actively engaged with the XR devices, each of the XR devices may provide to other XR device an indication indicating that its user is not currently available. In response to receiving such indication, the other XR device may notify its user that the other user is not currently available.

[0070] FIG. 7 shows a process 700 performed by an extended reality, XR, device (104) capable of rendering an XR scene, according to some embodiments. The process 700 may begin with step s702. Step s702 comprises obtaining information related to a cognitive state of a user of the XR device. Step s704 comprises, based at least on the obtained information, generating an availability indicator indicating an availability of the user of the XR device. Step s706 comprises displaying the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

[0071] In some embodiments, the process 700 comprises analyzing the information related to the cognitive state of the user of the XR device; and based on the analysis, determining whether the user is actively or passively engaged with content displayed on the XR device, wherein the availability indicator is displayed based on the determination.

[0072] In some embodiments, obtaining the information related to the cognitive state of the user of the XR device comprises determining one or more of: whether any application is running on the XR device; whether any particular application is running on the XR device; whether anycontent is displayed on one or more inward displays included in the XR device; an amount of user inputs involved in content displayed on one or more inward displays included in the XR device; whether any particular content is displayed on said one or more inward displays; an amount of content displayed on said one or more inward displays; an amount of data corresponding to content displayed on said one or more inward displays; a gazing direction of the user; a degree and / or a frequency of changing the gazing direction of the user; whether the user is moving; and / or an amount of the user’s movement; a gazing direction of the user of the XR device; a facial expression of the user of the XR device; actuation of user inputs associated with one or more processes running on the XR device; and / or a location of the XR device and / or a type of a space where the XR device is in.

[0073] In some embodiments, the process 700 comprises based on the obtained information related to the cognitive state of the user of the XR device, calculating a value of a cognitive state indicator indicating the cognitive state of the user, wherein the availability indicator is generated based on the value of the cognitive state indicator.

[0074] In some embodiments, the process 700 comprises comparing the calculated value of the cognitive state indicator to a threshold value, wherein the availability indicator is generated based on the comparison of the value of the cognitive state indicator and the threshold value.

[0075] In some embodiments, the process 700 comprises obtaining a mapping table or a mapping function that maps each of a plurality of values of the cognitive state indicator to each of a plurality of cognitive states; and using the mapping table or the mapping function, determining a cognitive state mapped to the calculated value of the cognitive state indicator, wherein the availability indicator is generated based on the mapped cognitive state.

[0076] In some embodiments, the process 700 comprises using one or more cameras included in the XR device, capturing a view of a part of an environment in which the XR device is located; analyzing the captured view; and determining whether any person is in the captured view, wherein the availability indicator is generated based on the determination of whether any person is in the captured view.

[0077] In some embodiments, the process 700 comprises determining that a person is in the captured view; and determining one or more of (i) whether the person is looking at the user of the XR device; (ii) whether the person is talking to another person; (iii) whether the person is approaching the XR device; and / or (iv) whether the person’s face is directed towards the user ofthe XR device, wherein the availability indicator is generated based on the above determination.

[0078] In some embodiments, the capturing of the view of the part of the environment is triggered as a result of determining that the calculated value of the cognitive state indicator satisfies a condition.

[0079] In some embodiments, the process 700 comprises using said one or more cameras and / or one or more distance measurement sensors included in the XR device, determining a distance between the XR device and a person in the view of the part of the environment; comparing the determined distance to a distance threshold value; the availability indicator is generated based on the determination that the determined distance is less than or equal to the distance threshold value.

[0080] In some embodiments, the process 700 comprises using said one or more cameras included in the XR device, recapturing a view of a part of the environment; analyzing the recaptured view; determining whether any person is in the recaptured view; and removing the availability indicator from the outward display or changing the availability indicator based on the determination that no person is in the recaptured view.

[0081] In some embodiments, the process 700 comprises determining whether any person is in the captured view comprises determining that a person is in the captured view, the process 700 comprises determining an orientation and / or a gazing direction of the person in the captured view, and the availability indicator is generated based on the determined orientation and / or the gazing direction of the person in the captured view.

[0082] In some embodiments, determining whether any person is in the captured view comprises determining that a person is in the captured view, the process 700 comprises performing a facial recognition and / or a voice recognition of the person in the captured view, and the availability indicator is generated based on a result of performing the facial recognition and / or the voice recognition.

[0083] In some embodiments, the process 700 comprises using one or more inward displays included in the XR device, indicating to the user of the XR device that the availability indicator is being displayed on the outward display.

[0084] In some embodiments, the process 700 comprises receiving from another XR device information related to a cognitive state of a user of said another XR device, wherein the availability indicator is generated based on the received information related to the cognitive state of the userof said another XR device.

[0085] FIG. 8 is a block diagram of the XR device 104, according to some embodiments. As shown in FIG. 8, the XR device 104 may comprise: processing circuitry (PC) 802, which may include one or more processors (P) 855 (e.g., one or more general purpose microprocessors and / or one or more other processors, such as an application specific integrated circuit (ASIC), field- programmable gate arrays (FPGAs), and the like); communication circuitry 848, which is coupled to an antenna arrangement 849 comprising one or more antennas and which comprises a transmitter (Tx) 845 and a receiver (Rx) 847 for enabling the XR device 104 to transmit data and receive data (e.g., wirelessly transmit / receive data); and a storage unit (a.k.a., “data storage system”) 808, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 802 includes a programmable processor, a computer readable storage medium (CRSM) 842 may be provided. CRSM 842 may store a computer program (CP) 843 comprising computer readable instructions (CRI) 844. CRSM 842 may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI 844 of computer program 843 is configured such that when executed by PC 802, the CRI causes the XR device 104 to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, the XR device 104 may be configured to perform steps described herein without the need for code. That is, for example, PC 802 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software.

[0086] While various embodiments are described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of this disclosure should not be limited by any of the above described exemplary embodiments. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0087] As used herein transmitting a message “to” or “toward” an intended recipient encompasses transmitting the message directly to the intended recipient or transmitting the message indirectly to the intended recipient (i.e., one or more other nodes are used to relay the message from the source node to the intended recipient). Likewise, as used herein receiving amessage “from” a sender encompasses receiving the message directly from the sender or indirectly from the sender (i.e., one or more nodes are used to relay the message from the sender to the receiving node). Further, as used herein “a” means “at least one” or “one or more.”

[0088] Additionally, while the processes described above and illustrated in the drawings are shown as a sequence of steps, this was done solely for the sake of illustration. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, and some steps may be performed in parallel.

Claims

CLAIMS1. A method (700) performed by an extended reality, XR, device (104) capable of rendering an XR scene, the method comprising: obtaining (s702) information related to a cognitive state of a user (102) of the XR device; based at least on the obtained information, generating (s706) an availability indicator indicating an availability of the user of the XR device; and displaying (s706) the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

2. The method of claim 1, comprising: analyzing the information related to the cognitive state of the user of the XR device; and based on the analysis, determining whether the user is actively or passively engaged with content displayed on the XR device, wherein the availability indicator is displayed based on the determination.

3. The method of claim 1 or 2, wherein obtaining the information related to the cognitive state of the user of the XR device comprises determining one or more of: whether any application is running on the XR device; whether any particular application is running on the XR device; whether any content is displayed on one or more inward displays included in the XR device; an amount of user inputs involved in content displayed on one or more inward displays included in the XR device; whether any particular content is displayed on said one or more inward displays; an amount of content displayed on said one or more inward displays; an amount of data corresponding to content displayed on said one or more inward displays; one or more of a degree and a frequency of changing the gazing direction of the user; whether the user is moving;an amount of the user’s movement; a gazing direction of the user of the XR device; a facial expression of the user of the XR device; actuation of user inputs associated with one or more processes running on the XR device; or one or more of a location of the XR device and a type of a space where the XR device is in.

4. The method of any of claims 1-3, comprising: based on the obtained information related to the cognitive state of the user of the XR device, calculating a value of a cognitive state indicator indicating the cognitive state of the user, wherein the availability indicator is generated based on the value of the cognitive state indicator.

5. The method of claim 4, comprising: comparing the calculated value of the cognitive state indicator to a threshold value, wherein the availability indicator is generated based on the comparison of the value of the cognitive state indicator and the threshold value.

6. The method of claim 4, comprising: obtaining a mapping table or a mapping function that maps each of a plurality of values of the cognitive state indicator to each of a plurality of cognitive states; and using the mapping table or the mapping function, determining a cognitive state mapped to the calculated value of the cognitive state indicator, wherein the availability indicator is generated based on the mapped cognitive state.

7. The method of any one of claims 1-6, comprising: using one or more cameras included in the XR device, capturing a view of a part of an environment in which the XR device is located; analyzing the captured view; anddetermining whether any person is in the captured view, wherein the availability indicator is generated based on the determination of whether any person is in the captured view.

8. The method of claim 7, comprising: determining that a person is in the captured view; and determining one or more of (i) whether the person is looking at the user of the XR device; (ii) whether the person is talking to another person; (iii) whether the person is approaching the XR device; and (iv) whether the person’s face is directed towards the user of the XR device, wherein the availability indicator is generated based on the above determination.

9. The method of claim 7 when claim 7 depends on any one of claims 4-6, wherein the capturing of the view of the part of the environment is triggered as a result of determining that the calculated value of the cognitive state indicator satisfies a condition.

10. The method of any one of claims 7-9, comprising: using one or more of said one or more cameras and one or more distance measurement sensors included in the XR device, determining a distance between the XR device and a person in the view of the part of the environment; and comparing the determined distance to a distance threshold value, wherein the availability indicator is generated based on the determination that the determined distance is less than or equal to the distance threshold value.

11. The method of any one of claims 7-10, comprising: using said one or more cameras included in the XR device, recapturing a view of a part of the environment; analyzing the recaptured view; determining whether any person is in the recaptured view; and removing the availability indicator from the outward display or changing the availability indicator based on the determination that no person is in the recaptured view.

12. The method of any one of claims 7-11, wherein determining whether any person is in the captured view comprises determining that a person is in the captured view, the method comprises determining one or more of an orientation and a gazing direction of the person in the captured view, and the availability indicator is generated based on one or more of the determined orientation and the gazing direction of the person in the captured view.

13. The method of any one of claims 7-12, wherein determining whether any person is in the captured view comprises determining that a person is in the captured view, the method comprises performing one or more of a facial recognition and a voice recognition of the person in the captured view, and the availability indicator is generated based on a result of performing one or more of the facial recognition and the voice recognition.

14. The method of any one of claims 1-13, comprising: using one or more inward displays included in the XR device, indicating to the user of the XR device that the availability indicator is being displayed on the outward display.

15. The method of any one of claims 1-14, comprising: receiving from another XR device information related to a cognitive state of a user of said another XR device, wherein the availability indicator is generated based on the received information related to the cognitive state of the user of said another XR device.

16. A computer program (843) comprising instructions (844) which when executed by processing circuitry (802) cause the processing circuitry to perform the method of any one of claims 1-15.

17. A carrier containing the computer program of claim 16, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

18. An extended reality, XR, device (104) capable of rendering an XR scene, the XR device being configured to: obtain (s702) information related to a cognitive state of a user (102) of the XR device; based at least on the obtained information, generate (s706) an availability indicator indicating an availability of the user of the XR device; and display (s706) the availability indicator on an outward display included in the XR device, wherein the outward display is configured to display content towards outside of the XR device.

19. The XR device of claim 18, wherein the XR device is configured to perform the method of any one of claims 2-15.

20. An apparatus, the apparatus comprising: a memory (842); and processing circuitry (802) coupled to the memory, wherein the apparatus is configured to perform the method of any one of claims 1-15.

Images