Power management in an extended reality headset

Abstract

Engagement in scheduled activities by an Extended Reality, XR, device (203) is controlled by an auxiliary device (103, 201, 401, 403). Proposed first activity parameters of a proposed first activity are obtained (407, 501), the proposed first activity parameters including a proposed start time, a proposed first activity mode, and one or both of a proposed duration and a proposed ending time. Accepted activity parameters of an accepted activity are determined (411, 503) based on the proposed first activity parameters, wherein determining (411, 503) the accepted activity parameters comprises predicting (505) an initial XR device charge level that the XR device will have at the proposed start time; determining (507) whether the predicted initial XR device charge level will be greater than or equal to an estimated amount of required charge to perform the proposed first activity using the proposed first activity parameters; and when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then producing modified activity parameters by modifying (509) at least one of the proposed first activity parameters. The accepted activity parameters are communicated (411, 505) to the XR device (203). Modifying at least one of the proposed first activity parameters includes: when the proposed start time is modified, selecting (511) a modified start time that is later than the proposed start time; when the proposed first activity mode is modified, then selecting (515) a modified activity mode that consumes power at a lower rate than a power consumption rate of the proposed first activity mode; when the proposed duration is modified, then selecting (517) a modified duration that is shorter than the proposed duration; and when the proposed ending time is modified, then selecting (519) a modified ending time that is sooner than the proposed ending time.

Description

[0001] POWER MANAGEMENT IN AN EXTENDED REALITY HEADSET

[0002] BACKGROUND

[0003] The present invention relates to power management in a battery-powered device, and more particularly to management of energy usage by, and charging of a battery-powered Extended Reality (XR) device that ensures that sufficient power will be stored in a device battery when the device is called on to perform one or more planned energy consuming activities at scheduled later times.

[0004] Some or all of the following abbreviations are used in this specification:

[0005] Abbreviation Explanation

[0006] Al Artificial Intelligence

[0007] AR Augmented Reality

[0008] HGM Holographic Group Meeting

[0009] HMD Head Mounted Display

[0010] DVFS Dynamic Voltage and Frequency Scaling

[0011] GPS Global Positioning System

[0012] MR Mixed Reality

[0013] NF C Near Fi el d C ommuni cati on

[0014] UI User Interface

[0015] UX User Experience

[0016] VR Virtual Reality

[0017] XR Extended Reality

[0018] Extended Reality, or XR, technology is becoming ever more popular in both work and non-work settings. The term XR is a broad, generic one that encompasses a number of types of reality-enhancing devices including Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) devices. XR devices, in general, give the user what has been described as an immersive experience. In the case of AR devices, for example, the user experiences (e.g., by seeing and / or hearing) not only what is physically present in their environment (sometimes called, “In Real Life”, or IRL), but also things that are digitally created and seemingly placed in their physical environment. When done with high quality, the real and digitally created experience are integrated so that the user feels as though they are all substantially real and present.

[0019] Mixed Reality devices build on the capabilities of AR devices by further enabling the user to interact with digital placed objects placed within their real-life environment.

[0020] Virtual Reality differs from AR and MR in that the user’s experience is entirely digitally created, with (ideally) none of the IRL world leaking into the user’s perceptions.

[0021] For devices such as but not limited to AR glasses to succeed in the marketplace, they need to have adequate capabilities in terms of processing capabilities and display. But additionally, they also need to be slim and lightweight and it can readily be seen that this requires that they also have a small battery. Whereas chipmaking companies measure progress in their chipset platforms in terms of processing capabilities and clever partitioning of the chipset to fit attractive form factors, they will still have limited operating time due to limited on-device battery.

[0022] A similar problem has already been addressed in connection with a much simpler product - the in-ear headphones (“earbuds”). Such devices were already typically supplied with a case to contain these very small, lightweight pairs of devices when not in use. Such a case is useful for preventing the two separate devices (left and right earbuds) from becoming separated from one another, and also for protecting them from outside dangers (e.g., moisture, impact forces when dropped, etc.) To address the problem of relatively short battery capacity, the protective case was adapted to also include its own battery and necessary circuitry and connector ports for charging each earbud when it is inserted in the case.

[0023] But XR devices present a much larger scale problem, one that is not addressed by the simple solution of the earbud charging case.

[0024] In currently available technology, XR devices (e.g., AR, MR, and VR headsets, glasses, head mounted displays and the like) are charged directly via a physical wire tied to a power source. The connection to the device is typically made through a standardized port, such as but not limited to a USB-C port. Whereas this is practical for its versatile usage, it is not specifically convenient in the field. To take an example, although AR glasses can be charged by connecting a USB cable to a portable power bank when one is on the move, the glasses are then typically powered off or in a low power state as they are not currently being used by the user. The usage of the glasses is not automatically coordinated with the charging; those are presently both under direct control by the user, based on the user’s own assessment of when and for how long the device should be charged, and of when, how, and for how long the device should be used. During operation, the rate of power consumption of the glasses depends on the activity level of the processing chips, the modem operation, the different sensors that are activated, and the display utilization. Just like many other device types, the power consumption can be higher or lower depending on the activity level and performance demands, and these can be throttled (i.e., reduced) in several different ways. In present day smart phones, performance throttling is typically engaged for purposes of heat management, to make certain the device and chipset do not overheat, and for purposes of preserving stored energy when the battery level reaches certain low levels - the device might throttle down to a much lower activity level so that power consumption will go down and the battery will in turn last a little longer.

[0025] The use of throttling is also applicable to XR devices, but in this case, it is even more critical: the maximum operating level can be quite demanding due to, on the one hand, the power-intensive generation of complex holographic projections, while on the other, the fact that the battery is so much smaller. However, in conventional technology, there is no automated way of relating any planned user activity with the charging procedure that will be necessary to support the planned usage; these are instead typically managed manually by the user.

[0026] In view of the above, there is a need for technology that automatically manages the usage of an XR device in coordination with its charging for purposes of best matching the user’s needs with the overall usage, and for addressing related problems.

[0027] SUMMARY

[0028] It should be emphasized that the terms “comprises” and “comprising”, when used in this specification, are taken to specify the presence of stated features, integers, steps or components; but the use of these terms does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0029] Moreover, reference letters may be provided in some instances (e.g., in the claims and summary) to facilitate identification of various steps and / or elements. However, the use of reference letters is not intended to impute or suggest that the so-referenced steps and / or elements are to be performed or operated in any particular order.

[0030] In accordance with one aspect of the present invention, the foregoing and other objects are achieved in technology (e.g., methods, apparatuses, nontransitory computer readable storage media, program means) that controls engagement in scheduled activities by an Extended Reality, XR, device, wherein the controlling is performed by an auxiliary device. In an aspect of some, but not necessarily all inventive embodiments, proposed first activity parameters of a proposed first activity are obtained, wherein the proposed first activity parameters comprise a proposed start time, a proposed first activity mode, and one or both of a proposed duration and a proposed ending time. Accepted activity parameters of an accepted activity are determined based on the proposed first activity parameters, wherein determining the accepted activity parameters comprises predicting an initial XR device charge level that the XR device will have at the proposed start time and determining whether the predicted initial XR device charge level will be greater than or equal to an estimated amount of required charge to perform the proposed first activity using the proposed first activity parameters. When the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then modified activity parameters are produced by modifying at least one of the proposed first activity parameters, wherein modifying at least one of the proposed first activity parameters comprises: when the proposed start time is modified, selecting a modified start time that is later than the proposed start time; when the proposed first activity mode is modified, then selecting a modified activity mode that consumes power at a lower rate than a power consumption rate of the proposed first activity mode; when the proposed duration is modified, then selecting a modified duration that is shorter than the proposed duration; and when the proposed ending time is modified, then a modified ending time that is sooner than the proposed ending time is selected.

[0031] Further, the accepted activity parameters are communicated to the XR device.

[0032] In another aspect of some, but not necessarily all inventive embodiments, when the proposed start time is modified, an initial XR device charging activity is scheduled that begins earlier than the modified start time.

[0033] In yet an aspect of some, but not necessarily all inventive embodiments, predicting the initial XR device charge level comprises detecting a current level of battery charge of the XR device.

[0034] In still another aspect of some, but not necessarily all inventive embodiments, predicting the initial XR device charge level comprises estimating a level of battery charge that the XR device will have at the end of a previously performed activity. In another aspect of some, but not necessarily all inventive embodiments, obtaining the proposed first activity parameters of the proposed first activity comprises obtaining the proposed first activity parameters from an activity calendar of the XR device.

[0035] In yet another aspect of some, but not necessarily all inventive embodiments, at least a subset of the activity calendar is shared with one or more other XR devices.

[0036] In still another aspect of some, but not necessarily all inventive embodiments, the activity calendar is stored in the auxiliary device.

[0037] In another aspect of some, but not necessarily all inventive embodiments, the activity calendar is stored in the XR device.

[0038] In yet another aspect of some, but not necessarily all inventive embodiments, the calendar stored in the auxiliary device is synchronized with the calendar stored in the XR device.

[0039] In still another aspect of some, but not necessarily all inventive embodiments, obtaining the proposed first activity parameters of the proposed first activity comprises obtaining the proposed first activity parameters from a proposed first activity request received from another device.

[0040] In another aspect of some, but not necessarily all inventive embodiments, a level of battery charge that the XR device will need to perform a next activity is estimated. Further, determining the accepted activity parameters comprises estimating that a level of battery charge that the XR device will have at the proposed ending time of the proposed first activity is less than the level of battery charge that the XR device will need to perform the next activity. In response to the determining, one or both of the following are performed: selecting the modified duration that is shorter than the proposed duration; and selecting the modified ending time that is sooner than the proposed ending time.

[0041] In yet another aspect of some, but not necessarily all inventive embodiments, when the estimated level of battery charge that the XR device will have at the end of the proposed first activity is less than the level of battery charge that the XR device will need to perform the next activity, then an XR device charging activity that begins no earlier than an accepted ending time is scheduled.

[0042] In still another aspect of some, but not necessarily all inventive embodiments, when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then determining the accepted activity parameters of the accepted activity comprises splitting the proposed first activity into at least first and second partial activities, wherein: the first partial activity has a first partial activity start time and a first partial activity end time; and the second partial activity has a second partial activity start time and a second partial activity end time.

[0043] Further, an intervening charging activity is scheduled to take place in between the first partial activity end time and the second partial activity start time.

[0044] In another aspect of some, but not necessarily all inventive embodiments, the charging of the XR device occurs when a charging circuit in the XR device is coupled to a charging circuit in the auxiliary device.

[0045] In yet another aspect of some, but not necessarily all inventive embodiments, content comprising one or more of an application program, data and files that will be used by the XR device in one of the scheduled activities is determined and supplied to the XR device while the charging circuit in the XR device is coupled to the charging circuit in the auxiliary device. In some but not necessarily all such embodiments, prior to supplying the content to the XR device, the content is downloaded from a server and stored in the auxiliary device.

[0046] In a further aspect of some but not necessarily all such embodiments, determining content that will be used by the XR device in one of the scheduled events comprises sending a command to the XR device, wherein the command instructs the XR device to send information identifying the content to the auxiliary device; and receiving the information identifying the content.

[0047] In another aspect of some but not necessarily all such embodiments, the content includes an avatar model of a user that that the XR device is to display during said one of the scheduled activities. In some embodiments, the content includes a portion of video and / or audio information that lasts no longer than a duration that the XR device is capable of playing without requiring a recharging of the XR device. The video and / or audio information can, but need not be one part of a longer video and / or audio stream that is downloaded by the auxiliary device.

[0048] In still another aspect of some, but not necessarily all inventive embodiments, determining the control schedule further comprises inserting an XR device control parameter that causes the XR device to operate at less than full power while performing the first activity.

[0049] In another aspect of some, but not necessarily all inventive embodiments, the accepted activity mode parameters include an accepted mode parameter that is selected from a set comprising an immersive interactive video conference, an immersive non-interactive video presentation, a two-dimensional video conference, an augmented reality application, and an audio-only application. In yet another aspect of some, but not necessarily all inventive embodiments, an initial XR device charge level that at least one other XR device will have at the proposed start time is predicted; and at least one determination outcome is produced by determining whether the predicted initial XR device charge level of said at least one other XR device will be greater than or equal to an estimated amount of required charge for said at least one other XR device to perform the proposed first activity using the proposed first activity parameters. In such embodiments, determining the accepted activity parameters is further based on the at least one determination outcome.

[0050] In still another aspect of some, but not necessarily all inventive embodiments, prior to communicating the accepted activity parameters to the XR device, the accepted activity parameters are communicated to at least one other entity as a set of acceptable activity parameters. A confirmation is received in response to the communication of the acceptable activity parameters to the at least one other entity. Further, communicating the accepted activity parameters to the XR device is performed in response to the confirmation. In some but not necessarily all such embodiments, the confirmation is a set of agreed on activity parameters and the agreed on activity parameters are communicated to the XR device as the accepted activity parameters.

[0051] In some but not necessarily all such embodiments, the at least one other entity comprises another XR device.

[0052] In some but not necessarily all such embodiments, the at least one other entity comprises a calendar request server.

[0053] In some but not necessarily all such embodiments, second proposed first activity parameters of the proposed first activity are received in response to the communication of the acceptable activity parameters to the at least one other entity; and an acceptance of the second proposed first activity parameters or a second set of acceptable activity parameters is sent to the at least one other entity.

[0054] BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The objects and advantages of the invention will be understood by reading the following detailed description in conjunction with the drawings in which:

[0056] Figure 1 illustrates an auxiliary device formed as a protective case into which an XR device can be placed when not being used and in which there is circuitry for carrying out actions in accordance with inventive embodiments. Figure 2 illustrates an exemplary auxiliary device and an XR device that it can interact with.

[0057] Figure 3 shows the calendars of two users that are to participate together in a proposed XR activity.

[0058] Figure 4 shows exemplary actions and signaling flow for smart scheduling of XR device activities in accordance with some inventive aspects.

[0059] Figures 5A and 5B depict actions that can be performed by an auxiliary device in some aspects of inventive embodiments.

[0060] Figure 6 illustrates aspects of a proposed activity and an accepted activity having modified parameters, in accordance with some aspects of inventive embodiments.

[0061] Figure 7 shows an exemplary controller that may be included in a device to cause any and / or all of the herein-described and illustrated actions associated with that device to be performed.

[0062] DETAILED DESCRIPTION

[0063] The various features of the invention will now be described with reference to the figures, in which like parts are identified with the same reference characters.

[0064] The various aspects of the invention will now be described in greater detail in connection with a number of exemplary embodiments. To facilitate an understanding of the invention, many aspects of the invention are described in terms of sequences of actions to be performed by elements of a computer system or other hardware capable of executing programmed instructions. It will be recognized that in each of the embodiments, the various actions could be performed by specialized circuits (e.g., analog and / or discrete logic gates interconnected to perform a specialized function), by one or more processors programmed with a suitable set of instructions, or by a combination of both. The term “circuitry configured to” perform one or more described actions is used herein to refer to any such embodiment (i.e., one or more specialized circuits alone, one or more programmed processors, or any combination of these). Moreover, the invention can additionally be considered to be embodied entirely within any form of non- transitory computer readable carrier, such as solid-state memory, magnetic disk, or optical disk containing an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein. Thus, the various aspects of the invention may be embodied in many different forms, and all such forms are contemplated to be within the scope of the invention. For each of the various aspects of the invention, any such form of embodiments as described above may be referred to herein as “logic configured to” perform a described action, or alternatively as “logic that” performs a described action.

[0065] For reasons including those set out in the Background, there is a need for technology that automatically manages the usage of an XR device in coordination with its charging for purposes of best matching the user’s needs with the overall usage. In an aspect of some inventive embodiments, the handling of charging and usage are coordinated in calendar tools with the goal being to ensure that the usage of the XR device in a meeting session or in a predicted personal usage will not be unrealistic based on charging needs and recharging sessions that will be necessary between or during the usages. As used herein, the terms “calendar”, “calendar tool”, and the like mean not only conventional calendar tools as have been known in the art for many years, but also calendar-like tools that do not necessarily have a user interface that looks like a calendar but which nonetheless organize the scheduling of a user and / or the user’s utilization of an XR device. For example, the term “calendar tool” encompasses, but is not limited to, technology in which an Artificial Intelligence (Al) makes decisions about how power levels and charging intervals should be planned, with the decisions being based on a device utilization history, other calendar information, and the like. To allow the user to have the ability to adjust the scheduling decisions if needed, they can be visualized and reviewed in a calendar-like tool.

[0066] In another aspect of some but not necessarily all inventive embodiments, when several people using XR devices might be involved in virtual meetings, a coordinated usage / charging strategy is produced that takes into consideration the requirements of all user devices, not just one. In some embodiments, this further takes into account the fact that different devices have different battery capacity, consume energy for different tasks (when not engaged in the commonly attended meeting), and have different charging speeds.

[0067] In another aspect of some but not necessarily all inventive embodiments, given that it is not always possible to have a perfect match between usage needs and estimated battery consumption, throttling of performance is scheduled when needed to stretch the usage time before the battery is exhausted. It is advantageous, but not essential, for such throttling to start as soon as possible in order not to become too disruptive (e.g., noticeable) for the planned use case.

[0068] In another aspect of some but not necessarily all inventive embodiments, the above and other aspects are embodied in a case or container for the XR device (e.g., a case for AR glasses).

[0069] In overview, among the various embodiments of the invention are methods and tangible apparatuses that perform the methods. When embodied as an apparatus, it is advantageous for the apparatus to be in the form of an auxiliary device that can interact with a user’s XR headset. Phy sically, the auxiliary device can serve several purposes: one being to carry out various aspects of inventive embodiments as described herein, and another serving as a protective case into which an XR headset can be placed when not in use. Figure 1 illustrates this point by showing an auxiliary device 100 formed as a protective case 101 into which an XR device can be placed when not being used. Within the protective case 101 is circuitry 103 configured to carry out some or all of the areas aspects that are described in the following.

[0070] Figure 2 illustrates an exemplary auxiliary device 201 and an XR device 203 that it can interact with. The XR device 203 can be, for example and without limitation, a pair of AR glasses or other form of XR headset.

[0071] The auxiliary device 201 includes a battery 205 and, in this exemplary embodiment, inductive charging circuitry 207 that together enable the auxiliary device 201 to charge 231 the XR device 203 via induction when it is resting in the auxiliary device 201. Alternative charging mechanisms are also contemplated within the scope of inventive embodiments including, but not limited to, electrical connections made by electrical contacts or by a charging port. The auxiliary device 201 further includes processing capability illustrated herein as the processor 209. It should be understood that, as used herein, references to a “processor” are intended to encompass embodiments in which there is a single processor, or multiple processors working together.

[0072] The auxiliary device 201 also includes a memory 211 that not only provides working storage as may be required by the processor 209 but also maintains an up-to-date context of the user's system, such as the state of the calendar. The memory 211 also stores media content, and any other storage and might be needed by the XR device 203.

[0073] The processor 209 in combination with the memory 211 (including calendar state information and one or more programs stored in the memory 211 and executed by the processor 209) are configured to have control of the charging of the XR device 203 and also other applications such as calendar support, memory, downloading, push / pull of content, application updates, and the like.

[0074] In operation, the XR device 203 and the auxiliary device 201 are kept fully synchronized in their content and status, including but not limited to cloud services such as calendar, social media, event handler, and the like. Communication and control signaling 233 between the auxiliary device 201 and the XR device 203 can be implemented by any number of ways, including but not limited to High-speed NFC-type, or radio-based, communication between the XR device 203 and the auxiliary device 201 when the XR device 203 is placed in the auxiliary device 201, enabling rapid pre-loading or synchronization of content at a minimal power consumption. This would enable pre-loading of data to the XR device 203 without draining its battery 223 when removed from the case.

[0075] The auxiliary device 201 doubles as a computational extension to the XR device 203, providing processing capabilities similar to those that one might find in a smartphone. The memory 211 (possibly supplemented with additional storage the auxiliary device 201 might have) can prepare content to be loaded into the XR device 203 while they are stored in or otherwise in communication with the auxiliary device 201 and are configured to stream / transfer content from the auxiliary device 201 to the XR device 203 from a short distance when these are being used and therefore not in the auxiliary device 201 (assuming the auxiliary device 201 is within reasonable proximity to the XR device 203). A modem 213 can be utilized for this purpose as well as for other communication needs (e.g., 5G wireless connectivity to the Internet) as are further described below.

[0076] The auxiliary device 201 is also equipped with sensors 215 configured to detect when the XR device 203 is in contact with, or sufficiently close to the auxiliary device 201 so as to enable charging and performance or at least triggering of other functionality as described herein. As will be described in greater detail below, this other functionality includes the communication 233 of control signaling between the two devices, and also an exchange of information.

[0077] It will be understood that the exemplary auxiliary device 201 as illustrated in Figure 2 may, in alternative embodiments, include more or fewer features than those illustrated here. A person of ordinary skill in the art will readily be able to adapt the auxiliary device 201 to suit particular needs.

[0078] XR devices such as the exemplary XR device 203 are generally known to include components such as a display 217, a processor 219, memory 221, and a battery 223. In accordance with some but not necessarily all inventive embodiments, an XR device 203 also includes sensors 225 and the modem 227 that facilitate interactions with the auxiliary device 201. Furthermore, the processor 219 and memory 221 are configured to operate with, and be responsive to, the various interactions with the auxiliary device 201 as described herein.

[0079] An aspect of inventive embodiments is a strategy in which the auxiliary device 201 asserts some level of control over the usage and charging of the XR device 203. Controlling the usage of the XR device 203 can, in some instances, include controlling when the device will be operated, controlling whether performance throttling will be applied and if so, what type, and the like. Controlling the charging of the XR device 203 can, in some instances, include scheduling times when the XR device 203 will need to be charged in order to perform an upcoming task. These and other aspects to be described are performed with the goal of minimizing the power consumption of the XR device 203 when necessary, and of maximizing the likelihood that the XR device 203 will have sufficient power to remain operational for any planned or expected sessions. One purpose can be to minimize unwanted disruption caused by lack of battery or by the device having to operate in a critical power save mode (which typically results in the user interface and / or user experience being drastically reduced).

[0080] In another aspect of some but not necessarily all embodiments, the auxiliary device 201 is configured to have a calendar function, including the planned usage of the XR device 203 in planned meetings and / or ongoing use cases. These use cases can include, but are not limited to, video / XR meetings, games, city -navigation, and shopping sessions in the mall.

[0081] To support this, the auxiliary device 201 in some embodiments keeps track of the charging status and charging time of the XR device 203. This information is matched against the calendar and / or the needs of the ongoing applications or use cases. Estimated power requirements corresponding to the calendar bookings are automatically compared with estimated power consumption of sessions that use the XR device 203 to make certain that there will be enough charge stored in the battery, or that charging periods are automatically inserted.

[0082] Furthermore, in some instances due to the anticipated duration of the activities and estimated power consumption of the activities, the system will automatically throttle the activities and processing to a certain extent to better match a suitable duration of those activities (e.g., to stretch out a holographic video call from 25 minutes to the planned, say, 30 minutes - instead of executing at full performance for the first 20 minutes and then continuing in a severe power-save mode during the last minutes with severe degradation of the user experience and risk of premature power failure.

[0083] In another aspect of some but not necessarily all inventive embodiments, in sessions involving multiple users, the charging and battery-duration of each of the involved XR devices can be automatically synchronized, meaning that the session and performance levels are adapted to benefit the complete group rather than just one or a few members of the group. In some embodiments, this coordination includes how availability is indicated to others with respect to virtual sessions, and the system supports the coordination of XR device usage and planned charging periods for all participants. In some embodiments, this may also or instead include a throttling of the experience for some of the participants so that their devices will remain operable for the duration of the planned session. In still another aspect of some but not necessarily all inventive embodiments, for ongoing use cases based on the users habitual use of the XR device 203 such as “walking to a city center with navigation”, the auxiliary device 201 can adapt the activity level and charging sessions of the XR device 203 to the status of the estimated path as well as the needs when approaching the destination, so that the XR device 203 will not be fully used during the first long distance to the city center only to have the device battery run flat a few minutes before arriving. This means that, not only can the XR device 203 automatically throttle performance and perhaps even go to sleep and wake-up again during the session, but can also support the performance of these actions by prompting or reminding the user (e.g., by projecting an image on the display 217) when to put the XR device 203 into the auxiliary device 201 for just a suitable duration to get sufficiently charged for the next active period. The auxiliary device 201 may also include GPS or other positioning support, and this can be programmed to notify the user when they have arrived at a relevant point of interest by waking up the XR device 203 from a sleep mode. Such notification can be enabled in any of a number of different ways, all of which are contemplated to be within the scope of inventive embodiments. For example, when the XR device 203 is inside the auxiliary device 201, the XR device can, in some embodiments, be equipped with some type of output device (e.g., audio, visual, tactile, etc.) that provides notifications to the user (e.g., sounds and / or flashing lights or text). Alternatively or in addition, the auxiliary device 201 can have a radio connection to another device that the user has, such as a smart watch or phone. In such cases, notifications can be communicated to and then presented by these other devices.

[0084] In other alternative instances in which the XR device 203 is being worn by the user but is in a low power mode (e.g., standby mode), it can receive messages / instructions from the auxiliary device 201 which cause the XR device 203 to wake up and provide the positioning information to the user at the relevant time.

[0085] Yet other functionality in some embodiments includes preparing data that is adapted to conform with the duration of a next activity, charging status, and estimated power consumption. This can be done by preparing content while the XR device 203 is in or otherwise connected to the auxiliary device 201 to minimize communication and for example resolution (or downsampling). One example is video or game consumption while the XR device 203 is being used on the subway for an estimated amount of time. When entering the subway, the user accesses the auxiliary device 201 for the planned use case, and the system has preloaded the content into the memory of the XR device 203 and guides the throttling for the estimated duration of the session. The guidance of what to do can come from the calendar, from an ongoing session from an earlier time, or via input from audio, smartphone, or smart watch. Furthermore, for a joint session between two friends, this can be automatically synchronized before the XR device 203 is removed from the case and the battery consumption must be minimized.

[0086] It will be understood from this overview that some but not necessarily all features relate to a system that automatically supports the adjustments of multiparty scheduled events, such as multiparty AR, MR, or VR meetings, with the adjustments taking the estimated battery consumption and charging time into consideration for all participants. The technology also provides for automatic agreement on, and application of dynamic throttling to adjust and align activity duration.

[0087] In some embodiments, data and / or files are preloaded into an XR device 203 while they are in or otherwise connected to an auxiliary device 201. This preloading minimizes power consumption by the XR device 203 and consequently prolongs the duration of activities before the stored charge in the XR device battery 223 is exhausted.

[0088] An additional feature in some but not necessarily all inventive embodiments is an automatic throttling of performance level during the full operation of activities for scheduled events in order for the agreed duration to be met with minimal impact on user experience.

[0089] Still another feature in some but not necessarily all inventive embodiments is automatically alerting the user of a need to recharge the XR device 203 in order to match one or more upcoming events even if the XR device 203 is not close to a critical battery level.

[0090] And yet another feature in some but not necessarily all inventive embodiments is adaptively selecting between low-power modes, power-off modes, and re-charging of the device during an activity (e.g., navigation) based on future estimated energy consumption of not only the current operation but also future activities in order to match not only the duration of events but also their needs (e.g., to handle a situation in which a need for more intense navigation guidance will be coming up after, say, 10 minutes).

[0091] These and other aspects are described in greater detail in the following.

[0092] One aspect of at least some inventive embodiments involves the auxiliary device 201 being configured to perform calendar-based coordination of the charging of multiple more XR devices 203. To illustrate this aspect, Figure 3 shows the calendars of two users, “A” and “B” at a time when a third user, “C” wants to schedule a holographic group meeting. Assume that a holographic group meeting (HGM demands more system resources than a two-person video meeting, and that the XR device of User A can run a HGM for 22 minutes when starting with a 100% battery charge whereas User B has a different headset that can run a similar HGM for 30 minutes when starting with a full battery. Further, assume that the system has the capability of extending the use time by a few minutes by throttling down the performance in predefined steps, where an 80% level reduces the resolution somewhat, resulting in fewer details and less generated Al content being displayed.

[0093] If User C has a free morning and is therefore available at any mutually agreeable time between User A and User B, the calendars (as shown in Figure 3) might appear to indicate that the system can schedule a 30-minute holographic group meeting 10.00 - 10.30. However, due to the battery constraints discussed above, this would not be possible. In particular, it can be predicted that User A, who has been busy first with a HGM from 09.00 - 09.30 and then with a video meeting between 09.30 and 10.00 might have a battery state of only 10% at 10.00. User B on the other hand is not scheduled to use their XR device 203 (the in-person meeting does not require one) and therefore is expected to have a full battery if User C’s proposed HGM were to start at 10.00. But this is not the end of the story, because User B’s calendar also shows a planned two-person video meeting starting at 10.30 and this will require that User B’s XR device 203 have a sufficient battery level available at the start of the 10.30 meeting in order to participate for the full 30-minute meeting duration.

[0094] To solve this problem, some aspects of inventive embodiments, implemented in the auxiliary device 201, control the charging and projected battery status of the XR device 203, based on the type of activity and what that activity will demand from said system. This information is used when scheduling activities and also to help guide the user towards complying with such schedules.

[0095] Further in this regard, Figure 4 shows exemplary actions and signaling flow for smart scheduling of VR device activities in accordance with some inventive aspects. As an illustration, two auxiliary devices, a first auxiliary device 401 and a second auxiliary device 403 are engaged in negotiating the scheduling of their associated XR devices (not shown in Figure 4) for purposes of conducting a meeting of generic type X, having a length, L. The negotiations take place between the first and second auxiliary devices 401, 403 with a calendar request server 405 acting as an intermediary. For each user’s XR device, the scheduling takes into account that device’s battery status, power consumption, charging rates and opportunities, and potential throttling. This is done by each first and second auxiliary device 401, 403 without the need for any involvement by their related XR devices, because each auxiliary device 401, 403 has the necessary knowledge about its associated XR device’s battery status and charging parameters. The actions performed by each auxiliary device 401, 403 follow the same logic flow, although it will be recognized that each is working with its own individual XR device’s related parameters, so the outcomes of decision-making may be different. Hence, negotiation in this type of endeavor is also involved.

[0096] If we assume the same calendaring goals as presented with respect to Figure 3 (i.e., some type of meeting request made by a third user), the calendar request server 405 sends 407 a request for an XR meeting of type X and length L. Focusing on the actions taken by the first auxiliary device 401 (actions taken by the second auxiliary device 403 will mirror those described here), the auxiliary device 401 responds to the received request by estimating 409, for its related XR device, the energy consumption required to conduct an XR meeting of type X and length L. The estimation is based on, for example:

[0097] • Power consumption rates associated with meeting type X (e.g., HGM) at full performance or for certain defined throttle-down scenarios that might be available for extending the usage time.

[0098] • Total amount of energy consumption for the duration L given full performance and optionally as well as for relevant throttle-down scenarios.

[0099] • Estimated battery levels expected to be available at the beginning of a potential timeslot

[0100] • Estimated required battery level required at the end of a potential timeslot to supporting next activity.

[0101] Based on this, the auxiliary device 401 returns 411 possible scenarios. This can be done graphically or by providing meta-data with the necessary information. Scenarios would then include requirements for re-charging periods, possible durations if the full requested duration L cannot be met, and certain throttle-down scenarios. The latter is important in case the different users align on certain reduced features.

[0102] The calendar request server 405 assesses 413 scheduling options received from each auxiliary device 401, 403.

[0103] To illustrate this aspect, in the above example, the system might get an indication that:

[0104] • User A: Can start at 10.05 for a 15-minute session operating at full performance, 10.10 for a 22-minute session operating at full performance, 10.05 for a 20 minute session operating at 80% performance (i.e., throttling applied). The proposed start time, 10.00, would instead be the start of an inserted charging activity. • User B: Can start at the proposed time of 10.00, but needs to stop at 10.26 to perform recharging in case a full performance meeting is to be started at 10.30, or stop at 10.27 if the HGM is conducted with the XR device throttling down to 80%.

[0105] Either the user or an automated aspect of the system selects 415 a new proposed timeslot (i.e., start and end times). For example, a revised calendar event might be proposed that schedules and HDM from 10.05 - 10.25 for a 20 minute meeting at 80% performance level for User A, whereas User B might do the meeting operating at full performance or at 80% performance. The revised proposal is then sent 417 to each of the first and second auxiliary devices 401, 403. The revised proposal has the same XR meeting type X for each auxiliary device. However, it will be observed that the proposed performance compromises (if any) are individualized for the different users (e.g., as illustrated, the first auxiliary device 401 receives a proposal that involves compromises Cl, whereas the second auxiliary device 403 receives a proposal that involves compromises C2).

[0106] Unless there is a reason not to accept the revised proposal, each auxiliary device 401, 403 updates 419 its calendar to include a scheduled HGM at the accepted timeslot, device settings and constraints. Each auxiliary device 401, 403 sends 421 a confirmation back to the calendar request server 405 which then enters this scheduling information into a shared calendar.

[0107] It is noted that Figure 4 is one of many possible exemplary embodiments, and that alternative actions and even nomenclature can be applied. For example, returning possible scenarios as illustrated in step 411 can be considered a communication of modified parameters or of acceptable parameters. In another alternative, the revised proposal of step 417 could be considered to be an agreed on set of activity parameters. Such could be the case, for example, if another entity is responsible for determining mutually agreeable parameters for all parties involved in the scheduling. And in step 419, once an auxiliary device 201, 401, 403 has reached what it considers to be a final set of parameters (either by deriving them itself or by receiving them from another entity), those parameters can then be considered to be accepted activity parameters that can be communicated / loaded into the XR device 203.

[0108] In yet other alternative embodiments, instead of the calendar request server 405 itself assessing possibilities (as in step 413) and selecting a timeslot and updating the requests (as in step 415), the system supports a user themself making a timeslot selection and triggering request updating. For example, the calendar request server 405 assesses possibilities based on collected scheduling options from participants, and then causes one or more suggested scheduling options to be presented (e.g., displayed) to each potential participant. Exactly what is suggested for each participant is dependent on the type of meeting being requested, since different types need to take different device resources into consideration. Importantly, in these embodiments the calendar request server 405 can only make suggestions; it is still up to a user to make a final selection. This is because, to take one non-limiting example, the requested meeting might be more important than other meetings in the user’s calendar, and then the calendar request server 405 cannot fully decide. But then conflicts might show up for other users, and those conflicts can go beyond merely determining whether the other user is busy or not at the requested time slot. The conflicts can, in fact, also take into consideration conflicts of resource types (e.g., whether a user’s XR device 203 might need to be charged before being able to participate in a suggested meeting). By presenting invited guests with suggested meeting times, they then have the option of considering all relevant aspects of the suggestion and possibly replanning their other engagements to eliminate conflicts in their calendars.

[0109] This description has so far described instances in which the automated scheduling inserts a charging activity, that is, an interval during which the XR device 203 needs to be charged in order to carry out one or more further scheduled activities. The auxiliary device 201, 401, 403 includes necessary means for carrying this out. However, the XR device 203 needs to be somehow connected to the auxiliary device 201, 401, 403 to make this happen, and that requires some sort of user action. Additionally, it is advantageous (although not required) for the user to also be reminded when a scheduled activity is about to begin. With one or both of these in mind, in another aspect of some but not necessarily all embodiments, in order to support the proper scheduling of events, the auxiliary device 201, 401, 403 alerts the user to start a meeting, or to start the charging. These alerts may be routed to the XR device 201, 401, 403 itself (assuming that the device is being worn) and / or via other communications means such as but not limited to a smart watch, smart ring, or a smartphone. In some but not necessarily all embodiments, the auxiliary device 201, 401, 403 has alert-opportunities built in, similar to a smartphone (with or without a display).

[0110] In another aspect of some but not necessarily all embodiments, in order to maximize usetime of the XR device 203, the auxiliary device 201, 401, 403 pre-loads content prior to usage. This can include preloading content in its entirety while for example an XR device 203 is placed in an auxiliary device 201, 401, 403. But when on-device storage in the XR device 203 is smaller than the storage in the auxiliary device 201, 401, 403 what is stored in the XR device’s local storage should be managed with care. This can be efficiently handled with the following mechanism in accordance with another inventive aspect:

[0111] 1. A calendar event or command indicates a type of use to be carried out in a certain timeslot, with the indicated use requiring that related content be stored in the memory 221 of the XR device 203.

[0112] 2. The auxiliary device 201, 401, 403 downloads the relevant content to its memory 211 in advance.

[0113] 3. The auxiliary device 201, 401, 403 pre-loads the data into the memory 221 of the XR device 203 during a charging session prior to its use.

[0114] 4. When the XR device 203 is removed or otherwise decoupled from the auxiliary device 201, 401, 403 with a full battery (or at a target-level for the next usage), the content is already preloaded to the XR device 203, thereby eliminating the need to expend energy downloading such content (e.g., wirelessly).

[0115] Step (1) above can be based on a command directed to the XR device 203 (for the next usage) or to the auxiliary device 201, 401, 403. It can be based on vocal input, or by some other type of UI. It can also be based on information in the meeting request (e.g., a common document, video, or similar that is needed).

[0116] The content to be preloaded can be based on an avatar model of the user that the user will interact with during the next session. For example, suppose User X schedules an avatar-based session with User A, and that the session is scheduled as an Avatar-Session. When the XR device 203 is charged prior to the meeting, the avatar model of User X is preloaded onto the XR device 203, so it is readily available before the start of the meeting. After the meeting, when the XR device 203 is re-charged for the next scheduled event, this avatar model is removed from the XR device 203, and other relevant data is loaded instead. This can instead be related to a volumetric presentation, video, and the like.

[0117] It can also be that during one part of a longer session, the meeting group agrees to use certain material or a certain application after some while. As the group performs the charging of their devices, the devices are also preloaded with the agreed upon content (which can be synchronized based on the selection by one of the participants) and be readily available on the devices when they are reactivated after charging.

[0118] Another example presents itself in connection with the viewing of a video during a user activity (e.g. commuting to or from work using public transportation). If the user watched the first 20 minutes of a video during the commute in the morning, the auxiliary device 201, 401, 403 can predict that the return trip in the afternoon might demand the next 20 minutes (depending on the predicted ride) and in response to this prediction, preloads the corresponding data into the XR device 203.

[0119] In another use case, if the user is watching a relatively long movie (e.g., 90 minutes long) and the XR device 203 can operate for only 30 minutes before needing to be recharged, the auxiliary device 201, 401, 403 can automatically preload the coming 30-40 minutes of video during charging, reducing the need for wireless streaming of the movie. But it need not preload the complete movie since it is not likely that the user will manage to watch it before the battery is exhausted.

[0120] In another embodiment, the system can preload documents related to, for example, a previous chat before starting a video or holographic meeting, since it is likely that those documents will be needed during the meeting. This is especially important in instances involving media files as those might be more demanding to transfer wirelessly when the XR device 203 is decoupled from the auxiliary device 201, 401, 403.

[0121] Intelligent charging and scheduling by the auxiliary device 201, 401, 403 is also useful in use cases involving navigation or device usage during a predefined path. One typical scenario is the use of an AR-application for an extended time, for example navigation while walking to or within a particular destination such as a city center. The navigation tool is able to estimate the expected duration, and the calendar might contain relevant information about the activity (e.g., whether there is a concurrent need to perform another activity (e.g., engage in a holographic call) either during the navigation or when arriving at the destination. Based on estimated energy use (see above, also considering throttle-down scenarios) and battery status, the system can schedule both energy-save states and re-charging periods during the session. The scheduling logic for this exemplary use case can be, for example:

[0122] • If there are deep-sleep states for certain periods that are sufficient to extend the usage (e.g., when no turn or other event occurs), then there is no re-charge needed unless there is a use case that needs to be performed when arriving at the destination. o Since the auxiliary device 201, 401, 403 would typically contain position support, (e.g., GPS), the XR device 203 can be put into a sleep mode, and the auxiliary device 201, 401, 403 can wake-up the XR device 203 at appropriate or predefined positions or after pre-defined time durations. Wake-up functionality can be supported by, for example, having the XR device 203 maintain some level of radio functionality even while in a sleep / low-power state. Wake-up features are well known in the art (e.g., in smartphone technology) and need not be described here in great detail.

[0123] • If this is not sufficient, then re-charging must be scheduled. In this instance, the user is prompted by the XR device 203, smart watch, and the like when to put the XR device 203 into the auxiliary device 201, 401, 403 and later, when to pick them out again. Navigation support in the meantime can be supported by the use of GPS in a smart watch or in the auxiliary device 201, 401, 403.

[0124] • Throttling-down performance to a minimum depending on required support is also available to make certain that the energy will be sufficient for the complete, planned usage.

[0125] • If there is extended battery drain (e.g., due to an unexpected holographic video call appearing during the walk), the user is notified on the energy-related consequences of this for the range / throttling-options, and can then guide a suitable scheduling of events, (e.g., whether it is alright to accept the holographic call but to add a re-charging activity during the next period, or whether to throttle-down the call to a low quality in order to extend the range of navigation so as not to lose time.

[0126] This handling is enabled by the estimated usage and throttling options in line with the smart calendar taking charging and energy consumption into consideration.

[0127] Aspects of various embodiments that are consistent with the invention involve making estimates of energy consumption and of charging times. The auxiliary device 201, 401, 403 is responsible for charging the XR device 203 and is therefore able to build an extensive set of statistics covering charging speeds and energy consumption / battery duration over its usage in different power and / or throttling modes. Therefore, estimation based on estimated energy usage and battery charge / drain models can be extended by improved models taking such real-life statistics into consideration. Even though the estimated energy use will differ from the actual instances, an active monitoring of the actual energy usage can be followed up during the use cases, and in addition to leading to an updated battery -usage estimate it can lead to dynamic throttling to align with the scheduled duration of the activity.

[0128] For example, consider the following example in which an estimated duration is 25 minutes. Suppose that the actual usage over the initial 5 minutes indicates a 10% increased use of energy due to, for example, communication needs being higher than anticipated (e.g., the device being further away from a base station base station and consequently requiring a higher power amplifier output power; richer content to be rendered; less battery capacity due to very low temperature). Then, since the system has a scheduled duration of operation, the system will automatically throttle down the performance gracefully to compensate for the increased activity level. By reducing the clock speed of the processor cores, reducing the resolution of generated images, reducing certain background activities, and the like, the system increases the likelihood of achieving the target duration with a minor impact on user experience than if it were to wait until the battery level reaches certain threshold limits before responding.

[0129] This dynamic fixed-duration throttling is imperative for the use cases, especially for activities where the duration is agreed on beforehand (e.g., a meeting with other participants). However, for other types of activities not including other participants or having a duration which is synchronized with external events and / or factors (e.g., navigation), the system can either by default continue with the target performance but for a shorter duration (which is the case for normal smartphones and tablets today) or provide feedback to the user and ask them to decide.

[0130] Some aspects of inventive embodiments relate to ad hoc activities and automatic coordination of multiparty activities. More particularly, in some of above examples, activities are scheduled in advance and this allows for planning of charging, duration of events, and throttling to gracefully adapt the duration of the events to match certain needs or alignment towards other participants. However, the smart charging and throttling is also useful in situations in which ad- hoc activities occur (e.g., a spontaneous video or holographic session between two participants). This is illustrated in the following:

[0131] (1) User A calls User B for a holographic session.

[0132] (2) A smart energy manager, located in each participant’s device, assesses the respective battery status and estimated duration of such a call for each participant.

[0133] (3) For the user that has the shortest capability with respect to call duration, the throttling opportunities are assessed to estimate the potential increase in call duration if a certain level of throttling is applied.

[0134] (4) If this results in a similar call duration for each of the users, or if both users are estimated to support a normal call duration, the corresponding throttling is applied.

[0135] (5) The possible call duration is optionally indicated to the users, allowing them to either agree on further throttling if the duration is not long enough, in which case both devices automatically adjust the individual throttling of each device to match that (within a reasonable amount to still support the type of application). (6) When the call duration limit is approached, before the battery runs too low, the system suggests that the users insert a break time for use as a charging activity, or to switch to a less energy-demanding call application (allowing dynamic handover to such application).

[0136] Furthermore, in case one or both of the involved users have an upcoming activity scheduled, the estimated call duration takes that upcoming activity into account, including the required battery level required at the start of that other activity to remain operational throughout, and the charging time needed to achieve that.

[0137] In line with the above mechanism, alternative embodiments can be applied for more than two involved participants.

[0138] In still other alternatives, a similar approach can be applied in scenarios in which a single-user, ad-hoc activity is started. In these cases, the estimated duration for each of a number of throttling schemes is estimated, taking potential upcoming scheduled activities into consideration, and taking typical or an expected desirable activity-duration into account (e.g., the length of a TV show, the typical gaming session duration, etc.).

[0139] In embodiments described above, the overall control for calendar and guidance of charging is handled by the auxiliary device 201, 401, 403 because it naturally has full insight into the status of the XR device 203, the charging statistics, and pre-loading of content.

[0140] However, in alternative classes of embodiments, this control function can reside in a smartphone or other device that is in communication with the XR device 203 and also the auxiliary device 201, 401, 403 for status, charging, and power consumption statistics. This alternative device can also be in control of the calendar contents and in control of the various notifications sent to related units.

[0141] In still further alternative embodiments, if the user does not comply with the schedule of returning the XR device 203 to the auxiliary device 201, 401, 403 for re-charging, the system alerts the user with increased information explaining why this is important (e.g., with information about upcoming consequences such as “The duration of your upcoming holographic call cannot be guaranteed”). Furthermore, if there is a planned joint re-charging break, and one of the participants misses putting their XR device 203 into the auxiliary device 201, 401, 403, a second reminder is, in further alternatives, also shown as an alert to the other participants to notify that user - this is important since the potential low battery of one of the user's XR devices might disrupt the quality of the joint activity for all participants. Still further aspects of some but not necessarily all inventive embodiments will now be described with reference to Figures 5A and 5B, which in one respect are each a flowchart of actions performed by a device that ensures sufficient energy for an XR device 201, 401, 403 to perform a proposed activity in accordance with some but not necessarily all inventive embodiments. In other respects, the blocks depicted in each of Figures 5 A and 5B can also be considered to represent means 500 (e.g., hardwired or programmable circuitry or other processing means) for carrying out the described actions.

[0142] Actions depicted in Figure 5 A and 5B can be performed by an auxiliary device 201, 401, 403 as described above. As shown in Figure 5A, the actions include obtaining (step 501) proposed first activity parameters of a proposed first activity. The proposed first activity parameters comprise a proposed start time, a proposed first activity mode, and one or both of a proposed duration and a proposed ending time. These parameters can be received from, for example, a calendar request server 405. However, this is not an essential aspect of inventive embodiments. To the contrary alternative ways of conveying this information are also contemplated to be within the scope of inventive embodiments.

[0143] Accepted activity parameters of an accepted activity are determined (step 503) based on the proposed first activity parameters. After accepted activity parameters are settled, they are communicated to the XR device 201, 401, 403 (Step 505).

[0144] In embodiments in which the actions shown in Figure 5A are part of a negotiation with one or more other XR devices (e.g., as illustrated in Figure 4), the accepted activity parameters may represent a counterproposal, and as such they may additionally or instead be communicated back to one or more other XR devices 201, 401, 403 (e.g., via a calendar request server 405). In such instances, it may be advantageous to await confirmation that the counter proposed activity parameters have been accepted by others, or notification that yet another proposal (e.g., a counter- counterproposal) is received. Negotiations may go back and forth any number of times. But when negotiations are involved the accepted activity parameters are communicated to the XR device 201, 401, 403 after confirmation of acceptance is received.

[0145] Actions involved in determining accepted activity parameters based on the proposed first activity parameters are shown in greater detail in Figure 5B. These actions include predicting an initial XR device charge level that the XR device will have at the proposed start time (step 505). A determination is then made to assess whether the predicted initial XR device charge level will be sufficient to perform the proposed first activity using the proposed first activity parameters. Sufficiency can be determined by, for example determining whether the predicted initial XR device charge level is greater than or equal to an estimated amount of required charge. Those of ordinary skill in the art will recognize that this test can be represented by equivalent expressions using other relational operators, and all are considered to be within the scope of inventive embodiments.

[0146] When the predicted initial XR device charge level is not sufficient (e.g., not greater than or equal to an estimated required XR device charge level to perform the proposed first activity when using the proposed first activity parameters) (“No” path out of decision block 507), then the accepted activity parameters are produced by modifying (step 509) at least one of the proposed first activity parameters.

[0147] In this exemplary embodiment, modifying at least one of the proposed first activity parameters comprises modifying the proposed start time, modifying the proposed first activity mode, modifying the proposed duration and / or modifying the proposed ending time.

[0148] More particularly, when the proposed start time is modified, an accepted start time is excepted such that it is later than the proposed start time (step 511). Optionally, and as needed, an XR device charging activity is inserted (step 513) that begins earlier than the accepted start time (equivalently expressed as “modified start time”) and ends no later than the accepted / modified start time.

[0149] When the proposed first activity mode is modified (step 515), then an accepted / modified activity mode is selected that consumes power at a lower rate than a power consumption rate of the proposed first activity mode.

[0150] When the proposed duration is modified (step 517), then an accepted / modified duration that is shorter than the proposed duration is selected.

[0151] When the proposed ending time is modified, then an accepted / modified ending time that is sooner than the proposed ending time is selected (step 519). Optionally, and as needed, an XR device charging activity is inserted (step 521) that begins earlier after the accepted / modified ending time.

[0152] Exemplary modifications of proposed activity parameters are illustrated in Figure 6. As shown, a proposed activity 601 as a proposed start time 603 and a proposed ending time 605. When the proposed start time 603 is modified, it is replaced by an accepted / modified start time 607. When the proposed ending time 605 is modified, it is replaced by an accepted / modified ending time 609. The scheduling of an accepted activity 611 accordingly has the accepted / modified start and ending times 607, 609. It will be recognized that either of these modifications could shorten the duration of the accepted activity 611 when compared to that of the proposed activity 601.

[0153] In another aspect, and as described earlier, modification of proposed activity parameters can also include insertion of one or more charging activities, such as insertion of a charging activity 613 that begins earlier than the accepted / modified start time 607 and / or insertion of a charging activity 615 that begins after the accepted / modified ending time 609.

[0154] Further aspects of embodiments consistent with the invention will now be described with reference to Figure 7, which shows an exemplary controller 701 that may be included in a device to cause any and / or all of the herein-described and illustrated actions associated with that device to be performed. In particular, the controller 701 includes circuitry configured to carry out any one or any combination of the various functions described herein. Such circuitry could, for example, be entirely hard-wired circuitry (e.g., one or more Application Specific Integrated Circuits - “ASICs”). Depicted in the exemplary embodiment of Figure 7, however, is programmable circuitry, comprising a processor 703 coupled to one or more memory devices 705 (e.g., Random Access Memory, Magnetic Disc Drives, Optical Disk Drives, Read Only Memory, etc.) and to an interface 707 that enables bidirectional communication with other elements of a device as described above. A complete list of possible other elements is beyond the scope of this description.

[0155] The memory device(s) 705 store program means 709 (e.g., a set of processor instructions) configured to cause the processor 703 to control other device elements so as to carry out any of the aspects described herein. The memory device(s) 705 may also store data (not shown) representing various constant and variable parameters as may be needed by the processor 703 and / or as may be generated when carrying out its functions such as those specified by the program means 709.

[0156] Embodiments consistent with aspects of the invention provide a number of advantages over conventional technology. Such advantages include, without limitation:

[0157] • Longer duration of XR activities before having to re-charge the battery.

[0158] • Predictability and guarantees that planned activities will be possible despite limited battery capacity in the XR device 201, 401, 403.

[0159] • Better alignment of multiple participants using XR devices for scheduled events, taking both the specific event and other individually scheduled events into consideration.

[0160] • Reduced risk of having to experience “critical battery -level” power saving modes during an activity with a disruptive impact on user experience. • Coordinated re-charging breaks of multiple participants in single or multiple scheduled sessions.

[0161] • Lower latency and lower battery consumption to access data and files in AR activity sessions. The invention has been described with reference to particular embodiments. However, it will be readily apparent to those skilled in the art that it is possible to embody the invention in specific forms other than those of the embodiment described above. Thus, the described embodiments are merely illustrative and should not be considered restrictive in any way. The scope of the invention is further illustrated by the appended claims, rather than only by the preceding description, and all variations and equivalents which fall within the range of the claims are intended to be embraced therein.

Claims

-28-CLAIMS:

1. A method of controlling engagement in scheduled activities by an Extended Reality, XR, device (203), wherein the method is performed by an auxiliary device (103, 201, 401, 403), the method comprising: obtaining (407, 501) proposed first activity parameters of a proposed first activity, wherein the proposed first activity parameters comprise a proposed start time, a proposed first activity mode, and one or both of a proposed duration and a proposed ending time; determining (411, 503) accepted activity parameters of an accepted activity based on the proposed first activity parameters, wherein determining (411, 503) the accepted activity parameters comprises: predicting (505) an initial XR device charge level that the XR device will have at the proposed start time; determining (507) whether the predicted initial XR device charge level will be greater than or equal to an estimated amount of required charge to perform the proposed first activity using the proposed first activity parameters; when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then producing modified activity parameters by modifying (509) at least one of the proposed first activity parameters, wherein modifying at least one of the proposed first activity parameters comprises: when the proposed start time is modified, selecting (511) a modified start time that is later than the proposed start time; when the proposed first activity mode is modified, then selecting (515) a modified activity mode that consumes power at a lower rate than a power consumption rate of the proposed first activity mode; when the proposed duration is modified, then selecting (517) a modified duration that is shorter than the proposed duration; and when the proposed ending time is modified, then selecting (519) a modified ending time that is sooner than the proposed ending time; and communicating (411, 505) the accepted activity parameters to the XR device (203).

2. The method of claim 1, comprising:when the proposed start time is modified, scheduling an initial XR device charging activity that begins earlier than the modified start time.

3. The method of any one of the previous claims, wherein predicting the initial XR device charge level comprises: detecting a current level of battery charge of the XR device (203).

4. The method of claim 1, wherein predicting the initial XR device charge level comprises: estimating a level of battery charge that the XR device (203) will have at the end of a previously performed activity.

5. The method of any one of the previous claims, wherein obtaining (407, 501) the proposed first activity parameters of the proposed first activity comprises: obtaining (407, 501) the proposed first activity parameters from an activity calendar (405) of the XR device (203).

6. The method of claim 5, wherein at least a subset of the activity calendar is shared with one or more other XR devices (203).

7. The method of claim 5 or claim 6, wherein the activity calendar is stored in the auxiliary device (103, 201, 401, 403).

8. The method of any one of claims 5 through 7, wherein the activity calendar is stored in the XR device (203).

9. The method of claim 8 when dependent from claim 6, comprising: synchronizing the calendar stored in the auxiliary device (103, 201, 401, 403) with the calendar stored in the XR device (203).

10. The method of claim of any one of claims 1 through 4, wherein obtaining (407, 501) the proposed first activity parameters of the proposed first activity comprises: obtaining (407, 501) the proposed first activity parameters from a proposed first activity request received from another device.

11. The method of any one of the previous claims, comprising: estimating a level of battery charge that the XR device (203) will need to perform a next activity, wherein determining (411, 503) the accepted activity parameters comprises: estimating that a level of battery charge that the XR device (203) will have at the proposed ending time of the proposed first activity is less than the level of battery charge that the XR device (203) will need to perform the next activity, and in response: performing one or both of: selecting the modified duration that is shorter than the proposed duration; and selecting the modified ending time (609) that is sooner than the proposed ending time (605).

12. The method of claim 11, comprising: when the estimated level of battery charge that the XR device (203) will have at the end of the proposed first activity is less than the level of battery charge that the XR device (203) will need to perform the next activity, then scheduling an XR device charging activity (615) that begins no earlier than an accepted ending time (609).

13. The method of any one of the previous claims, wherein when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then determining (411, 503) the accepted activity parameters of the accepted activity comprises: splitting the proposed first activity (601) into at least first and second partial activities, wherein: the first partial activity has a first partial activity start time and a first partial activity end time; and the second partial activity has a second partial activity start time and a second partial activity end time; and scheduling an intervening charging activity to take place in between the first partial activity end time and the second partial activity start time.

14. The method of any one of the previous claims, comprising; charging the XR device (203) when a charging circuit in the XR device (203) is coupled to a charging circuit in the auxiliary device (103, 201, 401, 403).

15. The method of claim 14, comprising: determining content comprising one or more of an application program, data and files that will be used by the XR device (203) in one of the scheduled activities; and supplying the content to the XR device (203) while the charging circuit in the XR device (203) is coupled to the charging circuit (207) in the auxiliary device (103, 201, 401, 403).

16. The method of claim 15, comprising: prior to supplying the content to the XR device (203), downloading the content from a server and storing the content in the auxiliary device (103, 201, 401, 403).

17. The method of any one of claims 15 and 16, wherein determining content that will be used by the XR device (203) in one of the scheduled events comprises: sending a command to the XR device (203), wherein the command instructs the XR device to send information identifying the content to the auxiliary device (103, 201, 401, 403); and receiving the information identifying the content.

18. The method of any one of claims 15 through 17, wherein the content includes an avatar model of a user that that the XR device (203) is to display during said one of the scheduled activities.

19. The method of any one of claims 15 through 18, wherein the content includes a portion of video and / or audio information that lasts no longer than a duration that the XR device (203) is capable of playing without requiring a recharging of the XR device (203).

20. The method of claim 19, wherein the video and / or audio information is one part of a longer video and / or audio stream that is downloaded by the auxiliary device (103, 201, 401,-32-21. The method of any one of the previous claims, wherein determining the control schedule further comprises: inserting an XR device control parameter that causes the XR device (203) to operate at less than full power while performing the first activity.

22. The method of any one of the previous claims, wherein the accepted activity mode parameters include an accepted mode parameter that is selected from a set comprising an immersive interactive video conference, an immersive non-interactive video presentation, a two- dimensional video conference, an augmented reality application, and an audio-only application.

23. The method of any one of the previous claims, further comprising: predicting an initial XR device charge level that at least one other XR device (203) will have at the proposed start time; and producing at least one determination outcome by determining whether the predicted initial XR device charge level of said at least one other XR device (203) will be greater than or equal to an estimated amount of required charge for said at least one other XR device to perform the proposed first activity using the proposed first activity parameters, wherein determining (411, 503) the accepted activity parameters is further based on the at least one determination outcome.

24. The method of any one of the previous claims, comprising: prior to communicating the accepted activity parameters to the XR device (203), communicating, to at least one other entity, the accepted activity parameters as a set of acceptable activity parameters; and receiving a confirmation in response to the communication of the acceptable activity parameters to the at least one other entity, wherein communicating the accepted activity parameters to the XR device (203) is performed in response to the confirmation.

25. The method of claim 24, wherein the confirmation is a set of agreed on activity parameters and the method comprises: communicating, to the XR device (203), the agreed on activity parameters as the accepted activity parameters.-33-26. The method of claim 24, wherein the at least one other entity comprises another XR device (203).

27. The method of any one of claims 24 and 26, wherein the at least one other entity comprises a calendar request server (405).

28. The method of any one of claims 24 through 27, comprising: receiving second proposed first activity parameters of the proposed first activity in response to the communication of the acceptable activity parameters to the at least one other entity; and sending an acceptance of the second proposed first activity parameters or a second set of acceptable activity parameters to the at least one other entity.

29. A computer program (709) comprising instructions that, when executed by at least one processor (703), causes the at least one processor (703) to carry out the method according to any one of claims 1 through 28.

30. A carrier comprising the computer program (709) of claim 29, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a non-transitory computer readable storage medium (705).

31. An apparatus of an auxiliary device (103, 201, 401, 403) for controlling engagement in scheduled activities by an Extended Reality, XR, device (203), wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: obtaining (407, 501) proposed first activity parameters of a proposed first activity, wherein the proposed first activity parameters comprise a proposed start time, a proposed first activity mode, and one or both of a proposed duration and a proposed ending time; determining (411, 503) accepted activity parameters of an accepted activity based on the proposed first activity parameters, wherein determining (411, 503) the accepted activity parameters comprises: predicting (505) an initial XR device charge level that the XR device will have at the proposed start time;-34- determining (507) whether the predicted initial XR device charge level will be greater than or equal to an estimated amount of required charge to perform the proposed first activity using the proposed first activity parameters; when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then producing modified activity parameters by modifying (509) at least one of the proposed first activity parameters, wherein modifying at least one of the proposed first activity parameters comprises: when the proposed start time is modified, selecting (511) a modified start time that is later than the proposed start time; when the proposed first activity mode is modified, then selecting (515) a modified activity mode that consumes power at a lower rate than a power consumption rate of the proposed first activity mode; when the proposed duration is modified, then selecting (517) a modified duration that is shorter than the proposed duration; and when the proposed ending time is modified, then selecting (519) a modified ending time that is sooner than the proposed ending time; and communicating (411, 505) the accepted activity parameters to the XR device (203).

32. The apparatus of claim 31, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: when the proposed start time is modified, scheduling an initial XR device charging activity that begins earlier than the modified start time.

33. The apparatus of any one of claims 31 through 32, wherein predicting the initial XR device charge level comprises: detecting a current level of battery charge of the XR device (203).

34. The apparatus of claim 31, wherein predicting the initial XR device charge level comprises: estimating a level of battery charge that the XR device (203) will have at the end of a previously performed activity.-35-35. The apparatus of any one of claims 31 through 34, wherein obtaining (407, 501) the proposed first activity parameters of the proposed first activity comprises: obtaining (407, 501) the proposed first activity parameters from an activity calendar (405) of the XR device (203).

36. The apparatus of claim 35, wherein at least a subset of the activity calendar is shared with one or more other XR devices (203).

37. The apparatus of claim 35 or claim 36, wherein the activity calendar is stored in the auxiliary device (103, 201, 401, 403).

38. The apparatus of any one of claims 35 through 37, wherein the activity calendar is stored in the XR device (203).

39. The apparatus of claim 38 when dependent from claim 36, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: synchronizing the calendar stored in the auxiliary device (103, 201, 401, 403) with the calendar stored in the XR device (203).

40. The apparatus of claim of any one of claims 31 through 34, wherein obtaining (407, 501) the proposed first activity parameters of the proposed first activity comprises: obtaining (407, 501) the proposed first activity parameters from a proposed first activity request received from another device.

41. The apparatus of any one of claims 31 through 40, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: estimating a level of battery charge that the XR device (203) will need to perform a next activity, wherein determining (411, 503) the accepted activity parameters comprises: estimating that a level of battery charge that the XR device (203) will have at the proposed ending time of the proposed first activity is less than the level of battery charge that the XR device (203) will need to perform the next activity, and in response: performing one or both of:-36- selecting the modified duration that is shorter than the proposed duration; and selecting the modified ending time (609) that is sooner than the proposed ending time (605).

42. The apparatus of claim 41, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: when the estimated level of battery charge that the XR device (203) will have at the end of the proposed first activity is less than the level of battery charge that the XR device (203) will need to perform the next activity, then scheduling an XR device charging activity (615) that begins no earlier than an accepted ending time (609).

43. The apparatus of any one of claims 31 through 42, wherein when the predicted initial XR device charge level is not greater than or equal to an estimated required XR device charge level to perform the proposed first activity using the proposed first activity parameters, then determining (411, 503) the accepted activity parameters of the accepted activity comprises: splitting the proposed first activity (601) into at least first and second partial activities, wherein: the first partial activity has a first partial activity start time and a first partial activity end time; and the second partial activity has a second partial activity start time and a second partial activity end time; and scheduling an intervening charging activity to take place in between the first partial activity end time and the second partial activity start time.

44. The apparatus of any one of claims 31 through 43, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform; charging the XR device (203) when a charging circuit in the XR device (203) is coupled to a charging circuit in the auxiliary device (103, 201, 401, 403).

45. The apparatus of claim 44, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform:-37- determining content comprising one or more of an application program, data and files that will be used by the XR device (203) in one of the scheduled activities; and supplying the content to the XR device (203) while the charging circuit in the XR device (203) is coupled to the charging circuit (207) in the auxiliary device (103, 201, 401, 403).

46. The apparatus of claim 45, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: prior to supplying the content to the XR device (203), downloading the content from a server and storing the content in the auxiliary device (103, 201, 401, 403).

47. The apparatus of any one of claims 45 and 46, wherein determining content that will be used by the XR device (203) in one of the scheduled events comprises: sending a command to the XR device (203), wherein the command instructs the XR device to send information identifying the content to the auxiliary device (103, 201, 401, 403); and receiving the information identifying the content.

48. The apparatus of any one of claims 45 through 47, wherein the content includes an avatar model of a user that that the XR device (203) is to display during said one of the scheduled activities.

49. The apparatus of any one of claims 45 through 48, wherein the content includes a portion of video and / or audio information that lasts no longer than a duration that the XR device (203) is capable of playing without requiring a recharging of the XR device (203).

50. The apparatus of claim 49, wherein the video and / or audio information is one part of a longer video and / or audio stream that is downloaded by the auxiliary device (103, 201, 401, 403).

51. The apparatus of any one of claims 31 through 50, wherein determining the control schedule further comprises: inserting an XR device control parameter that causes the XR device (203) to operate at less than full power while performing the first activity.-38-52. The apparatus of any one claims 31 through 51, wherein the accepted activity mode parameters include an accepted mode parameter that is selected from a set comprising an immersive interactive video conference, an immersive non-interactive video presentation, a two- dimensional video conference, an augmented reality application, and an audio-only application.

53. The apparatus of any one of claims 31 through 52, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: predicting an initial XR device charge level that at least one other XR device (203) will have at the proposed start time; and producing at least one determination outcome by determining whether the predicted initial XR device charge level of said at least one other XR device (203) will be greater than or equal to an estimated amount of required charge for said at least one other XR device to perform the proposed first activity using the proposed first activity parameters, wherein determining (411, 503) the accepted activity parameters is further based on the at least one determination outcome.

54. The apparatus of any one of claims 31 through 53, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: prior to communicating the accepted activity parameters to the XR device (203), communicating, to at least one other entity, the accepted activity parameters as a set of acceptable activity parameters; and receiving a confirmation in response to the communication of the acceptable activity parameters to the at least one other entity, wherein communicating the accepted activity parameters to the XR device (203) is performed in response to the confirmation.

55. The apparatus of claim 54, wherein the confirmation is a set of agreed on activity parameters and wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: communicating, to the XR device (203), the agreed on activity parameters as the accepted activity parameters.-39-56. The apparatus of claim 54, wherein the at least one other entity comprises another XR device (203).

57. The apparatus of any one of claims 54 and 56, wherein the at least one other entity comprises a calendar request server (405).

58. The apparatus of any one of claims 54 through 57, wherein the apparatus is configured to cause the auxiliary device (103, 201, 401, 403) to perform: receiving second proposed first activity parameters of the proposed first activity in response to the communication of the acceptable activity parameters to the at least one other entity; and sending an acceptance of the second proposed first activity parameters or a second set of acceptable activity parameters to the at least one other entity.

Images