Extended reality device information handling

Abstract

There is provided a method for handling information associated with a user of an extended reality (XR) device. The method comprises generating (502) first information associated with an output for the XR device. The first information comprises at least one 5 first information element. The at least one first information element is associated with the user of the XR device or an environment of the user. The method comprises modifying (504) the first information to generate second information. The method comprises initiating (506) transmission of a first request towards a first entity of the network. The first request is a request for the first entity to provide, based on the second 10 information, third information for generating the output at the XR device. The method comprises receiving (508) the third information from the first entity. The third information comprises the at least one substitute element. The method comprises modifying (510) the third information to generate fourth information. 15 Figure 6 to accompany the Abstract.

Description

[0001] EXTENDED REALITY DEVICE INFORMATION HANDLING

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to methods for handling information associated with a user of an extended reality device, and devices configured to operate in accordance with those methods.

[0004] BACKGROUND

[0005] Several privacy concerns need to be considered when engaging in content generation over a network on a remote server. One primary issue is data transmission. As data travels over a network, it can be intercepted by malicious actors if not properly encrypted, potentially exposing sensitive information. Once the data reaches a remote server, data storage becomes another concern. The server may store the data indefinitely, and inadequate security measures could lead to unauthorized access to the stored data, and potential data leaks.

[0006] Data usage by service providers is also a significant concern. Service providers may use data obtained from a network for analytics or marketing. In some scenarios, service providers may even share obtained user data with third parties, often without the user's full knowledge or consent, resulting in potential privacy violations. As such, access controls on a remote server in which data is being stored are crucial. Without proper authentication and authorization mechanisms, there is a risk of unauthorized personnel, and / or external hackers, accessing sensitive data.

[0007] Moreover, user anonymity is at risk if content generation processes associated with a user of the network inadvertently log user data, especially when combined with other datasets that could identify individuals. Data breaches are another issue, as remote servers are attractive targets for cyberattacks, and any compromise could expose stored data, leading to privacy breaches. Legal and regulatory compliance adds another layer of complexity, as different jurisdictions and authorities have varying privacy laws, rules and regulations. Examples of this variation include the General Data Protection Regulation (GDPR) in Europe and the California Consumer Privacy Act (CCPA) in California. Service providers must navigate these regulations to ensure compliance, which can be challenging across multiple regions. Furthermore, user consent is vital. Network users may not be fully informed about what data is collected, how said data isused, and / or who the data is shared with. Obtaining informed consent is important for ethical data handling.

[0008] To address these concerns, service providers must implement strong encryption for data transmission and storage, enforce robust access controls, provide transparent privacy policies, and ensure legal compliance. Service providers should also seek informed consent from users. Users should be aware of the services they use, understand their privacy settings, and know their data privacy rights. An example of data security can be found in US 2016 / 0050341 A1, which discloses a method for processing a digital image so as to provide the image with a security feature.

[0009] The above-mentioned issues associated with privacy concerns are prevalent in the field of extended reality (XR). XR technology is configured to combine and / or mirror the physical world with a digital world, providing users of the technology with an immersive digital experience (e.g. by being in a virtual or augmented environment). As such, XR devices frequently capture data related to a user of the device, and / or data related to a surroundings of the user, in order to generate XR content. As a result, data used forXR content generation can often include private and / or sensitive data. Privacy concerns are amplified when such XR content is generated at a remote server (e.g. remote to the XR device). In such cases, data can be exposed through the wire, or through the air. Moreover, the data may be stored on the remote server which is being used for content generation. The communicated and stored data can include personal data and / or information about a user of the XR device. For example, XR content generation may include requesting the remote server to generate a three dimensional 3D model of the user, and / or an overlay containing the user’s personal information. Therefore, the use of remote servers in XR content generation can lead to compromises in the security of sensitive data, such as facial characteristics, skin color, and / or religious or sexual preferences of a user.

[0010] SUMMARY

[0011] As mentioned above, there are certain challenges associated with existing techniques for handling data related to XR technologies. Indeed, some existing techniques for XR content generation rely on the transmission of personal and / or sensitive user data to a remote location such that the XR content can be generated. However, in doing so, the risk of exposure of user data is increased. There is thus a desire to enhance securityand user privacy in scenarios in which external entities (e.g. remote servers) participate in XR content generation.

[0012] Therefore, according to an aspect of the disclosure, there is provided a method for handling information associated with a user of an extended reality (XR) device. The method is performed by an XR device of a network. The method comprises generating first information associated with an output for the XR device. The first information comprises at least one first information element. The at least one first information element is associated with the user of the XR device or an environment of the user. The method comprises modifying the first information to generate second information. Modifying the first information comprises replacing the at least one first information element with at least one substitute element. The method comprises initiating transmission of a first request towards a first entity of the network. The first request is a request for the first entity to provide, based on the second information, third information for generating the output at the XR device. The method comprises receiving the third information from the first entity. The third information comprises the at least one substitute element. The method comprises modifying the third information to generate fourth information. Modifying the third information comprises replacing the at least one substitute element with the at least one first information element.

[0013] According to another aspect of the disclosure, there is provided an XR device comprising processing circuitry configured to operate in accordance with the method referred to herein. In some examples, the XR device may comprise at least one memory for storing instructions which, when executed by the processing circuitry, cause the XR device to operate in accordance with the method referred to herein.

[0014] According to another aspect of the disclosure, there is provided a computer program comprising instructions which, when executed by processing circuitry, cause the processing circuitry to perform the method referred to herein.

[0015] According to another aspect of the disclosure, there is provided a computer program product, embodied on a non-transitory machine-readable medium, comprising instructions which are executable by processing circuitry to cause the processing circuitry to perform the method referred to herein.Thus, in the manner described above, improved techniques for handling information associated with a user of an XR device are provided. Advantageously, the techniques enhance the security of external content generation by modifying first information associated with an output for the XR device, in the manner described herein. The techniques enable sensitive information to be stripped out of a request for content generation (e.g. as comprised in the first information referred to herein) before the request is passed to a remote site (e.g. the first entity referred to herein) for content generation. At least one first information element, which is associated with a user of the XR device or an environment of the user, is replaced with at least one substitute element. As such, private information about the user can be anonymized and exposure of the information, to a remote server and / or application, is avoided. The techniques described herein enable the replaced information to be reintroduced to the output generated at the XR device, thereby maintaining performance while increasing security and privacy. As such, the techniques described herein enable the generation of custom content over a network without exposing sensitive information to the network.

[0016] BRIEF DESCRIPTION OF THE DRAWINGS

[0017] For a better understanding of the techniques, and to show how they may be put into effect, reference will now be made, by way of example, to the accompanying drawings, in which:

[0018] Figures 1 to 3 are illustrations of existing XR architectures;

[0019] Figure 4 is a signalling diagram illustrating an existing technique for handling information associated with an XR device;

[0020] Figure 5 is a block diagram illustrating an XR device according to an embodiment;

[0021] Figure 6 is a block diagram illustrating a method performed by the XR device according to an embodiment;

[0022] Figure 7 is a block diagram illustrating a system according to an embodiment; and

[0023] Figures 8 and 9 are signalling diagrams illustrating an exchange of signals in a system according to some embodiments.DETAILED DESCRIPTION

[0024] Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

[0025] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject-matter disclosed herein, the disclosed subject-matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject-matter to those skilled in the art.

[0026] The techniques described herein relate to XR devices. An XR device, as referred to herein, can comprise any type of XR device. For example, the XR device referred to herein may be a hardware device that enables (e.g. user) interaction with immersive technologies (e.g. that blend a physical world with a digital world). An XR device may comprise one or more of a virtual reality (VR) device, an augmented reality (AR) device, and a mixed reality (MR) device. Depending on its purpose, an XR device may be configured to provide fully virtual experiences (e.g. by enhancing real-world views with digital overlays), or may be configured to blend physical and digital elements interactively.

[0027] OpenXR is an open standard developed by the Khronos Group to streamline the development of XR (e.g. virtual reality (VR) and augmented reality (AR)) applications. OpenXR offers a unified interface that allows developers to write a single application capable of running across a diverse array of devices and / or platforms without needing tocustomize the application for each specific XR (e.g. VR and / or AR) system. One of the main advantages of OpenXR is its cross-platform compatibility. By abstracting underlying hardware details, it enables applications to function seamlessly across different XR systems, such as those from Oculus, HTC, Microsoft, and Valve, as well as Android phones. OpenXR is designed with a layered architecture that enables separation of an application interface from device drivers, providing flexibility and making it easier to integrate new hardware and software components.

[0028] OpenXR also enhances interoperability within XR ecosystems by providing a standard interface between an XR application and an XR runtime. This facilitates better integration between various components like input devices, tracking systems, and rendering engines. Overall, OpenXR aims to simplify the development process for immersive experiences and foster a more cohesive and accessible ecosystem for XR technologies. By reducing fragmentation and offering a common framework, OpenXR helps developers concentrate on creating engaging content rather than grappling with the complexities of supporting multiple platforms.

[0029] The standard is also built to be extensible, supporting extensions that allow hardware vendors and software developers to introduce new features and capabilities beyond core specification. This ensures that OpenXR can evolve alongside technological advancements. Extensibility is granted by OpenXR Layers, which are described below, together with OpenXR’s other components, such as Loader and Runtime components.

[0030] Figure 1 is a schematic illustration of an existing XR architecture. As illustrated in Figure 1 , the architecture can comprise a system comprising a plurality of modules and / or components. The system can be adapted to provide a framework for creating portable, high-performance XR applications (e.g. that can run on different hardware and / or software environments). At least some or all of the modules (e.g. components) of the system illustrated in Figure 1 may be standardised by OpenXR. As such, the system illustrated in Figure 1 may be said to be an example of an OpenXR system.

[0031] As illustrated in Figure 1 , the system may be adapted to utilise a plurality of modules and / or components. In some examples, as illustrated in Figure 1, the system can comprise an application module 100, a loader module 102 (“Loader Trampoline”), and a runtime module 112. The loader module 102 can be configured to manage interaction between applications and underlying (e.g. OpenXR) runtime(s). The loader module 102can be a dynamic loader module. For example, the loader module 102 can be configured to provide an abstraction layer. The abstraction layer can allow applications to interface with an appropriate (e.g. OpenXR) runtime (e.g. for a target platform). The loader module 102 can be configured to (e.g. help) simplify the development process by handling details of runtime selection and initialization, enabling applications to seamlessly work with different XR devices and platforms.

[0032] The runtime module 112 may be a software component. For example, the runtime module 112 may be a software component configured to (e.g. directly) interact with an underlying (e.g. XR device) hardware and / or manage the execution of an XR application. The runtime module 112 may be responsible for handling processes such as device enumeration, tracking, rendering, input handling, and / or other (e.g. low-level) operations required for delivering immersive experiences to XR device users. The runtime module 112 may be referred to as a OpenXR implementation module. Runtime modules (e.g. OpenXR runtimes) are typically provided by hardware vendors and / or platform providers. The runtime module 112 may be used to enable interoperability and / or performance of (e.g. XR) applications across different devices and / or platforms.

[0033] As illustrated in Figure 1, the system can comprise one or more optional modules 106, 108, 110, such as a loader terminator module 110. As also illustrated in Figure 1 , the one or more optional modules 106, 108, 110 can comprise one or more layer modules 106, 108. A layer module may be referred to herein as a layer, and vice-versa. OpenXR layers are a component of the OpenXR standard. Layers can, for example, be software components. In some examples, a layer may be, and / or may comprise, an API layer. In some cases, a layer can be configured to intercept and / or modify the behavior of the runtime module 112. As such, a layer can allow for various forms of customization and / or enhancement of an application. In OpenXR, layers can be used to implement features such as debugging, performance profiling, input device emulation, and other forms of runtime modification. By enabling developers to add custom functionality to OpenXR runtime, layers provide a flexible and extensible framework for building XR applications. As illustrated in Figure 1, the one or more layer modules 106, 108 may comprise two layer modules. However, it will be understood that this is merely an example, and that the one or more layer modules 106, 108 can comprise any number (e.g. three, four, five, etc.) of layer modules. As described herein, the one or more layer modules 106, 108 are optional. As such, it will be understood that the system of Figure 1 may comprise no layer modules according to some examples.Through implementation of components such as those illustrated in Figure 1, OpenXR is able to provide a comprehensive framework for creating portable, high-performance XR applications that can run on diverse hardware and software environments. The standardization of these components helps to streamline development efforts and drive innovation in the field of XR.

[0034] Figure 2 is a block diagram illustrating an existing XR architecture. The architecture illustrated in Figure 2 may be said to be an example of an OpenXR architecture. The architecture illustrated in Figure 2 is an example of a monolith structure (e.g. implementation). As such, the architecture illustrated in Figure 2 does not comprise any distributed elements. For example, the architecture illustrated in Figure 2 may be comprised in a (e.g. single) XR device.

[0035] In the example architecture illustrated in Figure 2, XR device hardware (e.g. camera, sensors, control, display, etc.) is handled by an XR runtime module. As illustrated in Figure 2, an OpenXR module interfaces an application (“App”) engine with the XR runtime module, such that the same application could potentially be run in different XR devices (e.g. from different manufacturers).

[0036] Figure 3 is a block diagram illustrating an existing XR architecture. The architecture illustrated in Figure 2 may be said to be an example of an OpenXR architecture. In contrast to the monolithic architecture illustrated in Figure 2 above, Figure 3 illustrates a cloud-supported architecture (e.g. implementation). Such a cloud-supported architecture may be adopted by future releases of OpenXR. The architecture illustrated in Figure 3 targets the addition of a support of cellular network components. Moreover, the architecture illustrated in Figure 3 allows for the distribution of components between device and the cloud (e.g. a cloud entity). For instance, the device can host an XR runtime module that may interact with an OpenXR layer though a cloud XR runtime module, without disclosing the proprietary implementation on the device. Both the device and cloud XR runtime module can exchange information through a cellular network via streaming APIs and / or cellular network drivers.

[0037] The architecture illustrated in Figure 3 may be referred to as a split XR runtime (e.g. between a device and a cloud entity). The distribution of components between device and cloud, especially the device XR runtime, creates issues regarding security andprivacy of sensitive user data. To better illustrate the current challenges associated with handling information associated with a user of an XR device, reference will now be made to an existing technique

[0038] Figure 4 is a signalling diagram illustrating an exchange of signals in an environment according to an existing technique. As illustrated in Figure 4, the environment comprises a device 402 and a remote server 404. The device 402 comprises a first XR runtime. The first XR runtime can be for executing a program to generate XR content. The remote server 404 comprises a second XR runtime. The second XR runtime can be for generating the instructions (e.g. code) used to generate XR content (e.g. at the XR device). As illustrated in Figure 4, the remote server 404 can be a cloud server.

[0039] As illustrated by arrow 406 of Figure 4, the device 402 transmits a request to the remote server 404 for generation of content. Although not explicitly illustrated in Figure 4, the request comprises private information related to a user of the device 402. As such, the private information is exposed via communication to, and handling by, the remote server 404. The remote server 404 uses the data in the request to generate content to be delivered at the device. As illustrated by arrow 408 of Figure 4, the content (e.g. instructions to generate the content) is transmitted to the device 402. Thus, the device 402 receives the content. The transmission as described with reference to arrow 408 represents yet another opportunity for security breach and the potential exposure of private user information. Currently, as described with reference to Figure 4, the device 402 and the remote server 404 exchange sensitive information during content request and response (e.g. in order to generate an output at the device 402, such as a visual overlay with the name of a person and their credentials).

[0040] The architecture as described with reference to Figures 3 and 4 poses challenges associated with privacy aspects when content is generated at a remote server. It is desirable to provide a technique which preserves user privacy (e.g. of sensitive data) while still enabling the generation of content at a remote server of a network.

[0041] Figure 5 illustrates a XR device 10 of a network in accordance with an embodiment. The XR device 10 is for handling information associated with a user of the XR device 10. In some embodiments, the XR device 10 referred to herein can refer to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with the first entity referred to herein, and / or with other nodes or equipment to enable and / or toperform the functionality described herein. In some embodiments, the XR device 10 referred to herein can, for example, be a physical node (e.g. a physical machine or server) or a virtual node (e.g. a virtual machine, VM).

[0042] As illustrated in Figure 5, the XR device 10 comprises processing circuitry (or logic) 12. The processing circuitry 12 controls the operation of the XR device 10 and can implement the method described herein in respect of the XR device 10. The processing circuitry 12 can be configured or programmed to control the XR device 10 in the manner described herein. The processing circuitry 12 can comprise one or more hardware components, such as one or more processors, one or more processing units, one or more multi-core processors and / or one or more modules. In particular implementations, each of the one or more hardware components can be configured to perform, or is for performing, individual or multiple steps of the method described herein in respect of the XR device 10. In some embodiments, the processing circuitry 12 can be configured to run software to perform the method described herein in respect of the XR device 10. The software may be containerised according to some embodiments. Thus, in some embodiments, the processing circuitry 12 may be configured to run a container to perform the method described herein in respect of the XR device 10.

[0043] Briefly, the processing circuitry 12 of the XR device 10 is configured to generate first information associated with an output for the XR device. The first information comprises at least one first information element. The at least one first information element is associated with the user of the XR device or an environment of the user. The processing circuitry 12 of the XR device 10 is configured to modify the first information to generate second information. Modifying the first information comprises replacing the at least one first information element with at least one substitute element. The processing circuitry 12 of the XR device 10 is configured to initiate transmission of a first request towards a first entity of the network. The first request is a request for the first entity to provide, based on the second information, third information for generating the output at the XR device. The processing circuitry 12 of the XR device 10 is configured to receive the third information from the first entity. The third information comprises the at least one substitute element. The processing circuitry 12 of the XR device 10 is configured to modify the third information to generate fourth information. Modifying the third information comprises replacing the at least one substitute element with the at least one first information element.As illustrated in Figure 5, in some embodiments, the XR device 10 may optionally comprise a memory 14. The memory 14 of the XR device 10 can comprise a volatile memory or a non-volatile memory. In some embodiments, the memory 14 of the XR device 10 may comprise a non-transitory media. Examples of the memory 14 of the XR device 10 include, but are not limited to, a random access memory (RAM), a read only memory (ROM), a mass storage media such as a hard disk, a removable storage media such as a compact disk (CD) or a digital versatile disk (DVD), and / or any other memory.

[0044] The processing circuitry 12 of the XR device 10 can be communicatively coupled (e.g. connected) to the memory 14 of the XR device 10. In some embodiments, the memory 14 of the XR device 10 may be for storing program code or instructions which, when executed by the processing circuitry 12 of the XR device 10, cause the XR device 10 to operate in the manner described herein in respect of the XR device 10. For example, in some embodiments, the memory 14 of the XR device 10 may be configured to store program code or instructions that can be executed by the processing circuitry 12 of the XR device 10 to cause the XR device 10 to operate in accordance with the method described herein in respect of the XR device 10. Alternatively or in addition, the memory 14 of the XR device 10 can be configured to store any information, data, messages, requests, responses, indications, notifications, signals, or similar, that are described herein. The processing circuitry 12 of the XR device 10 may be configured to control the memory 14 of the XR device 10 to store any of the information, data, messages, requests, responses, indications, notifications, signals, or similar, that are described herein.

[0045] In some embodiments, as illustrated in Figure 5, the XR device 10 may optionally comprise a communications interface 16. The communications interface 16 of the XR device 10 can be communicatively coupled (e.g. connected) to the processing circuitry 12 of the XR device 10 and / or the memory 14 of the XR device 10. The communications interface 16 of the XR device 10 may be operable to allow the processing circuitry 12 of the XR device 10 to communicate with the memory 14 of the XR device 10 and / or vice versa. Similarly, the communications interface 16 of the XR device 10 may be operable to allow the processing circuitry 12 of the XR device 10 to communicate with any one or more nodes (e.g. first entity) referred to herein and / or any other node. The communications interface 16 of the XR device 10 can be configured to transmit and / or receive any of the information, data, messages, requests, responses, indications, notifications, signals, or similar, that are described herein. In some embodiments, theprocessing circuitry 12 of the XR device 10 may be configured to control the communications interface 16 of the XR device 10 to transmit and / or receive any of the information, data, messages, requests, responses, indications, notifications, signals, or similar, that are described herein.

[0046] Although theXR device 10 is illustrated in Figure 5 as comprising a single memory 14, it will be appreciated that the XR device 10 may comprise at least one memory (i.e. a single memory or a plurality of memories) 14 that operate in the manner described herein. Similarly, although the XR device 10 is illustrated in Figure 5 as comprising a single communications interface 16, it will be appreciated that the XR device 10 may comprise at least one communications interface (i.e. a single communications interface or a plurality of communications interfaces) 16 that operate in the manner described herein. It will also be appreciated that Figure 5 only shows the components required to illustrate an embodiment of the XR device 10 and, in practical implementations, the XR device 10 may comprise additional or alternative components to those shown.

[0047] Figure 6 is a block diagram illustrating a method performed according to an embodiment. The method is for handling information associated with a user of an XR device 10. The XR device 10 described earlier with reference to Figure 5 can be configured to operate in accordance with the method of Figure 6. The method can be performed by or under the control of the processing circuitry 12 of the XR device 10 according to some examples. The XR device 10 is an XR device of a network.

[0048] The network referred to herein can be any type of network. For example, the network referred to herein may be a communications network and / or a telecommunications network. In some examples, the network can be a mobile network, such as a fifth generation (5G) mobile network or any other generation mobile network (e.g. 6G). For example, the network can be a 5G core (5GC) network. In some examples, the network can be a radio access network (RAN). Although some examples have been provided for the type of network referred to herein, it will be understood that the network referred to herein can be any other type of network.

[0049] With reference to Figure 6, as illustrated at block 502, first information is generated. The XR device 10 (e.g. the processing circuitry 12 of the XR device 10) can be configured to generate the first information. The first information is associated with an output for the XR device. For example, the first information can comprise information to be used togenerate a (e.g. visual) output at the XR device. The first information comprises at least one first information element. The at least one first information element is associated with the user of the XR device 10 or an environment of the user.

[0050] The at least one first information element may be associated with private information of the user. For example, the at least one first information element may be information that should be protected to prevent identity theft and / or fraud. The at least one information element may correspond to data that is not meant to be shared publicly. For example, the at least one first information element may be data that is sensitive, personal and / or confidential. In some examples, the at least one first information element may be associated with personal information of the user of the XR device. Personal information of the user may be information that can be used to identify the user (e.g. directly or indirectly). Alternatively, or in addition, in some examples the at least one first information element can be associated with sensitive user information. Sensitive user information may comprise any data that, if disclosed, could cause harm, identity theft, financial loss, discrimination, and / or a security risk to the user.

[0051] In some examples, the at least one first information element can be indicative of a characteristic of the user. For example, the at least one first information element may be indicative of one or more of a name of the user, a birth date of the user, a facial characteristic of the user, a hair colour of the user, a hair length of the user, a height of the user, a weight of the user, a gender of the user, a sex of the user, a skin colour of the user, a religious orientation of the user, and a sexual orientation of the user. The facial characteristic of the user may comprise one or more of facial hair and eye colour. In a specific example, the at least one first information element may be indicative that the user is male, has brown eyes, and has facial hair (e.g. a beard). In some examples, the at least one first information element may comprise information associated with a (e.g. geographical) location of the user. In some examples, the at least one first information element may comprise a plurality of first information elements.

[0052] As described herein, the at least one first information element can be associated with the user of the XR device 10 or an environment of the user. The environment of the user can comprise a surroundings of the user (e.g. during the time interval in which the user is using the XR device 10). The at least one first information element may be associated with one or more people and / or one or more objects in the environment of the user (e.g. around the user). In some examples, the at least one first information element may becomprised in, and / or derived from, information obtained by one or more sensors of the XR device 10. For example, the at least one first information element may be associated with an image indicating a license plate, a user address, and / or facial features of a person in the vicinity of the user of the XR device 10. In some examples, the at least one first information element may be associated with sensitive objects, such as personal objects, and / or identification (ID) cards. In some examples, the first information, as defined herein, can be derived from information obtained and / or measured by one or more sensors of the XR device 10. The one or more sensors can include a visual sensor (e.g. a camera), an auditory sensor (e.g. a microphone), and / or a haptic sensor (e.g. a touch pad and / or joystick).

[0053] Although not illustrated in Figure 6, in some examples, the method may comprise identifying the at least one first information element, as defined herein. As described herein, identifying the at least one first information element may comprise using a machine learning (ML) model to identify the at least one first information element. In some examples, identifying the at least one first information element can comprise analysing the first information using the ML model. For example, the first information can be input to the ML model for identification of the at least one first information element. The ML model referred to herein may be a trained ML model. For the purposes of the present disclosure, the term “ML model” can encompass, within its scope, an ML algorithm, comprising processes or instructions through which data may be used in a training process to generate a model artefact for performing a given task, or for representing a real world process or system. Alternatively, or in addition, the term “ML model” may encompass the model artefact that is created by such a training process, and which comprises the computational architecture that performs the task. Alternatively, or in addition, the term “ML model” may encompass the process performed by the model artefact in order to complete the task. References to “ML model”, “model”, model parameters”, “model information”, etc., may thus be understood as relating to any one or more of the above concepts encompassed within the scope of “ML model”. The ML model may be referred to herein as a “ML agent”.

[0054] As illustrated at block 504 of Figure 6, the first information is modified to generate second information. The XR device 10 (e.g. the processing circuitry 12 of theXR device 10) can be configured to modify the first information. Modifying the first information comprises replacing the at least one first information element with at least one substitute element.In some examples, the at least one substitute element may be generated based on the at least one first information element. The at least one substitute element can be an information element (e.g. a string, visual data, and / or a segment of code). Although not illustrated in Figure 6, in some examples, the method may comprise generating the at least one substitute element based on the at least one first information element. In these examples, each substitute element of the at least one substitute element may correspond (e.g. uniquely) to a first information element of the at least one first information element. In some examples, the XR device may generate the substitute element based on the first information element according to a (e.g. stored) protocol and / or schema. In some examples, the at least one substitute element may not be associated with the user or the environment of the user. For example, the at least one substitute element may comprise information that is not indicative of the information comprised in the at least one first information element. The replacement of the at least one first information element with the at least one substitute element can be done to anonymize the first information. The at least one substitute element may be a placeholder for the at least one first information element. In some examples, replacing the at least one first information element with the at least one substitute element can include removing sensitive information from the first information. In some examples, the at least one substitute element may be a null value element and / or a zero value element. A null value element may be an element in a dataset, database, or structure that holds no value and / or is missing data. A null value may represent the absence of information. As such, in some examples, the at least one substitute element can represent an absence of an information element in the second information, as defined herein. In an example, the at least one first information element may correspond to an information element of a data field (e.g. NameField) in the first information. In such an example, replacing the at least one first information element with the at least one substitute element may comprise replacing the information element of the data field with an empty or missing element (i.e. null element). A null element is a commonly used data element in many coding languages.

[0055] In some examples, the at least one first information element can comprise a first string. In these examples, replacing the at least one first information element may comprise replacing the first string with a second string. The first string and the second string can be different. In some examples, the at least one first information element may be in a graphics library transmission format (gITF). Alternatively or in addition, in some examples the at least one substitute element may be in a gITF. gITF is a standard file format for 3D scenes and models. Although not illustrated in Figure 6, in some examples,the method may comprise storing a mapping indicative of an association between the at least one first information element and the at least one substitute element. In some examples, the XR device 10 may store the mapping in the memory 14 of the XR device 10.

[0056] As illustrated at block 506 of Figure 6, transmission of a first request is initiated towards a first entity of the network. The XR device 10 (e.g. the processing circuitry 12 of the XR device 10) can be configured to initiate transmission of the first request (e.g. via the communication interface 16 of the XR device 10). Herein, the term “initiate” can mean, for example, cause or establish. Thus, the XR device 10 (e.g. the processing circuitry 12 of the XR device 10) can be configured to itself transmit the first request (e.g. via the communications interface 16 of theXR device 10) or can be configured to cause another entity to transmit the first request. The first request is a request for the first entity to provide, based on the second information, third information for generating the output at the XR device 10. The first request can be referred to herein as a content generation request. The first request may be, and / or may be comprised in, a hypertext transfer protocol (HTTP) request (e.g. message). The third information may comprise instructions which, when executed, cause the XR device 10 (e.g. the processing circuitry 12 of the XR device 10) to generate the output at the XR device 10. Thus, in some examples, the first request for the first entity to provide third information may be a request for the first entity to generate and provide the instructions to be executed by the XR device 10 to generate the output (e.g. based on the original first information).

[0057] The output, as referred to herein, may be an XR output and / or XR content. For example, the output may, at least partially, comprise an XR (e.g. AR, MR, and / or VR) type output for the user of the XR device 10. The output may comprise one or more of a visual output, a haptic output, and an audio output. In some examples, the output may comprise a two dimensional (2D) display, and / or a three dimensional (3D) display. For example, the first request may be for the first entity to provide third information which can be used by the XR device 10 to generate a 2D display and / or a 3D display at the XR device 10. In some examples, the first request can be a request for a (e.g. XR) runtime system of the first entity to provide the third information. In some examples, the first entity can be a remote server of the network. Alternatively, or in addition, the first entity may be a cloud entity of the network. For example, the first entity may be a cloud entity as described with reference to Figure 3 above.As illustrated at block 508 of Figure 6, the third information is received from the first entity. The XR device 10 (e.g. the processing circuitry 12 of the XR device 10) can be configured to receive the third information (e.g. via the communication interface 16 of the XR device 10). The third information comprises the at least one substitute element, as defined herein. As illustrated at block 510 of Figure 6, the third information is modified to generate fourth information. The XR device 10 (e.g. the processing circuitry 12 of the XR device 10) can be configured to modify the third information. Modifying the third information comprises replacing the at least one substitute element with the at least one first information element. Although not illustrated in Figure 6, in some examples, the method may comprise generating the output based on the fourth information. For example, the XR device 10 can generate the output based on the fourth information (i.e. modified third information). In some examples, modifying the third information may comprise modifying the third information based on the (e.g. stored) mapping, as defined herein. For example, the mapping may be used to replace the at least one substitute element with a corresponding at least one first information element.

[0058] In some examples, the XR device 10 can be one or more of an augmented reality (AR) device, a virtual reality (VR) device, and a mixed reality (MR) device. The XR device 10 can comprise one or more of a user equipment (UE), a wearable device, a headset, glasses (e.g. smart glasses), a television, a smartphone, and a tablet. In some examples, the XR device 10 may comprise a 3D display, such as a holographic display.

[0059] Thus, some of the techniques described herein enable the removal of sensitive information from information that is needed to request custom content for XR output generation. This is especially advantageous in scenarios in which information to be used to generate an output comprises sensitive user information. For example, an output to be generated by an XR device may be associated with the display of an avatar of the user of the XR device. Generating the information necessary to display such content may require user information such as facial characteristics and skin color. When the generation of such content occurs over the network (e.g. at a remote server) the techniques described herein increase information security by sanitizing sensitive information before providing the content related information to external entities (e.g. the first entity referred to herein).

[0060] As an example, in scenarios in which the first information comprises textual information, the techniques described herein can detect sensitive information in the text (e.g. to besent to a remote server) and replace it with placeholder information and / or placeholder data. When the third information (e.g. generated content), as defined herein, is received by the XR device 10, the placeholders can be swapped with the original sensitive information.

[0061] As another example, in scenarios in which the first information comprises 3D model information, the techniques described herein can removes sensitive information (e.g. skin color of an avatar) from the body of the request (e.g. remove at least one first information element from the first information by modifying the first information in the manner described herein). When the third information (e.g. generated content), as defined herein, is received by the XR device 10, the sensitive information can be added to the 3D model. Such a 3D model may, for example, be associated with a gITF file.

[0062] Figure 7 is a block diagram illustrating a system according to an embodiment. As illustrated in Figure 7, the system can comprise an XR device 10, as defined herein, and a first entity 608, as defined herein. The XR device 10 and the first entity 608 can be connected to (e.g. be part of) a network as defined herein.

[0063] As illustrated in Figure 7, in some examples, theXR device 10 can comprise one or more modules 602, 604, 606. The one or more modules 602, 604, 606 can comprise a first runtime module 602, a privacy module 604, and / or a detector module 606. The example illustrated in Figure 7 shows the XR device 10 comprising three modules. However, it will be understood that this is merely an example and that the XR device 10 may comprise any number of one or more modules. For example, the functionality of each of the modules illustrated in Figure 7 may be performed by a single module according to some examples. As also illustrated in Figure 7, the first entity 608 may comprise a second runtime module 610. The first entity 608 may be a server of the network. For example, the first entity 608 may be a server that is external to the XR device 10 (e.g. a remote server). One or more of the first runtime module 602 and the second runtime module 610 may be an XR runtime module.

[0064] In some examples, the privacy module 604 may be configured to operate as an intermediary module between the first runtime module 602 and (e.g. the second runtime module 610 of) the first entity 608. In some examples, the privacy module 604 may be an application programming interface (API) module. The first runtime module 602 may communicate with the second runtime module 610 via the privacy module 604. Forexample, the privacy module 604 may intercept communication(s) between the first runtime module 602 and (e.g. the second runtime module 610 of) the first entity 608. The first runtime module 602 may generate the first information, as defined herein. In some examples, the first runtime module 602 may request content generation (e.g. generation of the output as defined herein) at the first entity 608 via the second runtime module 610. Although not illustrated in Figure 7, in some examples, (e.g. the second runtime module 610 of) the first entity 608 may forward requests from the XR device 10 (e.g. the first request referred to herein) to other entities and / or systems. For example, the first entity 608 may forward request information to an external system.

[0065] The first runtime module 602 can be configured to generate the first information, as described herein. For example, the first runtime module 602 can be configured to perform the step as described with reference to block 502 of Figure 6. One or more of the privacy module 604 and the detector module 606 can be configured to modify the first information, as described herein. For example, one or more of the privacy module 604 and the detector module 606 can be configured to perform the step as described with reference to block 504 of Figure 6. Thus, in some examples, the privacy module 604 can intercept sensitive information comprised in the first information before transmission (e.g. through the network).

[0066] As illustrated in Figure 7, information associated with the user, or an environment of the user, (e.g. sensitive information) can be kept within the XR device 10. As described herein, the first information is modified to generate second information by replacing the at least one first information element, as defined herein, with at least one substitute element. In this way, only non-sensitive (e.g. anonymized) information may be sent through the network (e.g. to the first entity 608). In some examples, the first information may be comprised in, and / or comprise, a second request, as described herein. The second request can be a request for the provision of information for generating an output based on the first information. As such, the second request can be referred to as a content generation request. The second request may be, and / or may be comprised in, a hypertext transfer protocol (HTTP) request (e.g. message). The privacy module 604 can forward the content generation request from the first runtime module 602 to the detector module 606. The detector module 606 can be configured to detect the at least one first information element (e.g. sensitive information) and replace it with the at least one substitute element (e.g. placeholder and / or anonymized data). Therefore, in some examples, the detector module 606 can be configured to identify the at least one firstinformation element, as described herein. The identification of the at least one first information element can be performed using an ML model and / or an algorithm. For example, the identification of the at least one first information element may be performed using one or more of a large language module (LLM), a neural network, and / or an algorithm-based procedure.

[0067] In some examples, if the first information includes the name of a person (e.g. the user), modifying the first information can comprise replacing the name with an alphanumeric string, such as “#1ft@”. The privacy module 604 can generate and / or store a mapping indicative of an association between the at least one first information element and the at least one substitute element, as defined herein. In this way, the privacy module 604 can keep track of mappings between sensitive information (e.g. of the first information and / or the fourth information) and anonymized information (e.g. of the second information and / or the third information). When (e.g. the second runtime module 610 of) the first entity 608 provides the third information (e.g. requested content), as defined herein, the privacy module 604 may replace (e.g. swap) the at least one substitute element (e.g. anonymized information) with the (e.g. corresponding) at least one first information element (e.g. sensitive information). In some examples, the third information may be in the form of a string and / or a 3D model (e.g. in a gITF file format).

[0068] The privacy module 604 may be implemented as a standalone library or as a (e.g. OpenXR) layer. An OpenXR layer based solution is convenient as it enables the automatic interception of requests.

[0069] Figure 8 is a signaling diagram illustrating an exchange of signal in a system according to an embodiment. The system of Figure 8 comprises an XR device 10, as described herein, and a first entity 608, as described herein. As illustrated in Figure 8, the system can comprise a privacy module 604. It will be understood that the privacy module 604 can be comprised in the XR device 10 according to some examples. For example, the functionality of the privacy module 604 can be performed by the (e.g. processing circuitry 12 of the) XR device 10.

[0070] As illustrated by arrow 708 of Figure 8, the XR device 10 generates the first information, as defined herein. In some examples, the first information may be comprised in a second request. In some examples, the first information may comprise the second request. The second request can be a request to generate an output (e.g. content) based on the firstinformation. The second request may be a request to generate text-based information. For example, the second request may be a request for generation of an overlay that includes one or more names and / or other sensitive information (e.g. associated with the user). As illustrated by arrow 710 of Figure 8, the privacy module 604 can modify the first information to generate second information, as defined herein. In some examples, modifying the first information may comprise anonymising the first information. For example, the second information may correspond to an anonymised (e.g. sanitised) version of the first information.

[0071] As illustrated by arrow 712 of Figure 8, the privacy module 604 can initiate transmission of a first request, as defined herein, towards the first entity 608. Thus, the first entity 608 can receive the first request. The first entity 608 can be an entity configured to generate information (e.g. instructions) which can be used by the XR device 10 to generate a desired output. As illustrated by arrow 714, the first entity 608 can initiate transmission of third information, as defined herein, towards the privacy module 604 (e.g. of the XR device 10). Thus, the privacy module 604 can receive the third information from the first entity 608. The third information can comprise instructions for generating the output, as defined herein, at the XR device 10. The third information comprises the at least one substitute element as defined herein. In examples in which the output is associated with an overlay, as mentioned herein, the third information may comprise information associated with a 3D model of the overlay to be used for rendering the output.

[0072] As illustrated by arrow 716 of Figure 8, the privacy module 604 can modify the third information to generate fourth information, as defined herein. For example, the privacy module 604 can replace the at least one substitute element in the third information with the (e.g. corresponding) at least one first information element. In this way, the privacy module 604 can reintroduce sensitive information into the third information to generate the fourth information. The third information may be understood to correspond to an anonymised (e.g. sanitised) version of the fourth information. As illustrated by arrow 718 of Figure 8, the privacy module 604 can provide the fourth information to the (e.g. processing circuitry 12 of the) XR device 10. The fourth information can comprise instructions to be executed to generate the output, as described herein, at the XR device 10. The fourth information can be provided in response to the generation of the first information (e.g. including the second request), as defined herein. The XR device 10 may generate (e.g. display) the output based on the fourth information. For example, the XR device 10 may execute the fourth information in order to generate the output.Thus, in the manner described with reference to Figure 8, the user of the XR device 10 can avoid sharing personal and / or sensitive information about themselves, and / or other people, with the first entity 608 (e.g. which may be an entity of a service provider).

[0073] The techniques described herein can be implemented, at least partially, in an OpenXR runtime. The techniques described herein may be implemented as a local implementation and / or a split rendering implementation. The local implementation may be referred to herein as a monolithic (e.g. OpenXR) architecture implementation. The split rendering implementation may be referred to herein as a cloud-assisted (e.g. OpenXR) architecture implementation.

[0074] In the monolithic implementation, when the (e.g. application running on the) XR device 10 requests content generation from an external provider, a (e.g. OpenXR) Layer may intercept such a request and trigger the identification (e.g. detection) and / or removal of at least one first information element (e.g. sensitive information) within the request. The detection of the at least one first information element (e.g. sensitive information) may be performed with one or more of the methodologies described herein. When the third information (e.g. content) is generated and returned to the device, the privacy module 604 may modify the third information to generate the fourth information, as defined herein. For example, the privacy module 604 may insert, into the third information (e.g. generated content), the sensitive user information.

[0075] In the cloud-assisted architecture implementation, when the (e.g. application running on the) XR device 10 requests content generation from an external provider, a device (e.g. OpenXR) Layer may intercept the request and trigger the identification (e.g. detection) and / or removal of at least one first information element (e.g. sensitive information) within the request. The detection of such sensitive information may be performed with one or more of the methodologies described herein. A cloud-based (e.g. OpenXR) runtime can receive the anonymized content generation request and forwards it to a content generation provider. Once the content is received, the cloud-based (e.g. OpenXR) runtime can render the content and send rendered content (e.g. a rendered frame) to the XR device 10.

[0076] Figure 9 is a signalling diagram illustrating an exchange of signals in a system according to an embodiment. The system illustrated in Figure 9 can be as described with referenceto Figure 9. The functionality of the system will now be described with reference to some exemplary use cases.

[0077] In some examples, the XR device 10 may comprise a light sensor, such as a camera. The light sensor may be configured to obtain visual data (e.g. of the environment of the user and / or the XR device 10). As illustrated by arrow 808 of Figure 9, the XR device 10 may perform facial recognition of one or more faces based on obtained visual data. For example, the XR device 10 can perform facial recognition on faces detected in a camera frame.

[0078] The method step illustrated by arrow 810 of Figure 9 can be as described with reference to arrow 708 of Figure 8. In some examples, the first information may comprise information indicative of the visual data and / or the facial recognition. For example, the first information may comprise information indicative of one or more names (e.g. of detected persons), and / orother sensitive information (e.g. age and / or gender of detected persons). The first information can comprise a second request, as defined herein. As illustrated in Figure 9, in some examples, the second request can be a request to generate an overlay with the one or more names and / or other sensitive information comprised in the first information.

[0079] The method step illustrated by arrow 812 of Figure 9 can be as described with reference to arrow 710 of Figure 8. In some examples, the privacy module 604 may intercept the first information (e.g. comprising the second request). The privacy module 604 can modify the first information, as described herein. For example, the privacy module 604 can anonymize the first information by replacing sensitive information with mock information. As an example, in scenarios in which the first information includes one or more names, the name (e.g. string) “Mario Rossi” may be replaced with (e.g. the string) “#1@ft #2%ra”. In this example, “Mario Rossi” may be considered to correspond to at least one first information element, as defined herein, and “#1@ft #2%ra” may be considered to correspond to at least one (e.g. corresponding) substitute element, as defined herein. A mapping between the at least one first information element (e.g. sensitive information) and the at least one substitute element (e.g. mock information) can be securely stored in the privacy module 604 (e.g. of the XR device 10).

[0080] The method steps illustrated by arrow 814 to 820 of Figure 9 can be as described with reference to arrows 712 to 718 of Figure 8, respectively. An anonymized version of thesecond request (e.g. the first request referred to herein) can be sent to the first entity 608 (e.g. cloud XR runtime). The first entity 608 can generate requested content and send it back to the privacy module 604 (e.g. as third information). The privacy module 604 may modify (e.g. change) the anonymized information (e.g. the at least one substitute element in the third information) into the original sensitive information (e.g. the at least one first information element of the fourth information). The requested content with the required information (e.g. the fourth information) can be provided to the (e.g. XR runtime of the) XR device 10. Thus, in the manner described above, the XR device 10 can securely request the generation of overlay content from the first entity 608.

[0081] Another exemplary implementation may involve remote rendering of sensitive information in avatar generation. For example, the XR device 10 may request remote generation of a user’s avatar. The XR device 10 may obtain a user input which is indicative of a request to generate an output associated with an avatar of the user, and / or an avatar of one or more nearby individuals. In such a scenario, the first information, as referred to herein, may comprise at least one first information element indicative of skin color and / or a facial characteristic of the user and / or the one or more nearby individuals. The privacy module 604 can intercept the first information and replace (e.g. swap out) the at least one first information element (e.g. sensitive information, such as skin color and significant facial characteristics) with at least one substitute element (e.g. mock data). For example, in scenarios in which the at least one first information element is indicative of a skin color of the user, an indicator of the skin color may be replaced with a placeholder indicator, such as skin color “#FFFFFF”. The privacy module 604 may use general purpose embedded LLMs, and / or specific algorithms targeting the at least one first information element, to anonymize the first information. The second information (e.g. anonymized first information) can be sent to the first entity 608 (e.g. cloud XR runtime) to generate third information (e.g. content for output), as defined herein. The first entity 608 can send the third information back to the privacy module 604.

[0082] The privacy module 604 can replace the at least one substitute element (e.g. anonymized content) with the (e.g. corresponding) at least one first information element (e.g. sensitive information). For example, a mock skin color can be replaced with a real skin color (e.g. by applying a correct shader to a 3D model). In some examples, the third information may comprise one or more (e.g. 3D) image frames. In these examples, the privacy module 604 may add at least one first information element (e.g. sensitive information) on top of the one or more image frames. For example, the privacy module604 may add a new graphical layer that adds colors, facial characteristics (e.g. facial hair and / or eye color), and / or textual information (e.g. name and / or birth date). The privacy module 604 can provide the requested content to the (e.g. runtime of the) XR device 10. Thus, in the manner described above, the XR device 10 is able to securely request the generation of avatar content from the first entity 608 without sharing sensitive, private, and / or personal information with the first entity 608.

[0083] As mentioned herein, the XR device 10 modifies first information to generate second information. Modification of the first information comprises replacing at least one first information element, as defined herein, with at least one substitute element, as defined herein. In some examples, the XR device 10 may identify the at least one first information element from the first information. The first information can comprise textual information, and / or image information, such as 3D model information. As such, the XR device 10 may be configured to identify the at least one first information element from information in multi-modal forms (e.g. comprising one or more of information types and / or formats). Identifying the at least one first information element may be referred to herein as sensitive information detection. In some examples, the privacy module 604 referred to herein may be configured to identify the at least one first information element.

[0084] In some examples, identifying the at least one first information element may be performed using a database. For example, the database may be used to compare previously fetched examples of at least one first information element (e.g. sensitive information) with the first information (e.g. comprising a second request), as defined herein. The database may include (e.g. a mix of) multi-modal information that may be used to detect personal, private, and / or sensitive information in the form of text, 3D models, images, sounds, etc. In some examples, one or more algorithms may be used to compare the first information with information (e.g. content) stored in the database in order to identify (e.g. detect) the at least one first information element.

[0085] In some examples, an LLM may be used to identify (e.g. detect) the at least one first information element. For example, an LLM may be used to detect textual (and / or other modality information) sensitive information in the first information. In some examples, an LLM can be used to replace (e.g. swap) the at least one first information element with the at least one substitute element (e.g. non-sensitive information). As described herein, in some examples, the at least one substitute element can comprise placeholder information and / or a tag. Use of an LLM is a relatively simple form of detection as itrequires little development for application to the techniques described herein. It will be understood that other ML models may be used to identify the at least one first information element, as defined herein. For example, segmentation frameworks are able to identify sensitive information, such as facial characteristics and hair color.

[0086] It will be understood that it is possible to detect textual (e.g. sensitive) information using a plurality of approaches. For example, algorithms detecting string similarity may be used to identify the at least one first information element (e.g. sensitive user information). Alternatively, or in addition, a (e.g. proper) prompt can lead an LLM to identify (e.g. and strip) the at least one first information element (e.g. sensitive information) in a body of text. It is also possible to detect sensitive information in images using vision LLMs, object detection models, and / or segmentation models. Such models can detect objects and separate them from other (e.g. adjacent) objects. Classic computer vision algorithms can detect edges and borders. In some examples, classic computer vision algorithms can be used to separate sensitive information from non-sensitive information. It is also possible to detect sensitive sounds. For example, an ML model can be trained to label sounds. This labelling can be used to identify the at least one first information element, as defined herein, in the first information. Additionally, classic algorithms for speech recognition can be used to identify sensitive sounds and / or speeches.

[0087] There is also provided a computer program comprising instructions which, when executed by processing circuitry (such as the processing circuitry 12 of the XR device 10 described herein), cause the processing circuitry to perform at least part of the method described herein. There is provided a computer program product, embodied on a non-transitory machine-readable medium, comprising instructions which are executable by processing circuitry (such as the processing circuitry 12 of the XR device 10 described herein) to cause the processing circuitry to perform at least part of the method described herein. There is provided a computer program product comprising a carrier containing instructions for causing processing circuitry (such as the processing circuitry 12 of the XR device 10 described herein) to perform at least part of the method described herein. In some embodiments, the carrier can be any one of an electronic signal, an optical signal, an electromagnetic signal, an electrical signal, a radio signal, a microwave signal, or a computer-readable storage medium.

[0088] In some embodiments, the XR device 10 functionality described herein can be performed by hardware. Thus, in some embodiments, theXR device 10 described herein can be ahardware entity. However, it will also be understood that optionally at least part or all of the XR device 10 described herein can be virtualised. For example, the functions performed by theXR device 10 described herein can be implemented in software running on generic hardware that is configured to orchestrate the XR device 10 functionality described herein.

[0089] Therefore, as described herein, there are provided improved techniques and apparatus for handling information associated with a user of an XR device. The techniques can be used to advantageously preserve user privacy and sensitive user data while still enabling XR content to be generated over a network. The techniques described herein can be smoothly integrated with cloud-supported (e.g. OpenXR) architectures and other future split rendering architectures. Furthermore, the techniques described herein enable split rendering of content at the XR device while preventing an external server (e.g. the first entity referred to herein) from fully knowing what input the XR device is obtaining and / or what output the XR device is generating. Beneficially, implementation of the techniques described herein does not require modification of existing software

[0090] It should be noted that the above-mentioned embodiments illustrate rather than limit the idea, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim, “a” or “an” does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the claims. Any reference signs in the claims shall not be construed so as to limit their scope.

Claims

CLAIMS1. A method for handling information associated with a user of an extended reality, XR, device (10), wherein the method is performed by an XR device (10) of a network, the method comprising:generating (502) first information associated with an output for the XR device (10), wherein the first information comprises at least one first information element, and wherein the at least one first information element is associated with the user of the XR device (10) or an environment of the user;modifying (504) the first information to generate second information, wherein modifying the first information comprises replacing the at least one first information element with at least one substitute element;initiating (506) transmission of a first request towards a first entity (608) of the network, wherein the first request is a request for the first entity (608) to provide, based on the second information, third information for generating the output at the XR device (10);receiving (508) the third information from the first entity (608), wherein the third information comprises the at least one substitute element; andmodifying (510) the third information to generate fourth information, wherein modifying the third information comprises replacing the at least one substitute element with the at least one first information element.

2. The method as claimed in claim 1, the method comprising:generating the output based on the fourth information.

3. The method as claimed in any of the preceding claims, the method comprising: storing a mapping indicative of an association between the at least one first information element and the at least one substitute element.

4. The method as claimed in claim 3, the method comprising:storing the mapping in a memory (14) of the XR device (10).

5. The method as claimed in claim 3 or 4, wherein modifying (510) the third information comprises modifying the third information based on the mapping.

6. The method as claimed in any of the preceding claims, wherein the at least one substitute element is generated based on the at least one first information element.

7. The method as claimed in claim 6, the method comprising:generating the at least one substitute element based on the at least one first information element.

8. The method as claimed in any of the preceding claims, wherein the at least one substitute element is not associated with the user or the environment of the user.

9. The method as claimed in any of the preceding claims, wherein:the at least one first information element comprises a first string;replacing the at least one first information element comprises replacing the first string with a second string; andthe first string and the second string are different.

10. The method as claimed in any of claims 1 to 8, wherein:the at least one first information element is in a graphics library transmission format, gITF; and / orthe at least one substitute element is in a gITF.

11. The method as claimed in any of the preceding claims, wherein the at least one substitute element is:a null value element; and / ora zero value element.

12. The method as claimed in any of the preceding claims, the method comprising: identifying the at least one first information element.

13. The method as claimed in claim 12, wherein identifying the at least one first information element comprises using a machine learning, ML, model to identify the at least one first information element.

14. The method as claimed in claim 13, wherein identifying the at least one first information element comprises analysing the first information using the ML model.

15. The method as claimed in claim 13 or 14, wherein the ML model is a trained ML model.

16. The method as claimed in any of the preceding claims, wherein the at least one first information element is associated with personal information of the user of the XR device (10).

17. The method as claimed in any of the preceding claims, wherein the at least one first information element is associated with sensitive user information.

18. The method as claimed in any of the preceding claims, wherein the at least one first information element is indicative of a characteristic of the user.

19. The method as claimed in any of the preceding claims, wherein the at least one first information element is indicative of one or more of:a name of the user;a birth date of the user;a facial characteristic of the user;a hair colour of the user;a hair length of the user;a height of the user;a weight of the user;a gender of the user;a sex of the user;a skin colour of the user;a religious orientation of the user; anda sexual orientation of the user.

20. The method as claimed in claim 19, wherein the facial characteristic of the user comprises one or more of:facial hair; andeye colour.

21. The method as claimed in any of the preceding claims, wherein the first request is a request for a runtime system (610) of the first entity (608) to provide the third information.

22. The method as claimed in any of the preceding claims, wherein the first entity (608) is a remote server of the network.

23. The method as claimed in any of the preceding claims, wherein the first entity (608) is a cloud entity of the network.

24. The method as claimed in any of the preceding claims, wherein the output comprises one or more of:a visual output;a haptic output; andan audio output.

25. The method as claimed in claim 24, wherein the output comprises:a two dimensional display, and / ora three dimensional display.

26. The method as claimed in any of the preceding claims, wherein the XR device (10) is one or more of:an augmented reality, AR, device;a virtual reality, VR, device; anda mixed reality, MR, device.

27. The method as claimed in any of the preceding claims, wherein the XR device (10) comprises one or more of:a user equipment, UE;a wearable device;a headset;glasses;a television;a smartphone; anda tablet.

28. The method as claimed in any of the preceding claims, wherein the network comprises a telecommunications network.

29. An XR device (10) comprising:processing circuitry (12) configured to cause the XR device (10) to:generate first information associated with an output for the XR device (10), wherein the first information comprises at least one first information element, and wherein the at least one first information element is associated with a user of the XR device (10) or an environment of the user;modify the first information to generate second information, wherein modifying the first information comprises replacing the at least one first information element with at least one substitute element;initiate transmission of a first request towards a first entity (608) of a network, wherein the first request is a request for the first entity (608) to provide, based on the second information, third information for generating the output at the XR device (10);receive the third information from the first entity (608), wherein the third information comprises the at least one substitute element; andmodify the third information to generate fourth information, wherein modifying the third information comprises replacing the at least one substitute element with the at least one first information element.

30. An XR device (10) as claimed in claim 29, wherein:the processing circuitry (12) is configured to cause the XR device (10) to perform the method of any of claims 2 to 28.

31. A computer program comprising instructions which, when executed by processing circuitry, cause the processing circuitry to perform the method according to any of claims 1 to 28.

32. A computer program product, embodied on a non-transitory machine-readable medium, comprising instructions which are executable by processing circuitry to cause the processing circuitry to perform the method according to any of claims 1 to 28.

Images