Content migration in split XR systems

Abstract

This disclosure provides systems, devices, apparatuses, and methods, including computer programs encoded on storage media, for content migration in split XR systems. A graphics processor may render, at a wearable display device, first content based on a first set of characteristics associated with an independent mode associated with the wearable display device. A graphics processor may obtain an indication that the wearable display device is to be switched to a companion mode associated with the wearable display device. The graphics processor may receive, based on the indication, second rendered content from the companion device based on a second set of characteristics associated with the companion pattern. The graphics processor may process the first rendered content and the second rendered content. The graphics processor may configure a combined display buffer based on the processed first rendered content and the processed second rendered content.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of Indian Provisional Application No. 202321031173, filed May 2, 2023, entitled “CONTENT MIGRATION IN SPLIT XR SYSTEMS,” the entire contents of which are expressly incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates generally to processing systems, and more particularly, to one or more techniques for graphics processing. BACKGROUND

[0004] Computing devices often perform graphics and / or display processing (e.g., with a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output frames. A central processing unit (CPU) can control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Modern CPUs are often capable of executing multiple applications simultaneously, each of which can require utilization of the GPU during execution. A display processor can be configured to convert digital information received from the CPU to analog values and can issue commands to a display panel to display visual content. A device that provides content for visual presentation on a display can utilize a CPU, GPU, and / or display processor.

[0005] Current techniques for transitioning between standalone and companion modes of an extended reality (XR) device can involve restarting an XR application on the XR device. Improved techniques related to transitioning between standalone and companion modes of an XR device are desired. SUMMARY

[0006] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0007] In one aspect of the disclosure, a method, a computer-readable medium, and an apparatus for graphics processing at a wearable display device are provided. The apparatus includes a memory and a processor coupled to the memory and, based on information stored in the memory, the processor is configured to render, at a wearable display device, first content based on a first set of characteristics associated with a standalone mode, the standalone mode being associated with the wearable display device, obtain an indication that the wearable display device is to be switched to a companion mode associated with the wearable display device, based on the indication, receive, from a companion device, second rendered content based on a second set of characteristics associated with the companion mode, process the first rendered content and the second rendered content, and configure a combined display buffer based on the processed first rendered content and the processed second rendered content.

[0008] To the accomplishment of the foregoing and related ends, one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative aspects of the one or more aspects. These aspects are indicative, however, of but a few of the various ways in which the principles of various aspects can be employed and the description is intended to include all such aspects and their equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a block diagram illustrating an example content generation system in accordance with one or more techniques of this disclosure.

[0010] Figure 2 is an example graphics processor (e.g., graphics processing unit (GPU)) in accordance with one or more techniques of this disclosure.

[0011] Figure 3 is an example display framework including a display processor and a display in accordance with one or more techniques of this disclosure.

[0012] Figure 4 is a diagram illustrating an example aspect of augmented reality (AR) in accordance with one or more techniques of this disclosure.

[0013] Figure 5 is a diagram illustrating an example aspect of a split extended reality (XR) head-mounted display (HMD) in accordance with one or more techniques of this disclosure.

[0014] Figure 6 is a diagram illustrating an example aspect of a split XR system in accordance with one or more techniques of this disclosure.

[0015] Figure 7 is a diagram illustrating an example aspect of transitioning from a standalone mode to a companion mode in a split XR system in accordance with one or more techniques of this disclosure.

[0016] is a diagram illustrating an example aspect of transitioning from a standalone mode to a companion mode in a split XR system in accordance with one or more techniques of this disclosure.

[0016] is a diagram illustrating an example aspect of transitioning from a standalone mode to a companion mode in a split XR system in accordance with one or more techniques of this disclosure.

[0016] Figure 8 is a diagram illustrating example aspects of data collection in accordance with one or more techniques of the present disclosure.

[0017] Figure 9 is a communication flow diagram illustrating example communications between a wearable display device (WDD) and a companion device in accordance with one or more techniques of the present disclosure.

[0018] Figure 10 is a flow diagram of an example method of graphics processing in accordance with one or more techniques of the present disclosure.

[0019] Figure 11 is a flow diagram of an example method of graphics processing in accordance with one or more techniques of the present disclosure. DETAILED DESCRIPTION

[0020] Various aspects of systems, apparatuses, computer program products, and methods are more fully described below with reference to the figures. The present disclosure may, however, be embodied in many different forms and should not be construed as limited to the aspects set forth throughout this present disclosure. Rather, these aspects are provided as illustrative of the present disclosure so that it will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. Based on the teachings herein one skilled in the art should appreciate that the scope of the present disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of, or combined with, other aspects of the present disclosure. For example, an apparatus can be implemented or a method can be practiced using any number of the aspects set forth herein. In addition, the scope of the present disclosure is intended to cover such an apparatus or method which is practiced using, as alternative to, or in combination with, the aspects of the present disclosure set forth herein. Any aspect disclosed herein can be embodied by one or more elements of a claim.

[0021] While aspects are described in this application by illustration to the best of the inventor's knowledge, the aspects are not limited to the examples given or the preferred embodiments. Variations of these aspects are possible as types of wireless systems vary. While certain aspects have been described as possibly being implemented, modifications, and other implementations are within the scope of the disclosure. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific aspects described herein. It is therefore, to be understood that any aspects disclosed herein are not intended to be limited to the specific aspects described herein, but rather only by the claims. The scope of the disclosure includes any other aspects falling within the scope of the claims.

[0022] Several aspects are presented with reference to various apparatus and methods. These apparatus and methods are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system.

[0023] By way of example, an element, or any portion of an element, or any combination of elements can be implemented with a "processing system" that includes one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on a chip (SOC), baseband processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. One or more processors in the processing system can execute software. Software can be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0024] The term application can refer to software. As described herein, one or more techniques can refer to an application (e.g., software) configured to perform one or more functions. In such examples, the application can be stored in a memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor, can be configured to execute the application. For example, the application can be described as including code that, when executed by the hardware, causes the hardware to perform one or more techniques described herein. As an example, the hardware can access the code from the memory and execute the code accessed from the memory to perform one or more techniques described herein. In some examples, components are identified in the present disclosure. In such examples, a component can be hardware, software, or a combination thereof. A component can be a separate component or a subcomponent of a single component.

[0025] In one or more examples described herein, the described functions can be implemented in hardware, software, or any combination thereof. If implemented in software, the described functions can be stored or encoded in one or more computer-readable media as one or more instructions or code. Computer-readable media includes computer storage media. Storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the foregoing, or any other medium that can be used to store computer executable code in the form of instructions or data structures that can be accessed by a computer.

[0026] A user can wear a display device in order to experience extended reality (XR) content. XR can refer to technologies that mix aspects of a digital experience and the real world. XR can include augmented reality (AR), mixed reality (MR), and / or virtual reality (VR). In AR, AR objects can be superimposed on a real-world environment perceived through a display device. In an example, AR content can be experienced through AR glasses that include a transparent or semi-transparent surface. When a user looks through the glasses at an environment, AR objects can be projected onto the transparent or semi-transparent surface of the glasses. Generally, AR objects can not exist in the real world and a user can not interact with AR objects. In MR, MR objects can be superimposed on a real-world environment perceived through a display device and a user can interact with the MR objects. In some aspects, MR objects can include “video see-through” that adds virtual content. In an example, a user can “touch” a MR object being displayed to the user (i.e., the user can place a hand at a location in the real world that appears to the user to be where the MR object is located from the user’s perspective) and the MR object can “move” based on the touched MR object (i.e., the location of the MR object on the display can change). Generally, MR content can be experienced through MR glasses (similar to AR glasses) worn by a user or through a head-mounted display (HMD) or headset worn by a user. A HMD can include a camera and one or more display panels. The HMD can capture images of an environment perceived through the camera and display images of the environment with MR objects superimposed thereon to the user. Unlike the transparent or semi-transparent surface of AR / MR glasses, the one or more display panels of a HMD can not be transparent or semi-transparent. In VR, a user can experience a fully immersive digital environment in which the real world is occluded. VR content can be experienced through a HMD or headset.

[0027] As used herein, examples of the term "content" can refer to "graphics content," "images," and the like, regardless of whether these terms are used as an adjective, a noun, or other part of speech. In some examples, the term "graphics content" as used herein can refer to content produced by one or more processes of a graphics processing pipeline. In other examples, the term "graphics content" as used herein can refer to content produced by a processing unit configured to perform graphics processing. In still further examples, as used herein, the term "graphics content" can refer to content produced by a graphics processing unit.

[0028] A split XR system can include a companion device (e.g., a phone) and a wearable display device (e.g., an HMD), where the companion device and the wearable display device can exchange data to facilitate display of content on the wearable display device. The companion device can have a greater amount of computing power compared to the amount of computing power of the wearable display device. In some split XR systems, the wearable display device can operate in a standalone mode or a companion mode. In standalone mode, the wearable display device can operate independent of the companion device. For example, the wearable display device can render frames and display the frames without sending or receiving data to / from the companion device. In companion mode, the wearable display device can exchange data with the companion device to facilitate display of content on the wearable display device. For example, the companion device can render frames and send the frames to the wearable display device. The wearable display device can receive the frames, process the frames, and present the processed frames for display. When the wearable display device transitions from companion mode to standalone mode (or vice versa), an application being executed by the wearable display device and / or the companion device can have to be restarted or closed. This can hinder the user experience. Furthermore, restarting the application can be associated with a computational overhead at the wearable display device and / or the companion device.

[0029] Various techniques related to content migration in split XR systems are described herein. In an example, a device (e.g., a wearable display device) renders first content at the wearable display device based on a first set of characteristics associated with a standalone mode, the standalone mode being associated with the wearable display device. The device (e.g., the wearable display device) obtains an indication that the wearable display device is to be switched to a companion mode associated with the wearable display device. The device (e.g., the wearable display device) receives, from a companion device, second rendered content based on a second set of characteristics associated with the companion mode based on the indication. The device (e.g., the wearable display device) processes the first rendered content and the second rendered content. The device (e.g., the wearable display device) configures a combined display buffer based on the processed first rendered content and the processed second rendered content. The wearable display device can seamlessly transition between the standalone mode and the companion mode with respect to obtaining the indication, processing the first rendered content and the second rendered content, and configuring the combined display buffer, which can provide an improved user experience. Moreover, by obtaining the indication, processing the first rendered content and the second rendered content, and configuring the combined display buffer, the wearable display device can seamlessly transition between the standalone mode and the companion mode without having to restart or close applications, which can save computational resources at the wearable display device.

[0030] Examples described herein can involve the use of and functionality of a graphics processing unit (GPU). As used herein, a GPU can be any type of graphics processor, and a graphics processor can be any type of processor designed or configured to process graphics content. For example, a graphics processor or GPU can be a specialized electronic circuit designed for processing graphics content. As additional examples, a graphics processor or GPU can be a general purpose processor configured to process graphics content.

[0031] Figure 1is a block diagram illustrating an example content generation system 100 configured to implement one or more techniques of this disclosure. The content generation system 100 includes a device 104. The device 104 can include one or more components or circuits to perform the various functions described herein. In some examples, one or more components of the device 104 can be components of a SOC. The device 104 can contain one or more components configured to perform one or more techniques of the present disclosure. In the illustrated example, the device 104 can include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, the device 104 can include multiple components (e.g., a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131). The display(s) 131 can refer to one or more displays 131. For example, the displays 131 can include a single display or multiple displays, which can include a first display and a second display. The first display can be a left eye display and the second display can be a right eye display. In some examples, the first display and the second display can receive different frames for presentation thereon. In other examples, the first display and the second display can receive the same frames for presentation thereon. In further examples, the results of the graphics processing can not be displayed on the device, e.g., the first display and the second display can not receive any frames for presentation thereon. Rather, the frames or graphics processing results can be communicated to another device. In some aspects, this can be referred to as split rendering.

[0032] The processing unit 120 can include an internal memory 121. The processing unit 120 can be configured to perform graphics processing using the graphics processing pipeline 107. The content encoder / decoder 122 can include an internal memory 123. In some examples, the device 104 can include a processor that can be configured to perform one or more display processing techniques on one or more frames generated by the processing unit 120 before the frames are displayed by one or more displays 131. While the processor in the example content generation system 100 is configured as a display processor 127, it should be understood that the display processor 127 is one example of a processor, and that other types of processors, controllers, etc. can be used as an alternative to the display processor 127. The display processor 127 can be configured to perform display processing. For example, the display processor 127 can be configured to perform one or more display processing techniques on one or more frames generated by the processing unit 120. The one or more displays 131 can be configured to display or otherwise present the frames processed by the display processor 127. In some examples, the one or more displays 131 can include one or more of a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.

[0033] Memory external to the processing unit 120 and the content encoder / decoder 122, such as system memory 124, can be accessible by the processing unit 120 and the content encoder / decoder 122. For example, the processing unit 120 and the content encoder / decoder 122 can be configured to read from and / or write to the external memory, such as system memory 124. The processing unit 120 can be communicatively coupled to the system memory 124 by a bus. In some examples, the processing unit 120 and the content encoder / decoder 122 can be communicatively coupled to the internal memory 121 by a bus or via a different connection.

[0034] The content encoder / decoder 122 can be configured to receive graphics content from any source, such as system memory 124 and / or the communication interface 126. The system memory 124 can be configured to store received encoded or decoded graphics content. The content encoder / decoder 122 can be configured to receive encoded or decoded graphics content in the form of encoded pixel data, for example, from the system memory 124 and / or the communication interface 126. The content encoder / decoder 122 can be configured to encode or decode any graphics content.

[0035] The internal memory 121 or system memory 124 can include one or more volatile or non-volatile memories or storage devices. In some examples, the internal memory 121 or system memory 124 can include RAM, static random access memory (SRAM), dynamic random access memory (DRAM), erasable programmable ROM (EPROM), EEPROM, flash memory, a magnetic data medium, or an optical storage medium, or any other type of memory. According to some examples, the internal memory 121 or system memory 124 can be a non-transitory storage medium. The term "non-transitory" can indicate that the storage medium is not embodied in a carrier wave or a propagating signal. However, the term "non-transitory" should not be interpreted to mean that the internal memory 121 or system memory 124 is non-removable or that its contents are static. As one example, the system memory 124 can be removed from the device 104 and moved to another device. As another example, the system memory 124 can not be removable from the device 104.

[0036] The processing unit 120 can be a CPU, GPU, GPGPU, or any other processing unit that can be configured to perform graphics processing. In some examples, the processing unit 120 can be integrated into a motherboard of the device 104. In further examples, the processing unit 120 can be present on a graphics card that is installed in a port of the motherboard of the device 104, or can be incorporated in other manners within a peripheral device that is configured to interoperate with the device 104. The processing unit 120 can include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, arithmetic logic units (ALUs), DSPs, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuitry, or any combinations thereof. If the techniques are implemented partially in software, a processing unit 120 can store instructions for the software in suitable, non-transitory computer-readable storage media (e.g., the internal memory 121) and can execute the instructions using one or more processors to perform the techniques of the present disclosure. Any of the foregoing can be considered a processor or one or more processors.

[0037] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 can be integrated into a motherboard of the device 104. The content encoder / decoder 122 can include one or more processors, such as one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuitry, or any combinations thereof. If the techniques are implemented partially in software, the content encoder / decoder 122 can store instructions for the software in a suitable, non-transitory computer-readable storage medium (e.g., the internal memory 123) and can execute the instructions using one or more processors of the device 104 to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., can be considered one or more processors.

[0038] In some aspects, the content generation system 100 can include a communication interface 126. The communication interface 126 can include a receiver 128 and a transmitter 130. The receiver 128 can be configured to perform any of the receiving functions described herein with respect to the device 104. Additionally, the receiver 128 can be configured to receive information from another device, such as eye or head position information, rendering commands, and / or position information. The transmitter 130 can be configured to perform any of the transmitting functions described herein with respect to the device 104. For example, the transmitter 130 can be configured to transmit information to another device, which can include a request for content. The receiver 128 and the transmitter 130 can be combined into a transceiver 132. In such examples, the transceiver 132 can be configured to perform any of the receiving functions and / or transmitting functions described herein with respect to the device 104.

[0039] Referring again to Figure 1 In certain aspects, the processing unit 120 can include a content migrator 198 configured to render, at a wearable display device, first content based on a first set of characteristics associated with a standalone mode, the standalone mode being associated with the wearable display device; obtain an indication that the wearable display device is to be switched to a companion mode associated with the wearable display device; based on the indication, receive, from a companion device, second rendered content based on a second set of characteristics associated with the companion mode; process the first rendered content and the second rendered content; and configure a combined display buffer based on the processed first rendered content and the processed second rendered content. Although the following description can focus on graphics processing, the concepts described herein can be applicable to other similar processing techniques, such as display processing. Furthermore, although the following description can focus on wearable display devices in an XR context, the concepts described herein can also be applicable to non-wearable display devices in an XR context.

[0040] A device, such as device 104, can refer to any device, apparatus, or system configured to perform one or more techniques described herein. For example, a device can be a server, a base station, a user device, a client device, a station, an access point, a computer (such as a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer), an end product, an appliance, a handset, a smartphone, a server, a video game platform or console, a handheld device (such as a portable video game device or a personal digital assistant (PDA)), a wearable computing device (such as a smartwatch, an augmented reality device, or a virtual reality device), a non-wearable device, a display or display device, a television, a television set-top box, an intermediary network device, a digital media player, a video streaming device, a content streaming device, an in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more techniques described herein. Processes herein can be described as being performed by a particular component (e.g., a GPU), but in other embodiments, other components (e.g., a CPU) can be used to perform consistent with the disclosed embodiments.

[0041] A GPU can process multiple types of data or data packets in a GPU pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. A context register packet can be a set of global state information, such as information about global registers, shading programs, or constant data, which can regulate how a graphics context will be processed. For example, a context register packet can include information about a color format. In some aspects of a context register packet, there can be one or more bits that indicate which workload belongs to the context register. Further, there can be multiple functions or programs that run simultaneously and / or in parallel. For example, a function or program can describe a certain operation, such as a color mode or color format. Thus, a context register can define multiple states of a GPU.

[0042] Context states can be utilized to determine how individual processing units (e.g., a vertex fetcher (VFD), a vertex shader (VS), a shader processor, or a geometry processor) function and / or in what mode the processing units function. To do so, a GPU can use context registers and program data. In some aspects, a GPU can generate workloads, such as vertex or pixel workloads, in a pipeline based on context registers of a mode or state. Certain processing units (e.g., a VFD) can use these states to determine certain functions, such as how to assemble vertices. As these modes or states can change, a GPU can need to change the corresponding context. Additionally, workloads corresponding to the modes or states can follow the changed modes or states.

[0043] Figure 2 An example GPU 200 according to one or more technologies disclosed herein is shown. Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-processing engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, an L2 cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 is shown to include processing units 220-238, but the GPU 200 may include multiple additional processing units. Furthermore, processing units 220-238 are merely examples, and according to this disclosure, the GPU may use any combination or order of processing units. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0044] like Figure 2 As shown, the GPU can utilize a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register packets (e.g., context register packet 260) and / or draw call data packets (e.g., draw call packet 212). CP 210 can then send the context register packet 260 or the draw call packet 212 to a processing unit or block within the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can simultaneously store the following information: the context register of context N, one or more draw calls of context N, the context register of context N+1, and one or more draw calls of context N+1.

[0045] GPUs can render images in a variety of different ways. In some cases, a GPU can render images using direct rendering and / or tiled rendering. In a tiled rendering GPU, an image can be divided or separated into different sections or tiles. After dividing the image, each section or tile can be rendered separately. A tiled rendering GPU can divide a computer graphics image into a grid format such that each section of the grid (i.e., a tile) is rendered separately. In some aspects of tiled rendering, during a binning pass, an image can be divided into different bins or tiles. In some aspects, during the binning pass, a visibility stream can be constructed in which visible primitives or draw calls can be identified. A rendering pass can be performed after the binning pass. In contrast to tiled rendering, direct rendering does not divide a frame into smaller bins or tiles. In fact, in direct rendering, an entire frame is rendered at a single time (i.e., there is no binning pass). Additionally, some types of GPUs can allow both tiled rendering and direct rendering (e.g., flexible rendering).

[0046] In some aspects, a GPU can apply a draw or render process to different bins or tiles. For example, a GPU can render to one bin and perform all of the drawing for the primitives or pixels in the bin. During the process of rendering to the bin, a render target can be located in a GPU internal memory (GMEM). In some cases, after rendering to one bin, the contents of the render target can be moved to system memory and the GMEM can be freed for rendering the next bin. Additionally, the GPU can render to another bin and perform drawing for the primitives or pixels in the bin. Thus, in some aspects, there can be a small number of intervals, such as four intervals, that cover all of the drawing in one surface. Furthermore, a GPU can loop through all of the drawing in one bin, but perform drawing for the draw calls that are visible (i.e., draw calls that include visible geometry). In some aspects, a visibility stream can be generated, for example, in a binning pass, to determine visibility information for each primitive in an image or scene. By way of example, this visibility stream can identify whether a primitive is visible. In some aspects, this information can be used to remove primitives that are not visible such that the not visible primitives are not rendered, for example, in a rendering pass. Likewise, at least some of the primitives that are identified as visible can be rendered in the rendering pass.

[0047] In some aspects of binning rendering, there can be multiple processing stages or passes. For example, rendering can be performed in two passes, such as a binning, visibility, or bin visibility pass and a rendering or bin rendering pass. During the visibility pass, the GPU can input the rendering workload, record the position of primitives or triangles, and then determine which primitives or triangles fall into which bin or region. In some aspects of the visibility pass, the GPU can also identify or mark the visibility of each primitive or triangle in the visibility stream. During the rendering pass, the GPU can input the visibility stream and process one bin or region at a time. In some aspects, the visibility stream can be analyzed to determine which primitives or vertices of primitives are visible or not visible. Thus, the visible primitives or vertices of primitives can be processed. By doing so, the GPU can reduce the unnecessary workload of processing or rendering primitives or triangles that are not visible.

[0048] In some aspects, during the visibility pass, certain types of primitive geometry can be processed, such as position geometry only. Additionally, depending on the positioning or location of the primitives or triangles, the primitives can be sorted into different bins or regions. In some cases, sorting the primitives or triangles into different bins can be performed by determining the visibility information of these primitives or triangles. For example, the GPU can determine or write the visibility information of each primitive in each bin or region, such as in system memory. This visibility information can be used to determine or generate a visibility stream. In the rendering pass, the primitives in each bin can be rendered individually. In these cases, the visibility stream can be extracted from memory and used to remove primitives that are not visible for the bin.

[0049] Some aspects of the GPU or GPU architecture can provide many different options for rendering, such as software rendering and hardware rendering. In software rendering, the driver or CPU can copy the entire frame geometry by processing each view Figure 1 Additionally, some different states can be changed depending on the view. As such, in software rendering, the software can copy the entire workload by changing some states that can be used to render each viewpoint in the image. In certain aspects, there can be an increased amount of overhead since the GPU can submit the same workload multiple times for each viewpoint in the image. In hardware rendering, the hardware or GPU can be responsible for copying or processing the geometry for each viewpoint in the image. Thus, the hardware can manage the copying or processing of primitives or triangles for each viewpoint in the image.

[0050] Figure 3 FIG. 3 is a block diagram 300 illustrating an example display framework including a processing unit 120, system memory 124, display processor 127, and display(s) 131 as can be identified in connection with device 104.

[0051] A GPU can be included in a device that provides content for visual presentation on a display. For example, processing unit 120 may include GPU 310, configured to render graphics data for display on a computing device (e.g., device 104), which may be a computer workstation, mobile phone, smartphone or other smart device, embedded system, personal computer, tablet computer, video game console, etc. The operation of GPU 310 can be controlled based on one or more graphics processing commands provided by CPU 315. CPU 315 can be configured to execute multiple applications simultaneously. In some cases, each of the multiple applications executing simultaneously can utilize GPU 310 simultaneously. Processing techniques can be performed by outputting frames via physical or wireless communication channels through processing unit 120.

[0052] System memory 124, executable by processing unit 120, may include user space 320 and kernel space 325. User space 320 (sometimes referred to as "application space") may include software applications and / or application frameworks. For example, software applications may include operating systems, media applications, graphics applications, workspace applications, etc. Application frameworks may include frameworks used by one or more software applications, such as libraries, services (e.g., display services, input services, etc.), application programming interfaces (APIs), etc. Kernel space 325 may also include display driver 330. Display driver 330 may be configured to control display processor 127. For example, display driver 330 may cause display processor 127 to synthesize frames and send frame data to the display.

[0053] Display processor 127 includes display control block 335 and display interface 340. Display processor 127 can be configured to manipulate the functions of displays(s) 131 (e.g., based on input received from display driver 330). Display control block 335 can be further configured to output image frames to displays 131 via display interface 340. In some examples, display control block 335 may additionally or alternatively perform post-processing of image data provided based on the processing unit 120's execution of system memory 124.

[0054] Display interface 340 can be configured to cause displays 131 (or multiple displays) to display image frames. Display interface 340 can output image data to displays 131 (or multiple displays) according to an interface protocol (e.g., MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface)). That is, displays 131 (or multiple displays) can be configured according to the MIPI DSI standard. The MIPI DSI standard supports video mode and command mode. In an example where displays 131 (or multiple displays) operate in video mode, display processor 127 can continuously refresh the graphic content of displays 131 (or multiple displays) . For example, the entire graphic content can be refreshed in each refresh cycle (e.g., line-by-line). In an example where displays 131 (or multiple displays) operate in command mode, display processor 127 can write the graphic content of a frame to buffer 350.

[0055] In some such examples, display processor 127 may not continuously refresh the graphics content of (multiple) displays 131. Instead, display processor 127 may use vertical sync (Vsync) pulses to coordinate the rendering and consumption of graphics content at buffer 350. For example, when a Vsync pulse is generated, display processor 127 may output new graphics content to buffer 350. Therefore, the generation of a Vsync pulse can indicate that the current graphics content has been rendered at buffer 350.

[0056] A frame is displayed on (multiple) monitors 131 based on a display controller 345, a display client 355, and a buffer 350. The display controller 345 can receive image data from a display interface 340 and store the received image data in the buffer 350. In some examples, the display controller 345 can output the image data stored in the buffer 350 to the display client 355. Therefore, the buffer 350 can represent the local memory of (multiple) monitors 131. In some examples, the display controller 345 can directly output the image data received from the display interface 340 to the display client 355.

[0057] Display client 355 may be associated with a touch panel that senses interaction between the user and the displays(s) 131. When the user interacts with the displays(s) 131, one or more sensors in the touch panel may output signals to display controller 345, indicating which of the one or more sensors are active, the duration of the sensor activity, the pressure applied to the one or more sensors, etc. Display controller 345 may use the sensor outputs to determine how the user interacts with the displays(s) 131. The displays(s) 131 may also be associated with / including other devices that operate in conjunction with display client 355, such as cameras, microphones, and / or speakers.

[0058] Some processing techniques of device 104 can be performed across three stages (e.g., stage 1: rendering stage; stage 2: compositing stage; and stage 3: display / transfer stage). However, other processing techniques can combine the compositing and display / transfer stages into a single stage, allowing the processing techniques to be performed based on a total of two stages (e.g., stage 1: rendering stage; and stage 2: compositing / display / transfer stage). During the rendering stage, GPU 310 can process the content buffer based on the execution of the application that generates content on a pixel-by-pixel basis. During the compositing and display stages, pixel elements can be assembled to form a frame, which is then transferred to the physical display panel / subsystem (e.g., display 131) that displays the frame.

[0059] Instructions executed by a CPU (e.g., software instructions) or a display processor can cause the CPU or display processor to search for and / or generate compositing strategies for compositing frames based on dynamic priorities and runtime statistics associated with one or more compositing strategy groups. A frame to be displayed by a physical display device, such as a display panel, may include multiple layers. Furthermore, frame compositing can be based on combining multiple layers into a frame (e.g., based on a frame buffer). After combining the multiple layers into a frame, the frame can be provided to a display panel for display. The process of combining each of the multiple layers into a frame can be referred to as compositing, frame compositing, compositing procedure, compositing process, etc.

[0060] The frame compositing process or strategy can correspond to a technique used to combine different layers from multiple layers into a single frame. Multiple layers can be stored in double data rate (DDR) memory. Each of the multiple layers can further correspond to a separate buffer. A compositor or hardware compositor (HWC) associated with a block or function determines the input to each layer / buffer and performs the frame compositing process to generate an output indicating the composite frame. That is, the input can be layers, and the output can be a frame compositing process used to synthesize the frame to be displayed on a display panel.

[0061] Some aspects of display processing can utilize different types of mask layers, such as shape masks. A mask layer is a layer that can represent a portion of a display or display panel. For example, an area of ​​the mask layer can correspond to an area of ​​the display, but the entire mask layer can depict a portion of the content actually displayed on the display or panel. For instance, a mask layer can include the top and bottom portions of the display area, but the middle portion of the mask layer can be empty. In some examples, multiple mask layers can exist to represent different portions of the display area. Furthermore, for certain portions of the display area, the contents of different mask layers can overlap each other. Therefore, a mask layer can represent a portion of the display area that may or may not overlap with other mask layers.

[0062] Figure 4 Figure 400 illustrates an example aspect of augmented reality (AR) according to one or more technologies of the present invention. Augmented reality (AR) can refer to the use of virtual content to enhance the real (physical) world. AR can be used in applications such as room design / virtual shopping, desktop AR games, turn-by-turn navigation assistance, food and health monitoring, AR video calling and / or virtual conferencing. AR can be performed via a wearable AR device capable of (1) mapping the physical world (i.e., the real world), (2) positioning the wearable AR device in the physical world, and (3) positioning / rendering virtual content on a near-eye display visible to the user. Some wearable AR devices can utilize hand / finger tracking to allow users to control the interface in the AR.

[0063] In the example, user 402 may wear HMD 404 on / above / around user 402's eyes. In the example, HMD 404 may be or include device 104. HMD 404 may be able to present AR content to the user via one or more displays 406. In the example, displays(multiple) 406 may be or include displays(multiple) 131. In the example, displays(multiple) 406 may include a left display corresponding to user 402's left eye and a right display corresponding to user 402's right eye, wherein the left display displays first content to user 402's left eye and the right display displays second content to user 402's right side. The content may be any suitable type of content, such as AR content, MR content, or XR content. The term "content" may refer to "graphic content," "image," etc. In the example, HMD 404 may be XR glasses, wearable display device (WDD), or headphones. HMD 404 may include camera 408, which enables HMD 404 to perceive the environment of HMD 404, and therefore the environment of user 402. HMD 404 may also include other sensors (e.g., inertial measurement unit (IMU) etc.) that enable HMD 404 to sense the environment of user 402.

[0064] In the example, user 402 can perceive the environment 410 of user 402 via the multiple displays 406 of HMD 404. In one example, user 402 can perceive real-world objects 412 via the multiple displays 406. In one example, HMD 404 can be XR glasses, and the multiple displays 406 can include multiple transparent or translucent display surfaces, and therefore user 402 can perceive real-world objects 412 without processing performed by HMD 404. In another example, HMD 404 can be an HMD, and the displays 406 can include opaque display surfaces. HMD 404 can perceive real-world objects 412 via multiple cameras 408, and HMD 404 can present a visual representation (i.e., an image) of real-world objects 412 on the multiple displays. This can be referred to as visual perspective content. When user 402 perceives the environment, HMD 404 can also present virtual object 414 to user 402 via display 406, where virtual object 414 may not actually exist in environment 410.

[0065] Figure 5 Figure 500 illustrates an example aspect of a split-type extended reality (XR) head-mounted display (HMD) according to one or more technologies of this disclosure. Thermal constraints can be challenging due to the small form factor used in XR HMDs (determined by fashion, comfort, etc.). For example, for user comfort, XR HMDs can be relatively lightweight and / or can have a relatively small size. The lightweight nature and / or small size of XR HMDs affect their heat dissipation capabilities. Furthermore, the lightweight nature and / or small size of XR HMDs may result in relatively limited battery life.

[0066] One approach to reducing power consumption in XR HMDs is to separate processing between the HMD and accompanying devices (e.g., pucks or cellular phones). This can be referred to as a “split design.” Several important considerations exist in a split design. In some split design configurations, rendering (e.g., high-power workloads) can be performed on the accompanying device; however, this may necessitate performing post-reprojection (LSR) on the XR HMD (e.g., XR glasses). LSR can refer to reprojecting previously rendered frames based on the user’s latest head pose to reduce motion photon latency and avoid user nausea. Depending on latency / power characteristics, perceptual workloads (hand tracking, head tracking, body tracking, 3D reconstruction, etc.) can be performed on either the XR HMD or the accompanying device.

[0067] In the example, the XR HMD 502 can be configured in a split-presentation configuration together with the accompanying device 504. In the example, the XR HMD 502 can be or includes device 104 or HMD 404. In the example, the XR HMD 502 can be or includes XR glasses. In one example, the accompanying device 504 can be or includes device 104. In one example, the accompanying device 504 can be or includes a smartphone, tablet, puck, video game console, server, cloud server, desktop computing device, laptop computing device, etc. The accompanying device 504 can include a first processing power, and the XR HMD 502 can include a second processing power, wherein the first processing power can be greater than the second processing power. For example, compared to one or more processors, memory, etc. of the XR HMD 502, the accompanying device 504 can include one or more faster processors, more memory, etc. In one example, the accompanying device 504 can include a greater battery life compared to the battery life of the XR HMD 502.

[0068] The XR HMD 502 may include a camera 506 and a display panel 508. In the example, camera 506 may be or include camera 408, and display panel 508 may be or include display 131 or display 406. The XR HMD 502 may also include an inertial measurement unit (IMU). Figure 5 (Not depicted in the image). The XR HMD 502 can sense its surroundings via camera 506 and / or IMU. The XR HMD 502 can display content (e.g., XR content) to the user via display panel 508. The XR HMD 502 may include a display processing unit (DPU) (e.g., DPU 510). In the example, DPU 510 may be or include display processor 127. The XR HMD 502 can be configured to determine the six degrees of freedom (6DOF) pose 512 of the user's head based on data generated by the IMU. The 6DOF pose 512 may include position information (surge, undulation, sway) and orientation information (roll, pitch, yaw) of the user's head when the user is wearing the XR HMD 502. The XR HMD 502 may also include client AR applications ( Figure 5 (Not shown in the image). In contrast, three-degree-of-freedom (3DOF) attitude can include orientation information (roll, pitch, yaw) but not position information (surge, undulation, roll).

[0069] The XR HMD 502 can be configured to perform segmentation-aware coding 514 on 6DOF pose 512 (or other data). The XRHMD 502 can transmit the encoded information (e.g., the (coded) 6DOF pose 512) to an accompanying device 504 via link 516. In the example, link 516 can be a wired link or a wireless link. In a particular example, link 516 can be or include a wireless local area network (WLAN) link, such as an IEEE 802.11ax 6GHz link. In another example, link 516 can be a 5G New Radio (5G NR) link.

[0070] The accompanying device 504 can receive encoded information via link 516, and can perform segmentation-aware decoding 518 on the encoded information to decode it. In one example, segmentation-aware decoding 518 can render a 6DOF pose 512 on the accompanying device 504. At 520, the accompanying device 504 can perform hand tracking (HaT), plane finding (PlaneF), and / or image tracking (IT) / object tracking (OT). For user interface (UI) purposes, hand tracking can identify and locate the user's hand. Plane finding can identify and locate planes in the environment (e.g., tables, walls, etc.), allowing virtual content to be positioned on the plane. Image tracking / object tracking can identify and locate images (e.g., quick-response (QR) codes) or objects (e.g., game pieces, controllers, etc.) to enable interactivity with real-world objects.

[0071] The accompanying device 504 may include an AR application 522 and a software development kit (SDK) 524. SDK 524 may include a collection of software development tools in an installable package. SDK 524 may facilitate the creation of (multiple) applications (e.g., (multiple) AR applications 522) by including compilers, debuggers, and / or software frameworks. SDK 524 may be specific to the hardware platform and / or operating system of the accompanying device 504. The accompanying device 504 may perform rendering 526 (e.g., rendering AR content) based on (decoded) 6DOF pose 512 via (multiple) AR applications 522 and / or SDK 524. The accompanying device 504 may perform encoding 528 on the rendered AR content. The accompanying device 504 may then transmit the (encoded) rendered AR content to XR HMD 502 via link 516.

[0072] The XR HMD 502 can receive (encoded) rendered AR content from the accompanying device 504. The XR HMD 502 can perform decoding 530 on the (encoded) rendered AR content. The XR HMD 502 can perform warping 532 on the (decoded) rendered AR content. The XR HMD 502 can present the (warped and decoded) rendered AR content on the display panel 508.

[0073] Figure 6 Figure 600 illustrates an example aspect of a split-type XR system according to one or more technologies of this disclosure. In a split-type XR system, the HMD (e.g., XR HMD 502) can operate in standalone mode as well as with an accompanying device (e.g., a mobile phone or a server). Table 1 below illustrates the modes of interest.

[0074] Tracking mode Rendering mode Rendered position Independent smart glasses mode Three DOF Head locked HMD Split XR viewer mode Six DOF World locked Accompanying

[0075] Table 1: Patterns of Interest

[0076] Dynamically switching between modes of interest can pose challenges for XR applications. In the first example 602, an XR device (e.g., HMD 404, XR HMD 502, smart glasses, etc.) can switch from 3DOF mode 604 (i.e., 3DOF tracking) to 6DOF mode 606 (i.e., 6DOF tracking). 3DOF can refer to three rotation axes, allowing for left / right rotation, up / down viewing, and tilting the view. 6DOF can additionally include three translation degrees (in addition to 3DOF), allowing for left / right, forward / backward, and up / down movement. In the first example 602, when a user returns home, the user (e.g., user 402) can use a head-locked smart glasses application (e.g., a navigation application) on the XR device in standalone mode. Standalone mode (e.g., single XR viewer mode) can be associated with a head-locked rendering mode (i.e., a rendering mode where head movement is locked). The user's home may include accompanying devices (e.g., accompanying device 504). Head-locked smart glasses apps can connect to companion devices and switch to companion mode (e.g., split-screen XR viewer mode). Companion mode can be associated with world-locked rendering mode (i.e., rendering mode where world movement is locked). However, to switch from standalone mode to companion mode, the head-locked smart glasses app may need to restart, which could impact the user experience.

[0077] In the second example 608, an XR device (e.g., HMD 404, XR HMD 502, smart glasses, etc.) can switch from 6DOF mode 606 (i.e., 6DOF tracking) to 3DOF mode 604 (i.e., 3DOF tracking). In the second example 608, the user is at home (the WLAN companion device (e.g., companion device 504) is located at home). The user may be using an AR viewer app on a world-locked XR device. For example, the user may be watching TV on the wall of their home through the XR device. The user may also decide to go for a run. The XR device can switch to head-locked content mode, where content is presented on the XR device. However, to switch to the aforementioned mode, the AR viewer app may restart, which could affect the user experience.

[0078] Figure 7 Figure 700 illustrates an example aspect of transitioning from standalone mode to companion mode in a split-type XR system according to one or more techniques of this disclosure. In one aspect, an application can be structured into two parts: a “simple application” that can be rendered from the HMD as one or more 2D planes, and a “rich” application that can be rendered from the companion device in full 3D. The simple part of the application can be rendered from the HMD, thus allowing for a smooth transition between the head-locked layer and distortion and compositing (on the HMD) to the specified plane as part of the companion-enabled rich application. The aspects presented herein may involve a transition between the two modes that can be triggered manually or automatically (when the companion device is paired). When paired, some data can be transferred between the simple application rendering and the rich application rendering to keep the applications synchronized with each other. When the HMD operates in standalone mode, the HMD completely takes over application rendering.

[0079] In one example, HMD 702 may operate in companion mode 704 together with companion device 706. In the example, HMD 702 may be or include device 104, HMD 404, or XR HMD 502. In one example, companion device 706 may be or include device 104 or companion device 504.

[0080] HMD 702 may include a first application 708 (i.e., a “simple application”), and accompanying device 706 may include a second application 710 (i.e., a “rich application”). The first application 708 and the second application 710 may exchange synchronization data 712, which enables the first application 708 and the second application 710 to synchronize with each other for presentation purposes. Synchronization data may refer to data used to keep local and remote applications synchronized, such as user interactions (e.g., button presses) and the currently displayed screen (e.g., a menu).

[0081] The first application 708 can perform a first rendering 714. In an example, the first rendering 714 may include rendering a two-dimensional (2D) plane. In an example, the first rendering 714 may correspond to a head-locked rendering mode. The first application 708 can perform a first rendering 714 regardless of whether it operates in companion mode 704. The second application 710 can perform a second rendering 716. In an example, the second rendering 716 may include rendering in a fully 3D manner. In an example, the second rendering 716 may correspond to a world-locked rendering mode. The second application 710 can transmit (e.g., via link 516) a world-locked anchor point 718 in a specified plane to the HMD 702 based on the second rendering 716. The second application 710 can also transmit (e.g., via link 516) the content rendered by the second renderer 716 (i.e., the second content) to the HMD 702. The HMD 702 can perform a warp 720 based on the first content rendered by the first rendering 714 and the world-locked anchor point 718. Distort 720 can generate distorted content (e.g., distorted frames). Distort can transform (e.g., scale, rotate, project, etc.) content so that it appears locked to a specific anchor point.

[0082] The second application 710 can also generate a content eye buffer 722 based on the second rendering 716. The second application 710 can transfer the content eye buffer 722 to the HMD 702. In the example, the content eye buffer 722 may include a left eye buffer for the left display of the HMD 702 and a right eye buffer for the right display of the HMD 702. The HMD 702 can perform compositing 724 based on (1) the warped content generated by warping 720 and (2) the content eye buffer 722. The HMD 702 can perform LSR 726 on the (composite) content eye buffer 722. The HMD 702 can then render the content eye buffer 722 on a display such as display 406. The compositing process can refer to generating a composite image from multiple images by blending two images in a specific z-order.

[0083] In another example, HMD 702 can operate in standalone mode 728. In standalone mode 728, the first application 708 can perform the first rendering 714 as described above. In standalone mode 728, HMD 702 can display content without assistance from the accompanying device 706.

[0084] HMD 702 can switch between companion mode 704 and stand-alone mode 728 automatically (e.g., based on companion device 706 being available for HMD 702) or manually (e.g., via user input). In one aspect, HMD 702 can switch between companion mode 704 and stand-alone mode 728 without exiting or restarting the first application 708.

[0085] Figure 8 Figure 800 illustrates an example aspect of data collection according to one or more techniques of this disclosure. When an XR device enters standalone mode, the XR device can continue to collect and buffer perceptual data (even if the perceptual data is not processed or used by a “simple application” executed on the XR device), thereby improving the “rich application” experience when the XR device is re-paired with a companion device. In the example, the XR device can continue to collect and search for 6DOF keyframes and / or 3D reconstruction (3DR) keyframes, even when the XR device is not actively running 6DOF and / or 3DR algorithms. When the XR device is re-paired with a companion device (e.g., a mobile phone), the XR device does not have to wait for remapping or 3DR to catch up with the rendering process.

[0086] In the example, HMD 702 may include multiple sensors 802 (e.g., multiple cameras) that enable HMD 702 to perceive its environment 804. HMD 702 can enter standby mode 728. When HMD 702 operates in standby mode 728, HMD 702 can collect perception data 806 via sensors 802 even when the first application 708 is not processing or otherwise utilizing perception data 806. In one example, perception data 806 may include 6DOF keyframes, 3DR keyframes, a summary of IMU data, and / or head pose. HMD 702 may store perception data 806 in buffer 810. When HMD 702 enters companion mode 704, HMD 702 may utilize perception data 806 in buffer 810 to facilitate content display.

[0087] Figure 9 This is a communication flowchart 900 illustrating example communication between a wearable display device (WDD) 902 and an accompanying device 904 according to one or more technologies of this disclosure. In the example, WDD 902 may be or include device 104, HMD 404, XR HMD 502 and / or HMD 702. In one example, accompanying device 904 may be or include device 104, accompanying device 504 and / or accompanying device 706.

[0088] At 906, WDD 902 can render first content at a wearable display device (e.g., WDD 902) based on a first set of features associated with an independent mode, which is associated with the wearable display device. At 916, WDD 902 receives an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device. At 920, WDD 902 can receive second rendered content based on a second set of features associated with the accompanying mode from an accompanying device (e.g., accompanying device 904) based on this indication. At 922, WDD 902 can process the first and second rendered content. At 924, WDD 902 can configure a combined display buffer based on the processed first and second rendered content. The display buffer may refer to a buffer that temporarily stores content ready for display.

[0089] At 918, WDD 902 can send synchronization data for an application running on the wearable display device, wherein receiving second rendered content at 920 can include further receiving second rendered content based on the synchronization data. At 912, WDD 902 can pair the wearable display device with a companion device, wherein an indication that the wearable display device will be switched to companion mode can be obtained after pairing. At 914, WDD 902 can receive user input at the wearable display device, wherein an indication that the wearable display device will be switched to companion mode can be obtained based on user input.

[0090] At 910, WDD 902 can output third content for display before receiving an instruction. At 926, WDD 902 can output content for display after receiving an instruction, where the content can be based on the second and fourth content.

[0091] At 928, WDD 902 can obtain a second indication that the wearable display device will be switched to standalone mode after processing. At 930, WDD 902 can render third content based on the second indication and a first set of characteristics associated with standalone mode. At 908, WDD 902 can collect data associated with the wearable display device before obtaining the indication and while the wearable display device is operating in standalone mode, wherein the processing of the first and second rendered content at 922 can be further based on this data.

[0092] Figure 10 This is a flowchart 1000 illustrating an example method of graphic processing according to one or more techniques of this disclosure. The method can be implemented by combining... Figures 1-9The method is executed using devices such as graphics processing devices, GPUs, CPUs, wearable display devices (e.g., HMD 404, WDD 902), XR HMD 502, HMD 702, device 104, wireless communication devices, etc. In this example, the method can be executed by content migrater 198.

[0093] At 1002, the device (e.g., a wearable display device) renders first content at the wearable display device based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device. For example, Figure 9 At 906, it is shown that WDD 902 can render first content at a wearable display device based on a first set of features associated with a standalone mode, which is associated with the wearable display device. In the example, the wearable display device can be device 104, HMD 404, WDD 902, XR HMD 502, or HMD 702. In one example, the standalone mode can be standalone mode 728. In the example, rendering the first content can correspond to first rendering 714. In one example, the first set of features can be associated with a standalone smart glasses mode in Table 1 above. In the example, 1002 can be performed by content migrater 198.

[0094] At 1004, the device (e.g., a wearable display device) receives an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device. For example, Figure 9 At 916, it is shown that WDD 902 can obtain an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device. In the example, the accompanying mode can be accompanying mode 704. In the example, the switch can be... Figure 6 Related. In the example, 1004 can be executed by content migrater 198.

[0095] At point 1006, the device (e.g., a wearable display device) receives, based on the instruction, second rendered content from the accompanying device based on a second set of features associated with the accompanying mode. For example, Figure 9 At 920, it is shown that WDD 902 can receive second rendered content based on a second set of features associated with the accompanying mode from the accompanying device. In one example, the accompanying device may be or include device 104, accompanying device 504, and / or accompanying device 706. In the example, the second rendered content may correspond to second render 716. In the example, the second set of features may be associated with the split XR viewer mode in Table 1 above. In the example, 1006 may be performed by content migrater 198.

[0096] At point 1008, the device (e.g., a wearable display device) processes the first rendered content and the second rendered content. For example, Figure 9 Figure 922 shows that WDD 902 can be used to process first and second rendered content. In the example, processing the first and second rendered content may correspond to warping 720, compositing 724, and / or LSR 726. In the example, 1008 may be performed by content migrater 198.

[0097] At 1010, the device (e.g., a wearable display device) configures a combined display buffer based on the processed first rendered content and the processed second rendered content. For example, Figure 9 At 924, it is shown that WDD 902 can configure a combined display buffer based on processed first rendered content and processed second rendered content. In the example, the combined display buffer can be associated with content eye buffer 722. In the example, 1010 can be performed by content migrater 198.

[0098] Figure 11 This is a flowchart 1100 illustrating an example method of graphic processing according to one or more techniques of this disclosure. The method can be performed by, for example, combining... Figures 1-9 The method is performed using devices such as graphics processing devices, GPUs, CPUs, wearable display devices (e.g., HMD 404, WDD 902), XR HMD 502, HMD 702, device 104, wireless communication devices, etc. In this example, the method (including the aspects detailed below) may be performed by content migrater 198.

[0099] At 1102, the device (e.g., a wearable display device) renders first content at the wearable display device based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device. For example, Figure 9 At 906, it is shown that WDD 902 can render first content at a wearable display device based on a first set of features associated with a standalone mode, which is associated with the wearable display device. In the example, the wearable display device can be device 104, HMD 404, WDD 902, XR HMD 502, or HMD 702. In one example, the standalone mode can be standalone mode 728. In the example, rendering the first content can correspond to first rendering 714. In one example, the first set of features can be associated with a standalone smart glasses mode in Table 1 above. In the example, 1102 can be performed by content migrater 198.

[0100] At 1112, the device (e.g., a wearable display device) receives an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device. For example, Figure 9 At 916, it is shown that WDD 902 can obtain an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device. In the example, the accompanying mode can be accompanying mode 704. In the example, the switch can be... Figure 6 Related. In the example, 1112 can be executed by content migrater 198.

[0101] At 1116, the device (e.g., a wearable display device) receives, based on the instruction, second rendered content from the accompanying device based on a second set of characteristics associated with the accompanying mode. For example, Figure 9 At 920, it is shown that WDD 902 can receive second rendered content based on a second set of features associated with the accompanying mode from the accompanying device. In one example, the accompanying device may be or include device 104, accompanying device 504, and / or accompanying device 706. In the example, the second rendered content may correspond to second render 716. In the example, the second set of features may be associated with the split XR viewer mode in Table 1 above. In the example, 1116 may be performed by content migrater 198.

[0102] At 1118, the device (e.g., a wearable display device) processes the first rendered content and the second rendered content. For example, Figure 9 Figure 922 shows that WDD 902 can be used to process first and second rendered content. In the example, processing the first and second rendered content may correspond to warping 720, compositing 724, and / or LSR 726. In the example, 1118 may be performed by content migrater 198.

[0103] At 1120, the device (e.g., a wearable display device) configures a combined display buffer based on the processed first rendered content and the processed second rendered content. For example, Figure 9 At 924, it is shown that WDD 902 can configure a combined display buffer based on processed first rendered content and processed second rendered content. In the example, the combined display buffer can be associated with content eye buffer 722. In the example, 1120 can be performed by content migrater 198.

[0104] In one aspect, the first set of features may include three-degree-of-freedom (3DOF) tracking, and the second set of features may include six-degree-of-freedom (6DOF) tracking. For example, 3DOF tracking may correspond to 3DOF mode 604, and 6DOF tracking may correspond to 6DOF mode 606.

[0105] In one aspect, the first set of features may include a head-locked rendering mode, and the second set of features may include a world-locked rendering mode. For example, head-locked rendering may be the head-locked rendering mode in Table 1 above, and the world-locked rendering mode may be the world-locked rendering mode in Table 1 above.

[0106] In one aspect, rendering the first content may include rendering at least one first two-dimensional (2D) plane, and the second rendered content may include at least one second 2D plane or a three-dimensional (3D) scene. For example, rendering the first content at 906 may include rendering at least one first two-dimensional (2D) plane, and the second rendered content (received at 920) may include at least one second 2D plane or a three-dimensional (3D) scene.

[0107] In one aspect, at 1114, the device (e.g., a wearable display device) may send synchronization data for an application executing on the wearable display device, wherein receiving the second rendered content may include: further receiving the second rendered content based on the synchronization data. For example, Figure 9 At 918, it is shown that WDD 902 can send synchronization data for an application executing on a wearable display device, wherein receiving second rendered content at 920 can include further receiving second rendered content based on the synchronization data. In one example, the synchronization data may be or include synchronization data 712. In the example, the application may be or include a first application 708. In the example, 1114 may be executed by content migrater 198.

[0108] In one aspect, at 1108, the device (e.g., a wearable display device) can pair the wearable display device with an accompanying device, wherein an indication that the wearable display device will be switched to accompanying mode can be obtained after pairing. For example, Figure 9 Figure 912 shows that WDD 902 can pair a wearable display device with a companion device, wherein in figure 916 an indication that the wearable display device will be switched to companion mode can be obtained after pairing. In the example, 1108 can be performed by content migrater 198.

[0109] In one aspect, at 1110, the device (e.g., a wearable display device) can receive user input at the wearable display device, wherein an indication that the wearable display device will be switched to an accompanying mode can be obtained based on the user input. For example, Figure 9 At 914, it is shown that WDD 902 can receive user input at the wearable display device, where an indication that the wearable display device will be switched to companion mode at 916 can be obtained based on the user input. In the example, 1110 can be performed by content migrater 198.

[0110] In one aspect, processing the first and second rendered content may include performing at least one of the following: warping the first and second rendered content, compositing the first and second rendered content, or performing a late reprojection (LSR) on the first and second rendered content. For example, processing the first and second rendered content at 922 may include performing at least one of the following: warping the first and second rendered content, compositing the first and second rendered content, or performing a late reprojection (LSR) on the first and second rendered content. In the example, warping may correspond to warping 720, compositing may correspond to compositing 724, and LSR may correspond to LSR 726.

[0111] In one aspect, an application can be executed at a wearable display device, wherein rendering first content, receiving instructions, receiving second rendered content, processing the first and second rendered content, and configuring a display buffer that combines the two can occur without restarting or closing the application. For example, the application could be a first application 708. In the example, rendering the first content at 906, receiving instructions at 916, receiving the second rendered content at 920, processing the first and second rendered content at 922, and configuring a display buffer that combines the two at 924 can occur without restarting or closing the first application 708.

[0112] In one aspect, presenting first content based on a first set of features associated with an independent mode may include: presenting third content at a first moment; and rendering fourth content at a second moment, wherein the first moment occurs before receiving an instruction, and wherein the second moment occurs after receiving an instruction. For example, presenting first content at 906 based on a first set of features associated with an independent mode may include: presenting third content at a first moment; and rendering fourth content at a second moment, wherein the first moment occurs before receiving an instruction at 916, and wherein the second moment occurs after receiving an instruction.

[0113] In one aspect, at 1106, the device (e.g., a wearable display device) can output third content for display before receiving an instruction. For example, Figure 9 At 910, it is shown that WDD 902 can output third content for display before receiving an instruction at 916. In the example, 1106 can be performed by content migrater 198.

[0114] In one aspect, at 1122, the device (e.g., a wearable display device) can output content for display after receiving an instruction, wherein the content can be based on a second content and a fourth content. For example, Figure 9At 926, it is shown that WDD902 can output content for display after receiving an instruction at 916, where the content can be based on a second content and a fourth content. In the example, 1122 can be performed by content migrater 198.

[0115] In one aspect, at 1124, the device (e.g., a wearable display device) can, after processing, obtain a second indication that the wearable display device will be switched to an independent mode. For example, Figure 9 At 928, it is shown that WDD 902 can obtain a second indication that the wearable display device will be switched to standalone mode after the processing at 922. In the example, 1124 can be performed by content migrater 198.

[0116] In one aspect, at 1126, the device (e.g., a wearable display device) can render third content based on a second instruction and a first set of features associated with an independent mode. For example, Figure 9 At 930, it is shown that WDD 902 can render third content based on a first set of features associated with an independent mode, based on a second instruction. In the example, 1126 can be performed by content migrater 198.

[0117] In one aspect, when the wearable display device operates in standalone mode, it can operate without exchanging data with the accompanying device, while when the wearable display device operates in companion mode, it can exchange data with the accompanying device. For example, when WDD 902 operates in standalone mode 728, WDD 902 can operate without exchanging data with the accompanying device 904, while when WDD 902 operates in companion mode 704, WDD 902 can exchange data with the accompanying device 904.

[0118] In one aspect, at 1104, the device (e.g., a wearable display device) may collect data associated with the wearable display device before receiving an instruction and when the wearable display device is operating in stand-alone mode, wherein processing of the first and second rendered content may be further based on this data. For example, ​ At 908, it is shown that WDD 902 can collect data associated with the wearable display device before receiving instructions at 916 and when the wearable display device is operating in standalone mode, wherein the processing of the first and second rendered content at 922 can be further based on this data. In the example, 1104 can be performed by the content migrater 198.

[0119] In one respect, the data may include sensed data. For example, sensed data may be or include sensed data 806.

[0120] In this configuration, a method or apparatus for graphics processing is provided. The apparatus may be a GPU, a CPU, or some other processor capable of performing graphics processing. In various aspects, the apparatus may be a processing unit 120 within device 104, or some other hardware within device 104 or another device. The apparatus (e.g., processing unit 120) may include components for rendering first content at a wearable display device based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device. The apparatus (e.g., processing unit 120) may further include components for obtaining an indication that the wearable display device will switch to an accompanying mode associated with the wearable display device. The apparatus (e.g., processing unit 120) may include components for receiving second rendered content from an accompanying device based on a second set of features associated with the accompanying mode, based on the indication. The apparatus (e.g., processing unit 120) may include components for processing the first and second rendered content. The apparatus (e.g., processing unit 120) may include components for configuring a combined display buffer based on the processed first and second rendered content. The apparatus (e.g., processing unit 120) may include components for transmitting synchronization data for an application executed on a wearable display device, wherein receiving second rendered content includes further receiving the second rendered content based on the synchronization data. The apparatus (e.g., processing unit 120) may include components for pairing the wearable display device with a companion device, wherein an indication that the wearable display device will be switched to companion mode is obtained after pairing. The apparatus (e.g., processing unit 120) may include components for receiving user input at the wearable display device, wherein the indication that the wearable display device will be switched to companion mode is obtained based on the user input. The apparatus (e.g., processing unit 120) may include components for outputting third content for display before obtaining an indication. The apparatus (e.g., processing unit 120) may include components for outputting content for display after obtaining an indication, wherein the content is based on second and fourth content. The apparatus (e.g., processing unit 120) may include components for obtaining a second indication that the wearable display device will switch to standalone mode after processing. The apparatus (e.g., processing unit 120) may include components for rendering third content based on a first set of features associated with a standalone mode, based on a second instruction. The apparatus (e.g., processing unit 120) may include components for collecting data associated with the wearable display device before receiving the instruction and while the wearable display device is operating in standalone mode, wherein processing of the first rendered content and the second rendered content is further based on said data.

[0121] It should be understood that the specific order or hierarchy of boxes / steps in the processes, flowcharts, and / or call flowcharts disclosed herein is illustrative of exemplary methods. It should be understood that the specific order or hierarchy of boxes / steps in these processes, flowcharts, and / or call flowcharts may be rearranged according to design preferences. Furthermore, some boxes / steps may be combined and / or omitted. Other boxes / steps may also be added. The appended method claims provide the elements of the various boxes / steps in an exemplary order, but this does not imply that they are limited to the given specific order or hierarchy.

[0122] To enable any person skilled in the art to implement the various aspects described herein, the foregoing has described these aspects. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be granted the full scope consistent with the language of the claims, wherein references to the singular form of an element, unless specifically stated otherwise, are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serves as an example, instance, or illustration.” Any aspect described herein as “exemplary” should not be construed as preferred or advantageous over other aspects.

[0123] Unless otherwise specified, the term "some" means one or more, and the term "or" can be interpreted as "and / or" unless otherwise indicated by the context. Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiples of A, multiples of B, or multiples of C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, wherein any such combination may include one or more members of A, B, or C. Elements of the various aspects described throughout this disclosure that are present or hereafter known to those skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be made public, whether or not such disclosure is explicitly stated in the claims. Terms such as “module,” “mechanism,” “element,” and “device” may not be substitutes for the term “apparatus.” Therefore, unless an element is explicitly described using the phrase “unit for…”, no claim element should be interpreted as a functional unit.

[0124] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" has been used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then such functionality, processing unit, technique, or other module may be stored on or transmitted via a computer-readable medium as one or more instructions or code.

[0125] Computer-readable media may include computer data storage media or communication media, wherein communication media includes any medium that facilitates the transfer of a computer program from one place to another. In this way, computer-readable media may generally correspond to: (1) a tangible computer-readable storage medium that is non-transitory; or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, optical disc read-only memory (CD-ROM) or other optical disc storage, disk storage, or other magnetic storage devices. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while optical discs optically copy data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0126] The technology of this invention can be implemented in a wide variety of devices or equipment, including wireless handheld devices, integrated circuits (ICs), or a set of ICs, such as chipsets. The various components, modules, or units described herein are intended to emphasize functional aspects of a device configured to perform the disclosed technology, but do not necessarily need to be implemented by different hardware units. Rather, as described above, the various units can be combined in any hardware unit, or provided by a collection of interoperable hardware units including one or more processors as described above, combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the technology described herein. Furthermore, the technology can be fully implemented in one or more circuit or logic elements.

[0127] Aspect 1 is a graphics processing method comprising: rendering first content at a wearable display device based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device; obtaining an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device; receiving, based on the indication, second rendered content based on a second set of features associated with the accompanying mode from an accompanying device; processing the first rendered content and the second rendered content; and configuring a combined display buffer based on the processed first rendered content and the processed second rendered content.

[0128] Aspect 2 can be combined with aspect 1, wherein the first set of features includes three-degree-of-freedom (3DOF) tracking, and wherein the second set of features includes six-degree-of-freedom (6DOF) tracking.

[0129] Aspect 3 can be combined with any one of Aspects 1-2, wherein the first feature set includes a head-locked rendering mode, and wherein the second feature set includes a world-locked rendering mode.

[0130] Aspect 4 can be combined with any of aspects 1-3, wherein rendering the first content includes rendering at least one first two-dimensional (2D) plane, and wherein the second rendered content includes at least one second 2D plane or a three-dimensional (3D) scene.

[0131] Aspect 5 may be combined with any of aspects 1 to 4, and further includes: sending synchronization data for an application to be executed on a wearable display device, wherein receiving the second rendered content includes: further receiving the second rendered content based on the synchronization data.

[0132] Aspect 6 can be combined with any of aspects 1-5, and further includes: pairing the wearable display device with an accompanying device, wherein, after pairing, an indication is obtained that the wearable display device will be switched to accompanying mode.

[0133] Aspect 7 may be combined with any of aspects 1 to 6, and further includes: receiving user input at a wearable display device, wherein the indication that the wearable display device will be switched to companion mode is obtained based on the user input.

[0134] Aspect 8 may be combined with any of Aspects 1-7, wherein processing the first rendered content and the second rendered content includes performing at least one of the following: warping the first rendered content and the second rendered content, compositing the first rendered content and the second rendered content, or performing a late reprojection (LSR) on the first rendered content and the second rendered content.

[0135] Aspect 9 can be combined with any of aspects 1-8, wherein the application is executed at a wearable display device, and wherein rendering the first content, obtaining an instruction, receiving the second rendered content, processing the first and second rendered content, and configuring the combined display buffer occur without restarting or closing the application.

[0136] Aspect 10 can be combined with any of aspects 1-9, wherein presenting the first content based on the first set of features associated with the independent mode includes: presenting the third content at a first moment; and rendering the fourth content at a second moment, wherein the first moment occurs before the instruction is received, and wherein the second moment occurs after the instruction is received.

[0137] Aspect 11 can be combined with aspect 10 and further includes: outputting the third content for display before receiving the instruction; and outputting content for display after receiving the instruction, wherein the content is based on the second content and the fourth content.

[0138] Aspect 12 may be combined with any of aspects 1-11, and further includes: after the processing, obtaining a second indication that the wearable display device will be switched to the standby mode; and based on the second indication, rendering third content based on a first set of features associated with the standby mode.

[0139] Aspect 13 may be combined with any of aspects 1-12, wherein when the wearable display device operates in the stand-alone mode, the wearable display device operates without exchanging data with the accompanying device, and wherein when the wearable display device operates in the accompanying mode, the wearable display device exchanges data with the accompanying device.

[0140] Aspect 14 may be combined with any of aspects 1 to 13, and further includes: collecting data associated with the wearable display device before receiving the instruction and when the wearable display device is operating in the stand-alone mode, wherein processing the first rendered content and the second rendered content is also based on the data.

[0141] Aspect 15 can be combined with aspect 14, wherein the data includes perceived data.

[0142] Aspect 16 is an apparatus for graphics processing, including a processor coupled to a memory, and the processor is configured to implement the method as described in any one of aspects 1-15 based on information stored in the memory.

[0143] Aspect 17 may be combined with aspect 16 and includes: the device is a wireless communication device, the wireless communication device including at least one of a transceiver or an antenna coupled to the processor, wherein, in order to receive the second presented content, the processor is configured to receive the second presented content via at least one of the transceiver or the antenna.

[0144] Aspect 18 is an apparatus for graphics processing, including components for implementing the method described in any one of aspects 1-15.

[0145] Aspect 19 is a computer-readable medium (e.g., a non-transitory computer-readable medium) storing computer-executable code that, when executed by a processor, causes the processor to implement the method as described in any one of aspects 1-15.

[0146] Various aspects have been described herein. These and other aspects are within the scope of the appended claims.

Claims

1. An apparatus for graphics processing, comprising: Memory; and A processor, coupled to the memory, and configured based on information stored in the memory, to: At the wearable display device, first content is rendered based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device; Obtain an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device; Based on the instruction, receive second rendered content from the accompanying device based on a second set of features associated with the accompanying mode; Process the first rendered content and the second rendered content; as well as Configure the combined display buffer based on the processed first rendered content and the processed second rendered content.

2. The apparatus according to claim 1, wherein, The first set of features includes three-degree-of-freedom (3DOF) tracking, and the second set of features includes six-degree-of-freedom (6DOF) tracking.

3. The apparatus according to claim 1, wherein, The first feature set includes a head-locked rendering mode, and the second feature set includes a world-locked rendering mode.

4. The apparatus according to claim 1, wherein, In order to render the first content, the processor is configured to render at least one first two-dimensional (2D) plane, and wherein the second rendered content includes at least one second 2D plane or a three-dimensional (3D) scene.

5. The apparatus according to claim 1, wherein, The processor is also configured to: The processor is configured to further receive the second rendered content based on the synchronization data in order to receive the second rendered content. This includes sending synchronization data for an application executed on the wearable display device.

6. The apparatus according to claim 1, wherein, The processor is also configured to: The wearable display device is paired with the companion device, wherein the processor is configured to receive an indication that the wearable display device will be switched to the companion mode after the pairing.

7. The apparatus according to claim 1, wherein, The processor is also configured to: The processor receives user input at the wearable display device, wherein, in order to obtain the indication that the wearable display device will be switched to the companion mode, the processor is configured to obtain the indication that the wearable display device will be switched to the companion mode based on the user input.

8. The apparatus according to claim 1, wherein, In order to process the first rendered content and the second rendered content, the processor is configured to perform at least one of the following: warping the first rendered content and the second rendered content, compositing the first rendered content and the second rendered content, or performing late reprojection (LSR) on the first rendered content and the second rendered content.

9. The apparatus according to claim 1, wherein, An application is executed at the wearable display device, and wherein, in order to render the first content, obtain the instruction, receive the second rendered content, process the first rendered content and the second rendered content, and configure the combined display buffer, the processor is configured to render the first content, obtain the instruction, receive the second rendered content, process the first rendered content and the second rendered content, and configure the combined display buffer without restarting or closing the application.

10. The apparatus according to claim 1, wherein, In order to render the first content based on the first set of features associated with the independent mode, the processor is configured to: Rendering third-party content in the first moment; and The fourth content is rendered at a second time, wherein the first time occurs before the instruction is received, and wherein the second time occurs after the instruction is received.

11. The apparatus according to claim 10, wherein, The processor is also configured to: Before receiving the instruction, output the third content for display; and After receiving the instruction, content for display is output, wherein the content is based on the second content and the fourth content.

12. The apparatus according to claim 1, wherein, The processor is also configured to: Following the processing, a second indication is obtained that the wearable display device will be switched to the standalone mode; and Based on the second instruction, render third content based on the first set of features associated with the independent mode.

13. The apparatus according to claim 1, wherein, When the wearable display device operates in the stand-alone mode, the wearable display device operates without exchanging data with the accompanying device, and wherein, when the wearable display device operates in the accompanying mode, the wearable display device exchanges the data with the accompanying device.

14. The apparatus according to claim 1, wherein, The processor is also configured to: Prior to receiving the instruction and while the wearable display device is operating in the stand-alone mode, data associated with the wearable display device is collected, wherein, in order to process the first rendered content and the second rendered content, the processor is further configured to: further process the first rendered content and the second rendered content based on the data.

15. The apparatus according to claim 14, wherein, The data includes sensory data.

16. The apparatus according to claim 1, wherein, The device is a wireless communication device, which further includes at least one of a transceiver or an antenna coupled to the processor, and wherein, in order to receive the second rendered content, the processor is configured to receive the second rendered content via at least one of the transceiver or the antenna.

17. A method for image processing, comprising: At the wearable display device, first content is rendered based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device; Obtain an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device; Based on the instruction, receive second rendered content from the accompanying device based on a second set of features associated with the accompanying mode; Process the first rendered content and the second rendered content; as well as Configure the combined display buffer based on the processed first rendered content and the processed second rendered content.

18. The method according to claim 17, wherein, The first set of features includes three-degree-of-freedom (3DOF) tracking, and the second set of features includes six-degree-of-freedom (6DOF) tracking.

19. The method of claim 17, wherein, The first feature set includes a head-locked rendering mode, and the second feature set includes a world-locked rendering mode.

20. A computer-readable medium storing computer-executable code, said computer-executable code, when executed by a processor, causing the processor to: At the wearable display device, first content is rendered based on a first set of features associated with an independent mode, the independent mode being associated with the wearable display device; Obtain an indication that the wearable display device will be switched to an accompanying mode associated with the wearable display device; Based on the instruction, receive second rendered content from the accompanying device based on a second set of features associated with the accompanying mode; Process the first rendered content and the second rendered content; as well as Configure the combined display buffer based on the processed first rendered content and the processed second rendered content.

Images