Dual-target wake-up time (TWT) window for splitting extended reality (XR)

Abstract

The aspects presented herein relate to methods and apparatuses for display processing, including devices such as a client or server. The apparatus can obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split XR process with a server. The apparatus can also calculate a first offset between a first rendering start time and a first TWT start time for a first TWT configuration, and a second offset between a second rendering start time and a second TWT start time for a second TWT configuration, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. The apparatus can also select either a first TWT configuration or a second TWT configuration for a split XR process with a client device.

Description

Cross-reference of related applications

[0001] This application claims the benefit of U.S. Non-Provisional Patent Application Serial No. 18 / 433,321, filed February 5, 2024, entitled “DUAL TARGET WAKE TIME (TWT) WINDOWS FOR SPLITEXTENDED REALITY (XR)”, the entire contents of which are expressly incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to processing systems, and more specifically, to one or more techniques for splitting extended reality (XR) applications. Background Technology

[0003] Computing devices typically perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones (such as smartphones), embedded systems, personal computers, tablet computers, and video game controllers. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output frames. A CPU controls the operation of a GPU by issuing one or more graphics processing commands to it. Modern CPUs are typically capable of executing multiple applications concurrently, each of which may require the GPU during execution. A display processor is configured to convert digital information received from the CPU into analog values ​​and can issue commands to a display panel to display visual content. Devices that provide content for visual presentation on a display may utilize a GPU and / or a display processor.

[0004] A display processor or display processing unit (DPU) can be configured to perform display processing procedures. However, with the advent of wireless communication and smaller handheld devices, the need for improved display processing is increasing. For example, current techniques for split rendering may not adequately address the target wake-up time (TWT) problem. Improved techniques for TWT in split rendering are needed. Summary of the Invention

[0005] The following is a simplified summary of one or more aspects to provide a basic understanding of these aspects. This summary is not a broad overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0006] In one aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be a client device, a server, a display processing unit (DPU), a graphics processing unit (GPU), or any apparatus capable of performing display processing. The apparatus may obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a server. The apparatus may also calculate, based on the first indication, (1) a first offset between a first rendering start time and a first TWT start time for a first target wake-up time (TWT) configuration, and (2) a second set of offsets between a second set of rendering start times and a second set of TWT start times for a second TWT configuration, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. Additionally, the apparatus may estimate a first motion-to-render-to-photon (m2r2p) latency for the first TWT configuration and a second m2r2p latency for the second TWT configuration based on the calculation of the first offset and the second set of offsets. The device can also calculate the difference between the first m2r2p latency configured in the first TWT and the second m2r2p latency configured in the second TWT; and determine whether the difference is less than an m2r2p threshold. The device can also adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, the device can estimate the first server processing time configured in the first TWT and the second server processing time configured in the second TWT based on the calculation of the first offset and the second offset set. The device can calculate the difference between the first server processing time configured in the first TWT and the second server processing time configured in the second TWT; and determine whether the difference is less than a server processing threshold. The device can also adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication to the server of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, the device can select either the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set. The device can output a second indication of the selection of the first TWT configuration or the second TWT configuration.The device can also communicate with the server via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0007] In another aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be a server, a client device, a display processing unit (DPU), a graphics processing unit (GPU), or any apparatus capable of performing display processing. The apparatus may compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with the client device. The apparatus may also output a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency for the split XR process with the client device. Additionally, the apparatus may, based on the first indication, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second set of offsets between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window, and the second TWT configuration includes two TWT windows. The device can also estimate a first motion-to-render-to-photon (m2r2p) latency for the first TWT configuration and a second m2r2p latency for the second TWT configuration based on the calculation of the first offset and the second offset set. The device can also calculate the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. Furthermore, the device can estimate a first server processing time for the first TWT configuration and a second server processing time for the second TWT configuration based on the calculation of the first offset and the second offset set. The device can calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold. The device can also adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication to the client device of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, the device can select either a first TWT configuration or a second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set. The device can output a second indication of the selection of the first TWT configuration or the second TWT configuration. The device can also communicate with the client device via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0008] Details of one or more examples of this disclosure are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of this disclosure will become apparent from the description, the drawings, and the claims. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the example content generation system.

[0010] Figure 2 An example graphics processing unit (GPU) is shown.

[0011] Figure 3 An example display frame including a display processor and a monitor is shown.

[0012] Figure 4 This is a diagram illustrating example communication of content / data based on the split rendering process.

[0013] Figure 5 This is a diagram illustrating an example timeline of the split rendering process.

[0014] Figure 6 This is a diagram illustrating an example of a split extended reality (XR) system.

[0015] Figure 7 This is a diagram illustrating an example timeline of the rendering process.

[0016] Figure 8 Includes a diagram illustrating an example timeline of the rendering process.

[0017] Figure 9 Includes a diagram illustrating an example timeline of the rendering process.

[0018] Figure 10 This is a diagram illustrating an example timeline of the rendering process.

[0019] Figure 11 Includes a diagram illustrating an example timeline of the rendering process.

[0020] Figure 12 Includes a graph illustrating example server processing time.

[0021] Figure 13 Includes a graph illustrating example server processing time.

[0022] Figure 14 Includes a graph illustrating example server processing time.

[0023] Figure 15 This is a graph illustrating an example of rendering time compared to motion to render to photon (m2r2p) time.

[0024] Figure 16 This is a communication flowchart illustrating an example of communication between a client device, a server, and a memory.

[0025] Figure 17 This is a flowchart showing an example of the processing method.

[0026] Figure 18 This is a flowchart showing an example of the processing method.

[0027] Figure 19 This is a communication flowchart illustrating an example of communication between a client device, a server, and a memory.

[0028] Figure 20 This is a flowchart showing an example of the processing method.

[0029] Figure 21 This is a flowchart showing an example of the processing method. Detailed Implementation

[0030] In some respects, transmitting a single pose within a TWT window can increase latency when using a TWT window. For example, transmitting a pose at the beginning or end of a TWT window can lead to increased latency. That is, uplink (UL) and downlink (DL) can be transmitted together within a TWT window, which can increase the latency between pose reception (Rx) and the start of rendering. Furthermore, if DL can be transmitted when the next TWT window opens, some additional latency may be added. Based on the above, it may be advantageous to transmit poses multiple times during a TWT window. For example, transmitting poses at the beginning and end of a TWT window may be advantageous. Furthermore, utilizing multiple TWT windows for pose transmission may be advantageous. Aspects of this disclosure can transmit poses multiple times during a TWT window. For example, aspects presented herein can transmit poses at the beginning and end of a TWT window.

[0031] The aspects of this disclosure may include several benefits or advantages. For example, the aspects presented herein may allow the renderer to select one of multiple poses at the beginning and end of a TWT window. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency. Additionally, the aspects of this disclosure may utilize multiple TWT windows for pose transmission. For example, the aspects of this disclosure may transmit one pose during a first TWT window and then another pose during a second TWT window. That is, the aspects presented herein may allow the renderer to select one pose within a first TWT window or another pose within a second TWT window. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency.

[0032] Various aspects of the systems, apparatuses, computer program products, and methods will be described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided to make this disclosure comprehensive and complete, and to fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of or in combination with other aspects of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover such apparatuses or methods implemented using structures, functionalities, or structures and functionalities other than or different from the various aspects of the disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.

[0033] Although various aspects are described herein, many variations and substitutions of these aspects fall within the scope of this disclosure. While some potential benefits and advantages of the aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transmission protocols, some of which are illustrated by way of example in the accompanying drawings and the description below. The detailed description and drawings are merely illustrative and not limiting of this disclosure, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0034] Several aspects are presented with reference to various apparatuses and methods. These apparatuses and methods are described in detail and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0035] For example, an element, any part of an element, or any combination of elements can be implemented as a “processing system” including one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic units, discrete hardware circuits, and other suitable hardware configured to perform the various functionalities described in this disclosure. One or more processors in a processing system can execute software. Software can be broadly interpreted as instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether expressed in terms of software, firmware, middleware, microcode, hardware description languages, or other terms. The term “application” can refer to software. As described herein, one or more technologies can refer to an application, i.e., software, configured to perform one or more functions. In such examples, the application may be stored on memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more technologies described herein. As an example, the hardware may access and execute code accessed from memory to perform one or more technologies described herein. In some examples, components are identified in this disclosure. In such examples, a component may be hardware, software, or a combination thereof. Each component may be a separate component or a subcomponent of a single component.

[0036] Therefore, in one or more examples described herein, the described functionality can be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality can be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media can be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disc storage devices, magnetic disk storage devices, other magnetic storage devices, combinations of computer-readable media of the types described above, or any other medium capable of being used to store computer-executable code in the form of instructions or data structures accessible to a computer.

[0037] Generally, this disclosure describes techniques for having a graphics processing pipeline in a single device or multiple devices, thereby improving the rendering of graphics content and / or reducing the load on processing units (i.e., any processing unit, such as a GPU, configured to perform one or more of the techniques described herein). For example, this disclosure describes techniques for performing graphics processing in any device that utilizes graphics processing. Other example benefits are described throughout this disclosure.

[0038] As used herein, instances of the term "content" can refer to "graphic content," "image," or vice versa. This is true regardless of whether these terms are used as adjectives, nouns, or other parts of speech. In some examples, as used herein, the term "graphic content" can refer to content produced by one or more processes in a graphics processing pipeline. In some examples, as used herein, the term "graphic content" can refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used herein, the term "graphic content" can refer to content produced by a graphics processing unit.

[0039] In some examples, as used herein, the term "display content" can refer to content generated by a processing unit configured to perform display processing. Graphical content can be processed to become display content. For example, a graphics processing unit can output graphical content (such as frames) to a buffer (which may be referred to as a frame buffer). A display processing unit can read graphical content (such as one or more frames) from the buffer and perform one or more display processing techniques on that display processing unit to generate display content. For example, a display processing unit can be configured to perform compositing on one or more rendering layers to generate frames. As another example, a display processing unit can be configured to composite, blend, or otherwise combine two or more layers into a single frame. A display processing unit can be configured to perform scaling on frames, such as zooming in or out. In some examples, a frame can refer to a layer. In other examples, a frame can refer to two or more layers that have been blended together to form the frame, i.e., the frame comprises two or more layers, and the frame comprising two or more layers can be subsequently blended.

[0040] Figure 1This is a block diagram illustrating an example content generation system 100 configured to implement one or more technologies of this disclosure. The content generation system 100 includes a device 104. Device 104 may include one or more components or circuitry for performing the various functions described herein. In some examples, one or more components of device 104 may be components of a System-on-a-Chip (SOC). Device 104 may include one or more components configured to perform one or more technologies of this disclosure. In the illustrated example, device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, device 104 may include multiple components, such as a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131. Reference to display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays. Display 131 may include a first display and a second display. The first display may be a left-eye display, and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation on the first and second displays. In other examples, the first and second displays may receive the same frames used for rendering on both displays. In further examples, the results of graphics processing may not be displayed on the devices; for example, the first and second displays may not receive any frames used for rendering on them. Instead, the frames or graphics processing results may be transferred to another device. In some respects, this can be referred to as split rendering.

[0041] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing, such as in a graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a display processor (such as display processor 127) to perform one or more display processing techniques on one or more frames generated by processing unit 120 prior to being rendered by one or more displays 131. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise render the frames processed by display processor 127. In some examples, one or more displays 131 may include one or more of the following: liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, projection display device, augmented reality display device, virtual reality display device, head-mounted display, or any other type of display device.

[0042] Memory (such as system memory 124) external to processing unit 120 and content encoder / decoder 122 may be accessible to processing unit 120 and content encoder / decoder 122. For example, processing unit 120 and content encoder / decoder 122 may be configured to read from and / or write to external memory (such as system memory 124). Processing unit 120 and content encoder / decoder 122 may be communicatively coupled to system memory 124 via a bus. In some examples, processing unit 120 and content encoder / decoder 122 may be communicatively coupled to each other via the bus or a different connection.

[0043] Content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store received encoded or decoded graphic content. Content encoder / decoder 122 can be configured to receive encoded or decoded graphic content from system memory 124 and / or communication interface 126, for example, in the form of encoded pixel data. Content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0044] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media or optical storage media or any other type of memory.

[0045] According to some examples, internal memory 121 or system memory 124 may be a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be construed as meaning that internal memory 121 or system memory 124 is immovable or that its contents are static. For example, system memory 124 may be removed from device 104 and moved to another device. Alternatively, system memory 124 may not be removable from device 104.

[0046] Processing unit 120 may be a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose GPU (GPGPU), or any other processing unit configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In some examples, processing unit 120 may reside on a graphics card mounted in a port on the motherboard of device 104, or may otherwise be incorporated into a peripheral device configured to interoperate with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, processing unit 120 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 121) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the above (including hardware, software, and combinations of hardware and software, etc.) can be considered as one or more processors.

[0047] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the content encoder / decoder 122 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 123) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0048] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any of the receiving functions described herein with respect to device 104. Additionally, the receiver 128 may be configured to receive information from another device, such as eye or head positioning information, rendering commands, or location information. The transmitter 130 may be configured to perform any of the transmitting functions described herein with respect to device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined to form a transceiver 132. In such an example, the transceiver 132 may be configured to perform any of the receiving and / or transmitting functions described herein with respect to device 104.

[0049] Refer again Figure 1In some aspects, the display processor 127 may include a TWT component 198 configured to obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a server. The TWT component 198 may also be configured to calculate, based on the first indication, (1) a first offset between a first rendering start time and a first TWT start time for a first target wake-up time (TWT) configuration, and (2) a second set of offsets between a second set of rendering start times and a second set of TWT start times for a second TWT configuration, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. The TWT component 198 may also be configured to estimate, based on the calculation of the first offset and the second set of offsets, a first motion-to-render-to-photon (m2r2p) latency for the first TWT configuration and a second m2r2p latency for the second TWT configuration. TWT component 198 may also be configured to calculate the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. TWT component 198 may also be configured to adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and to send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. TWT component 198 may also be configured to estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set. TWT component 198 may also be configured to calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold. TWT component 198 can also be configured to adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and to send an indication to the server of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. TWT component 198 can also be configured to select either a first TWT configuration or a second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set. TWT component 198 can also be configured to output a second indication of the selection of the first TWT configuration or the second TWT configuration.TWT component 198 can also be configured to communicate with the server via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0050] Refer again Figure 1In some aspects, the display processor 127 may include a TWT component 198 configured to compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with the client device. The TWT component 198 may also be configured to output a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency for the split XR process with the client device. The TWT component 198 may also be configured to calculate, based on the first indication, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. The TWT component 198 can also be configured to estimate a first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and a second m2r2p latency of the second TWT configuration based on the calculation of the first offset and the second offset set. The TWT component 198 can also be configured to calculate the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. The TWT component 198 can also be configured to estimate a first server processing time of the first TWT configuration and a second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set. The TWT component 198 can also be configured to calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold. TWT component 198 can also be configured to adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and to send an indication of the adjusted server processing threshold to the client device when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. TWT component 198 can also be configured to select either a first TWT configuration or a second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set. TWT component 198 can also be configured to output a second indication of the selection of the first TWT configuration or the second TWT configuration. TWT component 198 can also be configured to communicate with the client device via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.Although the following description may focus on display processing, the concepts described herein are applicable to other similar processing techniques.

[0051] As described herein, a device such as device 104 can refer to any device, apparatus, or system configured to perform one or more of the technologies described herein. For example, a device can be a server, base station, user equipment, client device, station, access point, computer (e.g., personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer), end product, apparatus, telephone, smartphone, server, video game platform or console, handheld device (e.g., portable video game device or personal digital assistant (PDA)), wearable computing device (e.g., smartwatch, augmented reality device, or virtual reality device), non-wearable device, display or display device, television, set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the technologies described herein. The processes described herein may be described as being performed by a specific component (e.g., GPU), but in further embodiments, other components consistent with the disclosed embodiments (e.g., CPU) may be used to perform them.

[0052] A GPU can process various types of data or data packets within its pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. Context register packets can be a collection of global state information, such as information about global registers, shaders, or constant data, which can regulate how the graphics context will be handled. For example, a context register packet may include information about the color format. In some aspects of a context register packet, there may be bits indicating which workload belongs to the context register. Furthermore, there may be multiple functions or programs running simultaneously and / or in parallel. For example, a function or program may describe an operation, such as a color mode or color format. Therefore, context registers can define multiple states of the GPU.

[0053] Context states can be used to determine how individual processing units (e.g., vertex extractors (VFDs), vertex shaders (VSs), shader processors, or geometry processors) operate and / or in which mode a processing unit operates. For this purpose, the GPU can use context registers and programming data. In some aspects, the GPU can generate workloads (e.g., vertex or pixel workloads) in the pipeline based on the context register definitions of modes or states. Certain processing units (e.g., VFDs) can use these states to determine certain functions, such as how to assemble vertices. Because these modes or states can change, the GPU may need to modify the corresponding context. Additionally, the workload corresponding to a mode or state may follow the changed mode or state.

[0054] Figure 2 Example GPU 200 is illustrated according to one or more technologies of this disclosure. For example... Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-process engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, a level 2 (L2) cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 shown includes processing units 220 to 238, but the GPU 200 may include multiple additional processing units. Additionally, processing units 220 to 238 are merely examples, and any combination or order of processing units may be used in the GPU according to this disclosure. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0055] like Figure 2 As shown, the GPU can use a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register groups (e.g., context register group 260) and / or draw call data groups (e.g., draw call group 212). Subsequently, CP 210 can transfer the context register group 260 or the draw call group 212 to a processing unit or block in the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can be constructed as follows: context register of context N, draw call of context N, context register of context N+1, and draw call of context N+1.

[0056] GPUs can render images in several different ways. In some cases, GPUs can use rendering and / or tiled rendering to render images. In a tiled rendering GPU, an image can be divided or segmented into different sections or tiles. After the image is divided, each section or tile can be rendered individually. A tiled rendering GPU can divide a computer graphics image into a grid format so that each part of the grid (i.e., a tile) is rendered individually. In some aspects, during a binning pass, the image can be divided into different bins or tiles. In some aspects, during a binning pass, a visibility stream can be constructed, where visible primitives or draw calls can be identified. In contrast to tiled rendering, direct rendering does not divide the frame into smaller bins or tiles. Instead, in direct rendering, the entire frame is rendered at once. Additionally, some types of GPUs allow both tiled rendering and direct rendering.

[0057] Figure 3 This is a block diagram 300 illustrating an example display frame including a processing unit 120, system memory 124, display processor 127, and display 131, as identified by incorporable device 104.

[0058] A GPU may be included in a device that provides content for visual presentation on a display. For example, processing unit 120 may include GPU 310 configured to render graphics data for display on a computing device (e.g., device 104), which may be a computer workstation, mobile phone, smartphone or other intelligent device, embedded system, personal computer, tablet computer, video game console, etc. The operation of GPU 310 may be controlled based on one or more graphics processing commands provided by CPU 315. CPU 315 may be configured to execute multiple applications concurrently. In some cases, each of the concurrently executing applications may utilize GPU 310 simultaneously. Processing techniques may be executed by outputting frames over a physical or wireless communication channel via processing unit 120.

[0059] System memory 124, executable by processing unit 120, may include user space 320 and kernel space 325. User space 320 (sometimes referred to as "application space") may include software applications and / or application frameworks. For example, software applications may include operating systems, media applications, graphics applications, workspace applications, etc. Application frameworks may include frameworks used by one or more software applications, such as libraries, services (e.g., display services, input services, etc.), application programming interfaces (APIs), etc. Kernel space 325 may further include display driver 330. Display driver 330 may be configured to control display processor 127. For example, display driver 330 may cause display processor 127 to synthesize frames and send data of the frames to the display.

[0060] Display processor 127 includes display control block 335 and display interface 340. Display processor 127 can be configured to (e.g., based on input received from display driver 330) manipulate the functions of display 131. Display control block 335 can also be configured to output image frames to display 131 via display interface 340. In some examples, display control block 335 may additionally or alternatively perform post-processing of image data provided based on the processing unit 120's execution of system memory 124.

[0061] Display interface 340 can be configured to cause display 131 to display image frames. Display interface 340 can output image data to display 131 according to an interface protocol, such as, for example, MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface) . That is, display 131 can be configured according to the MIPI DSI standard. The MIPI DSI standard supports video mode and command mode. In an example where display 131 operates in video mode, display processor 127 can continuously refresh the graphic content of display 131. For example, the entire graphic content can be refreshed in each refresh cycle (e.g., line by line). In an example where display 131 operates in command mode, display processor 127 can write the graphic content of a frame to buffer 350.

[0062] In some such examples, display processor 127 may not continuously refresh the graphics content of display 131. Instead, display processor 127 may use a vertical sync (Vsync) pulse to coordinate the rendering and consumption of graphics content at buffer 350. For example, when a Vsync pulse is generated, display processor 127 may output new graphics content to buffer 350. Thus, the generation of a Vsync pulse indicates that current graphics content has been rendered at buffer 350.

[0063] A frame is displayed on the monitor 131 based on the display controller 345, the display client 355, and the buffer 350. The display controller 345 can receive image data from the display interface 340 and store the received image data in the buffer 350. In some examples, the display controller 345 can output the image data stored in the buffer 350 to the display client 355. Therefore, the buffer 350 can represent local memory of the monitor 131. In some examples, the display controller 345 can directly output the image data received from the display interface 340 to the display client 355.

[0064] Display client 355 may be associated with a touch panel that senses interaction between the user and display 131. When the user interacts with display 131, one or more sensors in the touch panel may output signals to display controller 345 indicating which of the one or more sensors is active, the duration of the sensor activity, the pressure applied to the one or more sensors, etc. Display controller 345 may use the sensor outputs to determine how the user interacts with display 131. Display 131 may further be associated with / include other devices such as cameras, microphones, and / or speakers that operate in conjunction with display client 355.

[0065] Some processing techniques of device 104 can be performed through three stages (e.g., stage 1: rendering stage; stage 2: compositing stage; and stage 3: display / transfer stage). However, other processing techniques can combine the compositing stage and the display / transfer stage into a single stage, allowing the processing technique to be performed based on a total of two stages (e.g., stage 1: rendering stage; and stage 2: compositing / display / transfer stage). During the rendering stage, GPU 310 can process the content buffer based on the execution of the application that generates content on a pixel-by-pixel basis. During the compositing and display stages, pixel elements can be assembled to form a frame, which is then transferred to the physical display panel / subsystem (e.g., display 131) that displays the frame.

[0066] Instructions executed by the CPU (e.g., software instructions) or by the display processor can cause the CPU or display processor to search for and / or generate compositing strategies for compositing frames based on dynamic priorities and runtime statistics associated with one or more compositing strategy groups. A frame to be displayed by a physical display device (such as a display panel) may include multiple layers. Furthermore, frame compositing may be based on combining multiple layers into a frame (e.g., based on a frame buffer). After combining the multiple layers into a frame, the frame can be provided to the display panel for display on that display panel. The process of combining each of the multiple layers into a frame may be referred to as compositing, frame compositing, compositing process, compositing handling, etc.

[0067] A frame compositing process or strategy can correspond to a technique used to combine different layers from multiple layers into a single frame. Multiple layers can be stored in double data rate (DDR) memory. Each of the multiple layers can further correspond to a separate buffer. A compositor or hardware compositor (HWC) associated with a block or function determines the input to each layer / buffer and performs the frame compositing process to generate an output indicating the composite frame. That is, the input can be layers, and the output can be a frame compositing process used to synthesize the frame to be displayed on a display panel.

[0068] In some respects, the display device can present frames at different frame rates on the first and second display panels. For example, the display panels can present frames at 60 frames per second (FPS) on both the first and second display panels, or at 45 FPS on both the first and second display panels, and so on. The display device can synchronize the frame rate of the content with the refresh rate of the display panels (via a vertical synchronization process, which may be referred to as vsync, Vsync, VSync, or VSYNC). For example, the content may be available at 60 FPS, and the first and second display panels may have a refresh rate of 95 Hz. Through Vsync, the refresh rate of the first and second display panels can be set to 60 Hz to match the 60 FPS content.

[0069] As indicated in this article, VSync is a graphics technique that synchronizes the frame rate of an application / game with the refresh rate of the display (e.g., a monitor on a client device). VSync can be used as a way to handle screen tearing (i.e., sections of the screen displaying multiple frames at a time). This can cause the display to appear split along lines. Tearing can occur when the display refresh rate (i.e., how many times the display updates per second) is out of sync with the frames per second (FPS). VSync signals can synchronize the display pipeline (e.g., a pipeline that includes application rendering, compositors, and a hardware compositor (HWC) that renders the image on the display). For example, VSync signals can help synchronize the time when the application wakes up to begin rendering, the time when the compositor wakes up to composite the screen, and the display refresh cycle. This synchronization can help eliminate display refresh problems and improve visual performance. In some examples, the HWC can generate VSync events / signals and send these events / signals to the compositor.

[0070] In some aspects of graphics processing, content rendering can be performed at multiple locations and / or on multiple devices, for example, to divide the rendering workload among different devices. For instance, rendering can be split between a server and client devices; this can be referred to as "split rendering." In some cases, split rendering can be a method for bringing content to a client device, where a portion of the graphics processing can be performed outside the client device (e.g., at a server). In some aspects, the server can be at least one of the following: a telephone, a smartphone, a computer, or a cloud server. Furthermore, the client device can be at least one of the following: a headset, a head-mounted display (HMD), display glasses, or smart glasses.

[0071] Split rendering can be performed for multiple different types of applications, such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or extended reality (XR) applications. In VR applications, the content displayed on the client device may correspond to artificial or animated content. In XR, AR, or MR content, a portion of the content displayed on the client device may correspond to real-world content (e.g., real-world objects), and a portion of the content may be artificial or animated content. Moreover, artificial or animated content and real-world content can be displayed in optical or video perspective devices, allowing users to view real-world objects and artificial or animated content simultaneously. In some respects, artificial or animated content can be referred to as augmented content, and vice versa. Split rendering of XR, AR, or MR systems may also introduce latency in delivering the rendered content to the client display. In some respects, this latency may even be higher when rendering on a server compared to client rendering, but it can also enable more complex XR, AR, or MR applications. Furthermore, there may be a non-negligible latency between the time it takes to calculate the camera pose and the time it takes for the content to appear on the client display. For example, there may be a certain amount of latency in splitting XR, AR, or MR systems.

[0072] Figure 4 Diagram 400 illustrates content / data communication based on the split rendering process. For example... Figure 4 As shown, Figure 400 includes a server 410 and a client device 450 associated with the split rendering process. Figure 4 Multiple processes executed at server 410 and client device 450 are illustrated, including encoding process 420, packaging process 430, unpacking process 470, and decoding process 480. Server 410 and client device 450 also include sending component 440 and receiving component 460, respectively.

[0073] like Figure 4As shown, on server 410, data / content associated with an image / frame can be encoded during encoding process 420. After encoding process 420, the data / content can then undergo a packetization process 430, such as a Real-time Transport Protocol (RTP) packetization process. During the packetization process, the data / content can be converted into one or more frames 442. Frames 442 can then be sent from the sending component 440 of server 410 to the receiving component 460 of client device 450. In some cases, frames can be sent via User Datagram Protocol (UDP) Internet Protocol (IP) (UDP / IP) network protocol, Transmission Control Protocol (TCP) IP (TCP / IP) network protocol, or any other network protocol. On client device 450, frames 442 can be received via receiving component 460 (e.g., via UDP / IP network protocol, TCP / IP network protocol, or any other network protocol). Frames 442 can also undergo unpacking process 470 (e.g., a Real-time Transport Protocol (RTP) unpacking process or any other protocol unpacking process), which can convert packets into data / content. After unpacking, the data / content can be decoded during the decoding process 480. Finally, the decoded data / content can be transmitted to the display or HMD of the client device 450 for display.

[0074] As indicated above, various aspects of graphics processing can handle the rendering or display of different types of content, such as virtual reality (VR) applications, augmented reality (AR) applications, mixed reality (MR) applications, and / or extended reality (XR) applications. Content can be rendered or created on a server (e.g., a computer or telephone). To display this content, users can utilize different types of headsets or display glasses, which may be referred to as client devices. In some cases, when users want to use XR glasses for extended periods without the availability of charging facilities, it is desirable to conserve power at the server or client device. Furthermore, it is desirable to conserve power at the server or client device when the battery level of the client device or server becomes low (i.e., exceeds a threshold percentage determined by the user). Additionally, if users want to actively extend battery life, it is desirable to conserve power at the server or client device and provide longer battery life to either device.

[0075] In split-rendering applications, content can be rendered on a server and encoded / streamed via Wi-Fi to an XR-based HMD. As indicated above, split-rendering means distributing the XR workload between two devices (i.e., the host / server and the client / HMD). One use case could be a smartphone connected to an HMD / AR glasses device. AR glasses may not have high processing power, and heat dissipation could be an issue if all processing is performed on the client / glasses. Therefore, splitting rendering between the server and client devices is beneficial. In one aspect, poses (e.g., six degrees of freedom (6DOF) poses) can be generated on the client device. The client / HMD can then transmit the 6DOF pose data to the server via an uplink connection. The application or game can then use the transmitted 6DOF poses to render content on the server / phone. Furthermore, the encoding of the rendered content can occur on the server / phone. The encoded and compressed bitstream can then be sent from the server / phone to the HMD / client via a downlink connection. Afterward, video decoding and time-warping can be performed on the HMD / client using the latest 6DOF poses. Finally, the HMD / client can display the reprojected content.

[0076] Figure 5 Figure 500 illustrates an example timeline of the split rendering process. More specifically, Figure 5 A diagram 500 illustrates the timeline of different processing steps at a server (e.g., a phone, smartphone, or computer) and a client device (e.g., a headset, HMD, or smart glasses). For example, the client device can send multiple poses 510 (e.g., head poses) to the server. The server can then render the content of the frame at a rendering process 520 and encode the frame at an encoding process 530. Furthermore, the server can send the frame to the client device via a downlink (DL) 540. After receiving the frame, the client device can decode the frame at a decoding process 550. Figure 5 The vertical synchronization (VSync) 560 associated with each transmission is also shown.

[0077] like Figure 5As shown, head pose data can be sent from the client device to the server at a high rate (e.g., 500Hz) and / or low latency (via uplink (UL)). The client device (e.g., HMD / glasses) may be unaware of the server's (e.g., phone) rendering start time. The first frame can be rendered on the server at any time using the latest pose, followed by future frames rendered at a pre-configured frame rate per second (fps or FPS). The rendering thread can render frames as quickly as the GPU allows without any waiting time, and to limit fps, a waiting time can be added at the end of each rendering. Additionally, the rendering thread can sleep until the waiting time before the next frame's rendering begins. Furthermore, during rendering, each rendered frame can be immediately queued for encoding. Once encoding is complete, the encoded frames can be packaged and sent at any time (e.g., after the rendering time plus the encoding time) (via downlink (DL)). In some examples, the Wi-Fi modem can always be "on," allowing poses and frames to be sent with minimal latency. In addition, in some examples, multiple different types of information or data can be sent on the UL side (e.g., camera streaming data, color data or red (R) green (G) blue (B) (RGB) data, hand tracking data and / or three-dimensional (3D) rendering (3DR) data).

[0078] Split rendering can leverage several different features, such as Target Wake-up Time (TWT) and Timed Synchronization Function (TSF). The TWT feature allows the modem / radio frequency (RF) to connect at a fixed rhythm and for a known duration of service. This TWT feature can be used to save power on both server and client devices. While TWT ensures reduced power on the modem side, the choice of TWT parameters can affect XR performance, such as latency and frame multiplexing (i.e., jitter). In some cases, the TWT feature can allow UL (pose) data and DL (rendering + encoding) frame data to align with the TWT service period (on-time), meaning that transmission (Tx) and reception (Rx) on client devices and servers can occur continuously. When data is transmitted simultaneously within the same service period, the modem can be given the opportunity to sleep for a certain period (e.g., longer than a threshold duration), thus reducing power and thermal issues. In some split XR scenarios, each client can have a timer synchronized with the Timed Synchronization Function (TSF), which is associated with the server, for example. Additionally, early termination allows for the premature termination of a service period upon detection of inactivity in DL / UL data. The early termination process may include End of Service Period (EOSP). For example, ESOP can terminate a service interval (i.e., change the service interval from an "on" period to an "off" period). TWT allows modems to turn on and off at a defined pace. TWT may also support early termination, such as allowing a modem to turn off if no data is available for transmission within a defined time period.

[0079] As indicated herein, there may be multiple subsystems involved in an end-to-end XR pipeline. For example, CPUs, GPUs, encoders, decoders, networks, servers (e.g., smartphones), and / or client devices (e.g., headsets, HMDs, or AR glasses) may be involved in an end-to-end XR pipeline. Some types of client devices (e.g., wireless AR glasses) may require a sleek and lightweight design / form factor, which can cause a variety of different issues, such as battery consumption (e.g., approximately 800 mW for a system-on-a-chip (SOC) and double data rate (DDR) memory for some devices) and / or heat dissipation. Additionally, to achieve a high-quality user experience, servers and / or client devices may expect certain display characteristics or conditions. For example, for a high-quality user experience, a minimum amount of motion-to-render-to-photon (M2R2P) latency may be desired (i.e., m2r2p latency can be the waiting time between frame readiness and frame transmission). Furthermore, servers and / or client devices may expect a minimum amount of frame loss or duplication.

[0080] Figure 6Figure 600 illustrates an example of a split extended reality (XR) system 602 according to one or more techniques of this disclosure. As used herein, split XR can refer to an example of a client device (e.g., HMD) collaborating with a remote device (e.g., a server) to facilitate the display of graphical content on the client device's display, wherein the client device and the remote device can communicate with each other via wired and / or wireless communication. For example, a portion of a rendering task (or other task) may be offloaded to a server. The server may perform the rendering task (or other task). The server may send the output of the rendering task (or other task) to the client device. The client device may perform additional processing based on the received output in order to display graphical content. Split XR allows the client device to render relatively high-quality graphical content on its display while conserving the client device's battery life.

[0081] The split XR system 602 may include a server 604 and a client HMD 606. The server 604 may also be referred to as a companion device, a remote device, etc. In the example, the server 604 may be a cloud server, a desktop computing device (i.e., a personal computer (PC)), a server computing device, a game console, or a telephone. In the example, the server 604 may be or include device 104. The client HMD 606 may also be referred to as XR glasses, glasses, a client device, a client, an HMD, a wearable display device, etc. In the example, the client HMD 606 may be device 104. In the example, the client HMD 606 may be worn above / around / near one or more of the user's eyes. For example, the client HMD 606 may include a display positioned a few centimeters away from one or more of the user's eyes when the user wears the client HMD 606. The client HMD 606 may be able to present XR content to the user.

[0082] Server 604 and client HMD 606 can communicate with each other via wired means, and / or server 604 and client HMD 606 can communicate with each other wirelessly. In the example, server 604 and client HMD 606 can communicate with each other via a wireless local area network (WLAN) link, a cellular link (e.g., 5G New Radio (NR)), and / or Bluetooth. ™ The links (Bluetooth is a trademark of the Bluetooth Special Interest Group (SIG)) communicate with each other. The server 604 may have relatively greater computing power compared to the client HMD 606. For example, the server 604 may have a faster processor and / or more memory compared to the client HMD 606's processor and / or memory. Furthermore, the client HMD 606 may have a limited battery capacity, while the server 604 may not have a limited battery capacity.

[0083] In the split XR system 602, the client HMD 606 can send pose data 608 (i.e., HMD pose data, such as orientation information, hand / head pose information, etc.) to the server 604, where the pose data 608 is specific to the client HMD 606. The pose data 608 can be a six-degree-of-freedom (6DOF) pose, which includes the client HMD 606's position information (e.g., x, y, and z coordinates) and orientation information (e.g., tilt, pitch, and yaw). The client HMD 606 can also send other data to the server 604, such as state information of applications running on the client HMD 606. For example, the client HMD 606 can send perception data (e.g., hand tracking data, image tracking / object tracking (IT / OT) data, etc.) to the server 604. The client HMD 606 can also send camera video streams to the server 604.

[0084] Server 604 may render XR content based on pose data 608 (e.g., based on various poses). Server 604 may also additionally render XR content based on other data sent to server 604. Server 604 may compress the rendered XR content, encode the XR content, and send the rendered XR content (and the rendered pose) to client HMD 606 in an encoded bitstream 610. Furthermore, server 604 may extrapolate the rendering pose based on pose data 608, and the encoded bitstream 610 may also include an indication of the rendering pose. Client HMD 606 may receive and decompress the encoded bitstream 610 to obtain the rendered XR content. Client HMD 606 may warp the rendered XR content based on the client HMD 606's rendering pose and the latest display pose. Client HMD 606 may perform additional rendering on the rendered XR content to present the rendered XR content on the client HMD 606's display. Pose data 608 and encoded bit stream 610 can be associated with round-trip time 612. Round-trip time 612 can be the total time elapsed from the time pose data 608 is sent to server 604 to the time encoded bit stream 610 is received at client HMD 606.

[0085] In some respects, uplink and downlink communication can transmit data independently, so the modem may have to remain on for the entire duration, as it can receive packets at any time. The communication duration can be limited to a short period between the server and client, then the modem (at both the server and client) is in sleep mode for the remainder of the time period (considering a rendering cycle of frames per second (fps), the time period could be equal to 1 / fps). Uplink and downlink data can be transmitted continuously, allowing the modem to sleep for the remaining duration. The modem will only be on during the target wake-up time (TWT), and it can sleep at other times to conserve power. Therefore, the target wake-up time can allocate a fixed amount of time to the modem. That is, the TWT allows the modem to optimize the amount of power used. Higher end-to-end (e2e) latency (motion to render to photon (m2r2p) latency) can result from the constraints that may apply to the transmission of uplink and downlink data. Furthermore, the impact on latency may vary based on the rendering time spent. For example, when an application is required to render at 45 fps, the rendering time can vary between 6 ms and 20 ms depending on the content complexity. To meet the Time-to-Wake (TWT), rendering should begin at an offset relative to the start of the TWT, thus satisfying the target wake-up time.

[0086] Figure 7 This is diagram 700, illustrating an example timeline of the rendering process. More specifically, diagram 700 depicts a timeline of a split rendering process with varying rendering times. Figure 7 As shown, Figure 700 includes pose reception (Rx) 712, pose Rx 722, rendering start time of frame 701 714, rendering start time of frame 702 724, encoding end time 716, encoding end time 726, downlink transmission start time 718 and downlink transmission start time 728. Figure 7 The diagram depicts a time interval (1 / fps) between the rendering start time 714 of frame 701 and the rendering start time 724 of frame 702. There is a small delay between pose Rx 712 and the rendering start time 714. Furthermore, there is a large rendering and encoding time between the rendering start time 714 of frame 701 and the encoding end time 716. Additionally, there is a smaller rendering and encoding time between the rendering start time 724 and the encoding end time 726 of frame 702. Moreover, the downlink transmission start time 728 will begin immediately at the encoding end time 726.

[0087] Figure 7 The effect of different rendering times (V0) on the time-to-weighted output (TWT) of m2r2p is depicted. For example... Figure 7As shown, since the pose (part of the UL) can be transmitted independently of the DL, the pose Rx can be synchronized to minimize the latency between the pose Rx and the start of rendering. For example, the DL can be sent once encoding is complete, thus eliminating any additional latency. Figure 7 In the design, because the modem is always in listening mode, there is a considerable power impact.

[0088] Figure 8 This includes diagrams 800 and 850, which illustrate example timelines of the rendering process. More specifically, diagrams 800 and 850 depict timelines of split rendering processes with different rendering times and target wake-up times. Figure 8 As shown, Figure 800 includes pose reception (Rx) 812, pose Rx 822, rendering start time 814 for frame 801, rendering start time 824 for frame 802, rendering start time 834 for frame 803, encoding time end 816, encoding time end 826, downlink transmission start time 818, downlink transmission start time 828, and TWT window 820. Figure 800 depicts a 22.22 ms time interval between the rendering start time 814 of frame 801 and the rendering start time 824 of frame 802, and between the rendering start time 824 of frame 802 and the rendering start time 834 of frame 803. There is a DL wait time between the encoding time end 816 and pose Rx 812. Furthermore, there is a DL wait time between the encoding time end 826 and pose Rx 822. Additionally, there is a large delay between the pose Rx 812 and the rendering start time 834 of frame 803. In addition, the TWT window 820 includes pose Rx 812 and downlink transmission start time 818, as well as pose Rx 822 and downlink transmission start time 828.

[0089] Figure 800 depicts the default TWT design, where the pose is transmitted first, followed by the downlink. For example, the uplink (UL) and downlink (DL) may be transmitted together within a TWT window 820, thus increasing the pose Rx to rendering start latency. When the next TWT window 820 opens, the DL can be sent, and therefore, some additional latency may exist. Furthermore, Figure 800 illustrates that rendering may begin before or after a TWT window 820. Figure 850 depicts a small latency between pose Rx and rendering start time. Additionally, there is a DL wait before transmission. Figure 850 depicts rendering that may begin between TWT windows 820. Therefore, during runtime, if there is a switch between transmitting the pose before or after the DL, the TWT window may need to be renegotiated, at the cost of some latency.

[0090] Figure 8The effect of TWT on pose transmission and rendering latency is illustrated. For example, when using a TWT window, transmitting a single pose within the TWT window can increase latency. For instance, transmitting a pose at the beginning or end of the TWT window can lead to increased latency (e.g., m2r2p latency). That is, UL and DL can be transmitted together within the TWT window, which can increase the latency between pose Rx and the start of rendering. Furthermore, if DL can be transmitted when the next TWT window opens, some additional latency may be added. Based on the above, it may be beneficial to transmit poses multiple times during the TWT window. For example, transmitting poses at the beginning and end of the TWT window may be beneficial. Furthermore, utilizing multiple TWT windows for pose transmission may be advantageous.

[0091] The aspects of this disclosure can transmit poses multiple times during a TWT window. For example, the aspects presented herein can transmit poses at the beginning and end of a TWT window. That is, the aspects presented herein allow the renderer to select one of multiple poses at the beginning and end of a TWT window. By doing so, the aspects presented herein allow the renderer to select the latest pose, which can result in reduced latency. Additionally, the aspects of this disclosure can utilize multiple TWT windows for pose transmission. For example, the aspects of this disclosure can transmit one pose during a first TWT window and then another pose during a second TWT window. That is, the aspects presented herein allow the renderer to select one pose within a first TWT window or another pose within a second TWT window. By doing so, the aspects presented herein allow the renderer to select the latest pose, which can result in reduced latency.

[0092] Figure 9 This includes diagrams 900 and 950 illustrating example timelines of the rendering process. More specifically, diagrams 900 and 950 depict timelines of split rendering processes with different rendering times and target wake-up times. Figure 9As shown, Figure 900 includes pose reception (Rx) 912, pose Rx 922, rendering start time 914 for frame 901, rendering start time 924 for frame 902, rendering start time 934 for frame 903, encoding time end 916, encoding time end 926, downlink transmission start time 918, downlink transmission start time 928, and TWT window 920. Figure 900 depicts a time interval of 22.22 ms between the rendering start time 914 of frame 901 and the rendering start time 924 of frame 902, and between the rendering start time 924 of frame 902 and the rendering start time 934 of frame 903. There is a DL wait time between the encoding time end 916 and pose Rx 912. Furthermore, there is a DL wait time between the encoding time end 926 and pose Rx 922. Additionally, there is a small delay between the pose Rx 912 of frame 903 and the rendering start time 934. Furthermore, TWT window 920 includes two instances of pose Rx 912 and a downlink transmission start time 918. For example, pose Rx 912 is located at the beginning and end of TWT window 920. Additionally, TWT window 920 includes two instances of pose Rx 922 and a downlink transmission start time 928. For example, pose Rx 922 is located at the beginning and end of TWT window 920.

[0093] Figure 900 illustrates a TWT design where a pose Rx 912 is transmitted at the beginning of a TWT window 920, followed by a downlink, and then another pose Rx 912 is transmitted at the end of the TWT window 920. For example, multiple poses Rx 912 can be transmitted together within the TWT window 920, thus reducing latency. That is, the aspects presented herein allow the renderer to select one pose within a first TWT window or another pose within a second TWT window. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency. DL can be transmitted between poses Rx 912 within the TWT window 920. Figure 900 illustrates a scenario where a pose can be transmitted, then a DL, and then another pose can be transmitted, allowing the renderer to select the most recent pose, which reduces latency. The renderer can begin before or after the TWT window. Figure 900 illustrates that UL and DL can be transmitted together within the TWT window 920. In addition, DL can be sent when the next TWT window opens.

[0094] Figure 950 illustrates that rendering can begin between TWT windows 920. For example, a TWT window may include two instances of pose Rx 912, a downlink transmission start time 918, and a rendering start time 924. For example, pose Rx 912 may be transmitted at the beginning and end of TWT windows 920. There is a small delay between pose Rx 912 in frame 902 and the rendering start time 924. There is also a DL wait time between the end of encoding time 926 and pose Rx 922. As shown in Figure 950, by transmitting the pose at the beginning and end of the TWT window, the aspects presented herein achieve optimal pose Rx to rendering start delay, regardless of where rendering begins. If the pose is transmitted only at the beginning of TWT window 920, the delay can be larger. However, transmitting the pose at the beginning and end of TWT window 920 reduces the delay. That is, Figure 9 The aspects presented herein illustrate how the renderer can select one pose or another pose within the TWT window 920. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency. In fact, transmitting the pose at the beginning and end of the TWT window 920 reduces latency because the TWT window can be aligned close to the start of rendering, thus reducing the pose receiver's rendering start latency. Figure 9 The effect of TWT on M2R2P under different rendering times is described.

[0095] Figure 10 Diagram 1000 includes an example timeline illustrating the rendering process. More specifically, diagram 1000 depicts a timeline of a split rendering process with varying rendering times and target wake-up times. Figure 10As shown, Figure 1000 includes pose reception (Rx) 1012, pose Rx 1022, rendering start time 1014 for frame 1001, rendering start time 1024 for frame 1002, rendering start time 1034 for frame 1003, encoding time end 1016, encoding time end 1026, downlink transmission start time 1018, downlink transmission start time 1028, and TWT window 1020. Figure 1000 depicts a time interval of 22.22 ms between the rendering start time 1014 of frame 1001 and the rendering start time 1024 of frame 1002, and between the rendering start time 1024 of frame 1002 and the rendering start time 1034 of frame 1003. There is a DL wait time between the encoding time end 1016 and pose Rx 1012. Furthermore, there is a DL wait time between the encoding time end 1026 and pose Rx 1022. There is a small delay between pose Rx 1012 in frame 1002 and rendering start time 1024. Furthermore, there is a small delay between pose Rx 1022 in frame 1003 and rendering start time 1034. Additionally, there are multiple instances of TWT window 1020, each instance including an instance of pose Rx 1012 and a downlink transmission start time 1018 in one instance of TWT window 1020. For example, pose Rx 1012 is at a first instance of TWT window 1020, and pose Rx 1012 is at a second instance of TWT window 1020. Furthermore, there are multiple instances of TWT window 1020, each instance including an instance of pose Rx 1022 and a downlink transmission start time 1028 in one instance of TWT window 1020. For example, pose Rx 1022 is at a first instance of TWT window 1020, and pose Rx 1022 is at a second instance of TWT window 1020.

[0096] Figure 10 This paper describes how the aspects presented herein can transmit pose and deep learning (DL) in one TWT window 1020, and another pose in another TWT window 1020. By transmitting pose in two TWT windows, the aspects presented herein can minimize rendering start latency. The first TWT window 1020 can be aligned with the start of rendering, minimizing the time from pose reception to rendering start. The second TWT window is aligned with the start of DL to reduce DL latency. Furthermore, since the UL has a separate TWT window 1020, the aspects presented herein can synchronize this window to the start of rendering and minimize the latency between pose Rx time and rendering start time. Figure 10As shown, by transferring poses across multiple TWT windows, the aspects presented in this paper achieve optimal pose Rx to render start latency, regardless of where rendering begins. If poses are transferred only within a single TWT window 1020, the latency can be significant. However, transferring poses across multiple instances of TWT window 1020 reduces latency. That is, Figure 10 The aspects presented herein illustrate how the renderer can select one pose within a TWT window 1020 or another pose within another TWT window 1020. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency. In fact, transmitting poses across multiple instances of TWT window 1020 reduces latency because the TWT windows can be aligned close to the start of rendering, thus reducing the pose receiver's rendering start latency. Figure 10 The impact of TWT on M2R2P under different rendering times was also described.

[0097] In some aspects, both the pose Rx to render start time and the time before the encoded frame is ready (rendered frame) for transmission can be optimized for overall M2R2P optimization. The aspects presented in this paper can optimize the impact of various server processing times (rendering + encoding + socket write time) on M2R2P. Furthermore, the aspects presented in this paper can optimize M2R2P by jointly optimizing the pose Rx to render time and the DL wait time (frame ready to frame Tx). That is, if the pose Rx to render time and DL wait time are jointly optimized, the optimal point may depend on both the pose Rx to render start time and the DL wait time. In some instances, the aspects presented in this paper may involve power optimization using the TWT for splitting XR. For example, the aspects presented in this paper can optimize the processing time between the server and client, then allow the modem to remain in sleep mode for the remaining time period. To satisfy the TWT, the aspects presented in this paper may propose that rendering should begin at a certain offset relative to the start of the TWT.

[0098] Figure 11 Examples of example timelines for the rendering process are shown in Figures 1100, 1130, and 1160. More specifically, Figures 1100, 1130, and 1160 depict timelines of split rendering processes with different rendering times and target wake-up times. Figure 11As shown, Figures 1100, 1130, and 1160 include the rendering start time 1114 of frame 1101, the rendering start time 1124 of frame 1102, the rendering start time 1134 of frame 1103, TWT 1116, and TWT 1126. Figures 1100, 1130, and 1160 depict a 22.22 ms time interval between the rendering start time 1114 of frame 1101 and the rendering start time 1124 of frame 1102, and between the rendering start time 1124 of frame 1102 and the rendering start time 1134 of frame 1103. Figure 1100 depicts the server processing time distribution without any offset. Figure 1130 depicts a server processing time distribution with an offset between the rendering start time and the TWT start time of 65%. Therefore, 65% of DL frames can have low latency, while 35% of DL frames can have high latency. Figure 1160 depicts an offset between the rendering start time and the TWT start time that is 95% of the server processing time distribution. Therefore, 95% of DL frames can have low latency, while 5% of DL frames can have high latency. Figure 1160 also shows the high pose Rx to the rendering start time.

[0099] Figure 11 The effects of choosing different percentiles for the offset between the render start time and the TWT start time are depicted. For example... Figure 11 As shown, the aspects presented in this paper can determine the optimal points for minimizing pose Rx to render start + DL wait, in order to achieve optimal latency for any TWT system design. Additionally, Figure 11 The paper illustrates the offset between the selectable render start and TWT start for the aspects presented, enabling them to minimize pose reception to render start latency and DL wait latency. Therefore, the aspects presented attempt to minimize both of these latencies. This is achieved by observing server processing latency and channel characteristics, and then determining the optimal point based on these latencies.

[0100] Figure 12 Includes graphs 1200 and 1220 illustrating examples of server processing time. More specifically, graph 1200 depicts the pose Rx to rendering start time 1210, and graph 1220 depicts the m2r2p time 1230 with a TWT window. Figure 12As shown, graphs 1200 and 1220 include percentiles on the x-axis for starting the TWT window and time in milliseconds on the y-axis. Graph 1200 depicts the pose Rx to render start time 1211 and pose Rx to render start time 1212. Graph 1220 depicts server m2r2p 1231 with TWT mean, pose Rx to render start time 1232, and DL ready to Tx start time 1233. Figure 12 This paper demonstrates that by observing server processing latency and channel characteristics, the aspects presented can be optimized based on these latency factors (e.g., the optimization on a server m2r2p1231 with a mean TWT). Furthermore, the aspects presented can be used to optimize m2r2p by jointly optimizing pose Rx to rendering time and DL latency (i.e., frame ready to frame Tx time). Figure 12 It is shown that M2R2P can be optimized through joint optimization, compared to M2R2P without joint optimization.

[0101] Figure 13 Includes graphs 1300 and 1320 illustrating examples of server processing time. More specifically, graph 1300 depicts the pose Rx to rendering start time 1310, and graph 1320 depicts the m2r2p time 1330 with a TWT window. Figure 13 As shown, graphs 1300 and 1320 include percentiles on the x-axis for starting the TWT window and time in milliseconds on the y-axis. Graph 1300 depicts the time from pose 1301 Rx to render start time 1311, and pose 1302 Rx to render start time 1312. Graph 1320 depicts the server m2r2p with mean TWT 1331, pose Rx to render start time 1332, and DL ready to Tx start time 1333. Figure 13 Two poses are shown that can help reduce latency and achieve optimal pose Rx to render start latency, regardless of when rendering begins.

[0102] Additionally, Figure 13 This paper describes how the various aspects presented can be optimized for m2r2p by jointly optimizing the pose Rx to rendering time and DL wait time (i.e., frame ready to frame Tx time). Furthermore, Figure 13 It is shown that M2R2P can be optimized through joint optimization, compared to M2R2P without joint optimization.

[0103] Figure 14Includes graphs 1400 and 1420 illustrating examples of server processing time. More specifically, graph 1400 depicts the pose Rx to render start time 1410, and graph 1420 depicts the m2r2p time 1430 with a TWT window. Figure 14 As shown, graphs 1400 and 1420 include percentiles on the x-axis for starting the TWT window and time in milliseconds on the y-axis. Graph 1400 depicts the time from pose 1411 Rx to render start time, and from pose 1412 Rx to render start time. Graph 1420 depicts the time from pose Rx to render start time 1431, server m2r2p with average TWT 1432, server m2r2p with average TWT of pose 1411, server m2r2p with average TWT of pose 1412, and DL ready to Tx start time 1435. Figure 14 Two poses are shown that can help reduce latency and jitter (e.g., interference at the network), such as latency with varying characteristics (e.g., network latency). By doing so, the aspects presented in this paper achieve optimal pose Rx to render start latency, regardless of when rendering begins. Furthermore, the aspects presented in this paper can optimize m2r2p by jointly optimizing pose Rx to render time and DL wait time (i.e., frame ready to frame Tx time).

[0104] Figure 15 This is a graph 1500 illustrating an example of rendering time compared to motion-to-render-to-photon (m2r2p) time. More specifically, graph 1500 depicts the impact of rendering time on m2r2p under different TWT system designs. Figure 15 As shown, graph 1500 includes a first TWT configuration 1510 and a second TWT configuration 1520. Graph 1500 includes rendering time in milliseconds on the x-axis and m2r2p estimation in milliseconds on the y-axis. Figure 15 The various aspects presented herein are described as being dynamically selectable between the first TWT configuration 1510 and the second TWT configuration 1520 in order to achieve ideal latency. Figure 15The paper describes the various aspects of latency and power that can be optimized. In fact, both the first TWT configuration 1510 and the second TWT configuration 1520 utilize low latency and low power. For example, in the first TWT configuration 1510, the power may be equal to a certain power amount (e.g., X + Y mW). Furthermore, in the second TWT configuration 1520, the power may be equal to another power amount (e.g., X + Z mW). A TWT configuration can refer to a configuration associated with a TWT. For example, a TWT configuration can refer to a TWT window or a pose within a TWT window. For example, selecting the first TWT configuration or the second TWT configuration can refer to selecting one pose within the first TWT window or another pose within the second TWT window.

[0105] Based on the varying rendering time of the application (e.g., from 6ms to 20ms for a 45 fps use case), the aspects presented in this paper determine the number of TWT windows that need to be used in the cycle. According to simulations, the aspects presented in this paper determine that two TWT windows provide the greatest benefit in terms of latency depending on the rendering time (e.g., when the rendering time is between 11ms and 15ms). When the rendering time is not a specific duration (e.g., between 11ms and 15ms), the aspects presented in this paper can use a single TWT window. When the aspects presented in this paper use a single TWT window, the number of poses to Tx can be determined based on the rendering time. When the aspects presented in this paper use two TWT windows, the aspects presented in this paper can transmit poses and DL in the first window and another pose in the second window. Furthermore, when the aspects presented in this paper use two TWT windows, the aspects presented in this paper can transmit poses, UL camera frames, and DL in the first window and another pose in the second window. The aspects presented herein can also transmit pose and DL in the first window, and another pose and UL camera frame in the second window. The offset between the rendering and the first TWT window can be determined based on optimization. Furthermore, the offset between the first and second TWT windows can be determined based on the pose Rx, either with render start latency or UL camera latency. When the aspects presented herein use a single TWT window, the aspects can transmit pose and UL at the beginning of the TWT window, then DL, and then another pose at the end of the TWT window. The aspects presented herein can determine that the system design may have similar latency during a certain rendering time period (e.g., when the rendering time is approximately 7ms to 10ms). This can indicate that the pose at the beginning of the window should be used when the rendering latency is low (less than 10ms). Furthermore, the aspects presented herein can determine that some designs have improvements during a certain rendering time period (e.g., when the rendering time is between 10ms and 20ms). This can indicate that the pose at the end of the window should be used when the rendering latency is high (greater than 10ms).

[0106] The aspects of this disclosure may include several benefits or advantages. For example, the aspects presented herein may allow the renderer to select one of multiple poses at the beginning and end of a TWT window. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency. Additionally, the aspects of this disclosure may utilize multiple TWT windows for pose transmission. For example, the aspects of this disclosure may transmit one pose during a first TWT window and then another pose during a second TWT window. That is, the aspects presented herein may allow the renderer to select one pose within a first TWT window or another pose within a second TWT window. By doing so, the aspects presented herein allow the renderer to select the most recent pose, which can result in reduced latency.

[0107] Figure 16 This is a communication flowchart 1600 illustrating processing according to one or more techniques of this disclosure. For example... Figure 16 As shown, Figure 1600 includes example communication between a client device 1602 (e.g., a client, headset, HMD, AR glasses, server, telephone, or smartphone), a server 1604 (e.g., a server, telephone, smartphone, client, headset, HMD, or AR glasses), and a memory 1606 (e.g., a memory or cache) according to one or more technologies of this disclosure.

[0108] At 1610, client device 1602 may obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with the server (e.g., client device 1602 may obtain indication 1612 from server 1604). In some aspects, obtaining a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency may include receiving a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency from the server or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device. Furthermore, the set of rendering latency may be a set of rendering statistics associated with the latency of the split XR process of the server, the set of encoding latency may be a set of encoding statistics associated with the latency of the split XR process of the server, and the set of network latency may be a set of network statistics associated with the latency of the split XR process of the server.

[0109] At 1620, client device 1602 may, based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second set of offsets between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. In some aspects, calculating the first offset between the first rendering start time configured for the first TWT and the first TWT start time may include calculating the difference between the first rendering start time and the first TWT start time. Furthermore, calculating the second set of offsets between the second set of rendering start times configured for the second TWT and the second set of TWT start times may include: calculating the difference between an initial second rendering start time and an initial second TWT start time; and calculating the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time. In addition, calculating the second offset set between the second set of rendering start times for the second TWT configuration and the second set of TWT start times may include: calculating the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0110] At 1630, client device 1602 may estimate the first motion-to-render-to-photon (m2r2p) delay of the first TWT configuration and the second m2r2p delay of the second TWT configuration based on the calculation of the first offset and the second offset set.

[0111] Furthermore, at 1630, the client device 1602 can calculate the difference between the first m2r2p delay of the first TWT configuration and the second m2r2p delay of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. In some aspects, selecting the first TWT configuration or the second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination that the difference is less than the m2r2p threshold. Furthermore, the m2r2p threshold may be associated with the power level at the client device or the battery level at the client device.

[0112] At 1640, client device 1602 may adjust the m2r2p threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold; and send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold.

[0113] At 1650, client device 1602 may estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set.

[0114] Furthermore, at 1650, the client device 1602 can calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold. In some aspects, selecting the first TWT configuration or the second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination that the difference is less than the server processing threshold. Furthermore, the processing threshold may be associated with the power level at the client device or the battery level at the client device.

[0115] At 1660, client device 1602 may adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an instruction to the server on the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold.

[0116] At 1670, client device 1602 may select the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set.

[0117] At 1680, client device 1602 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. In some aspects, outputting the second indication of the selection of the first TWT configuration or the second TWT configuration may include sending the second indication of the selection of the first TWT configuration or the second TWT configuration to a modem at a server (e.g., client device 1602 may send indication 1682 to server 1604). Furthermore, outputting the second indication of the selection of the first TWT configuration or the second TWT configuration may include storing the second indication of the selection of the first TWT configuration or the second TWT configuration in a modem at the client device (e.g., client device 1602 may store indication 1684 in memory 1606).

[0118] Furthermore, at 1680, client device 1602 can communicate with the server via a split XR process based on a selection of a first TWT configuration or a second TWT configuration.

[0119] Figure 17This is a flowchart 1700 of an example method for display processing according to one or more technologies of this disclosure. The method may be performed by a client device, a headset, an HMD, AR glasses, a server, a telephone, a smartphone, a DPU (or other display processor), a CPU (or other central processing unit), a DPU driver, a DDIC, a GPU (or other graphics processing unit), a means for display processing, a wireless communication device, and / or an apparatus capable of performing such a combination. Figures 1 to 16 The example uses any device for frame processing to perform.

[0120] At 1702, the client device may obtain a first indication of at least one of the following: a set of rendering latency, a set of encoding latency, or a set of network latency used in the split extended reality (XR) process with the server, such as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1610, client device 1602 may obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with the server. Furthermore, step 1702 may be... Figure 1 The display processor 127 in the display process executes the operation. In some aspects, obtaining a first indication of at least one of the rendering latency set, encoding latency set, or network latency set may include receiving a first indication of at least one of the rendering latency set, encoding latency set, or network latency set from a server or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between a client device and a server, or (2) at least one split XR application running on a client device. Additionally, the rendering latency set may be a set of rendering statistics associated with the latency of the split XR process on the server, the encoding latency set may be a set of encoding statistics associated with the latency of the split XR process on the server, and the network latency set may be a set of network statistics associated with the latency of the split XR process on the server.

[0121] At 1704, the client device can calculate, based on the first instruction, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows, as combined Figures 1 to 16 The examples described in the document. For example, as... Figure 16As described in 1620, client device 1602 can calculate, based on the first instruction, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. Furthermore, step 1704 can be... Figure 1 The display processor 127 performs the operation. In some aspects, calculating the first offset between the first rendering start time and the first TWT start time for the first TWT configuration may include calculating the difference between the first rendering start time and the first TWT start time. Furthermore, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the difference between the initial second rendering start time and the initial second TWT start time; and calculating the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time. Additionally, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0122] At 1714, the client device can select either the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set, such as in combination. Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1670, client device 1602 may select the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set. Furthermore, step 1714 may be... Figure 1 The display processor 127 in the middle is executed.

[0123] At 1716, the client device can output a second indication of the selection of the first TWT configuration or the second TWT configuration, as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1680, client device 1602 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. Furthermore, step 1716 may be... Figure 1The display processor 127 executes the operation. In some aspects, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include sending a second indication of the selection of a first TWT configuration or a second TWT configuration to a modem at the server. Furthermore, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include storing the second indication of the selection of a first TWT configuration or a second TWT configuration in a modem at a client device.

[0124] Furthermore, at point 1716, the client device can communicate with the server via a split XR process based on the selection of a first TWT configuration or a second TWT configuration, such as in combination with... Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1680, client device 1602 can communicate with the server via a split XR process based on a selection of a first TWT configuration or a second TWT configuration. Furthermore, step 1716 can be... Figure 1 The display processor 127 in the middle is executed.

[0125] Figure 18 This is a flowchart 1800 of an example method for display processing according to one or more technologies disclosed herein. The method may be performed by a client device, a headset, an HMD, AR glasses, a server, a telephone, a smartphone, a DPU (or other display processor), a CPU (or other central processing unit), a DPU driver, a DDIC, a GPU (or other graphics processing unit), a means for display processing, a wireless communication device, and / or an apparatus capable of performing such a combination. Figures 1 to 16 The example uses any device for frame processing to perform.

[0126] At 1802, the client device may obtain a first indication of at least one of the following: a set of rendering latency, a set of encoding latency, or a set of network latency used in the split extended reality (XR) process with the server, such as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1610, client device 1602 may obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with the server. Furthermore, step 1802 may be performed by… Figure 1The display processor 127 in the display process executes the operation. In some aspects, obtaining a first indication of at least one of the rendering latency set, encoding latency set, or network latency set may include receiving a first indication of at least one of the rendering latency set, encoding latency set, or network latency set from a server or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between a client device and a server, or (2) at least one split XR application running on a client device. Additionally, the rendering latency set may be a set of rendering statistics associated with the latency of the split XR process on the server, the encoding latency set may be a set of encoding statistics associated with the latency of the split XR process on the server, and the network latency set may be a set of network statistics associated with the latency of the split XR process on the server.

[0127] At 1804, the client device may, based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows, as combined Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1620, client device 1602 may calculate, based on the first instruction, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. Furthermore, step 1804 may be... Figure 1The display processor 127 performs the operation. In some aspects, calculating the first offset between the first rendering start time and the first TWT start time for the first TWT configuration may include calculating the difference between the first rendering start time and the first TWT start time. Furthermore, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the difference between the initial second rendering start time and the initial second TWT start time; and calculating the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time. Additionally, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0128] At 1806, the client device can estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration based on this calculation of the first offset and the second offset set, as combined with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1630, client device 1602 can estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration based on this calculation of the first offset and the second offset set. Furthermore, step 1806 can be... Figure 1 The display processor 127 in the middle is executed.

[0129] Furthermore, at 1806, the client device can calculate the difference between the first m2r2p latency configured in the first TWT and the second m2r2p latency configured in the second TWT; and determine whether the difference is less than an m2r2p threshold, such as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1630, the client device 1602 can calculate the difference between the first m2r2p latency configured in the first TWT and the second m2r2p latency configured in the second TWT; and determine whether the difference is less than an m2r2p threshold. Furthermore, step 1806 can be performed by... Figure 1 The display processor 127 performs the operation. In some aspects, selecting a first TWT configuration or a second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the m2r2p threshold. Furthermore, the m2r2p threshold may be associated with the power level at the client device or the battery level at the client device.

[0130] At 1808, the client device can adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold, as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1640, client device 1602 can adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, step 1808 can be performed by... Figure 1 The display processor 127 in the middle is executed.

[0131] At 1810, the client device can estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on this calculation of the first offset and the second offset set, such as in combination. Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1650, client device 1602 can estimate the first server processing time for the first TWT configuration and the second server processing time for the second TWT configuration based on this calculation of the first offset and the second offset set. Furthermore, step 1810 can be... Figure 1 The display processor 127 in the middle is executed.

[0132] Furthermore, at 1810, the client device can calculate the difference between the first server processing time configured in the first TWT and the second server processing time configured in the second TWT; and determine whether the difference is less than a server processing threshold, such as in combination with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1650, client device 1602 can calculate the difference between the first server processing time configured in the first TWT and the second server processing time configured in the second TWT; and determine whether the difference is less than a server processing threshold. Furthermore, step 1810 can be performed by... Figure 1The display processor 127 in the system performs the operation. In some aspects, selecting a first TWT configuration or a second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold. Furthermore, the processing threshold may be associated with the power level at the client device or the battery level at the client device.

[0133] At 1812, the client device can adjust the server processing threshold when its power level is less than a power threshold or its battery level is less than a battery threshold; and send an indication to the server of the adjusted server processing threshold when its power level is less than the power threshold or its battery level is less than the battery threshold, as in combination with... Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1660, client device 1602 can adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication to the server of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, step 1812 can be performed by... Figure 1 The display processor 127 in the middle is executed.

[0134] At 1814, the client device can select either the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set, such as in combination. Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1670, client device 1602 may select the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second offset set. Furthermore, step 1814 may be... Figure 1 The display processor 127 in the middle is executed.

[0135] At 1816, the client device may output a second indication of the selection of the first TWT configuration or the second TWT configuration, as in conjunction with Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1680, client device 1602 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. Furthermore, step 1816 may be... Figure 1The display processor 127 executes the operation. In some aspects, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include sending a second indication of the selection of a first TWT configuration or a second TWT configuration to a modem at the server. Furthermore, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include storing the second indication of the selection of a first TWT configuration or a second TWT configuration in a modem at a client device.

[0136] Furthermore, at point 1816, the client device can communicate with the server via a split XR process based on the selection of a first TWT configuration or a second TWT configuration, such as in combination with... Figures 1 to 16 The examples described in the document. For example, as... Figure 16 As described in 1680, client device 1602 can communicate with the server via a split XR process based on a selection of a first TWT configuration or a second TWT configuration. Furthermore, step 1816 can be... Figure 1 The display processor 127 in the middle is executed.

[0137] Figure 19 This is a communication flowchart 1900 illustrating processing according to one or more techniques disclosed herein. For example... Figure 19 As shown, Figure 1900 includes example communication between a server 1902 (e.g., a server, telephone, smartphone, client, headset, HMD, or AR glasses), a client device 1904 (e.g., a client, headset, HMD, AR glasses, server, telephone, or smartphone), and a memory 1906 (e.g., a memory or cache) according to one or more technologies of this disclosure.

[0138] At 1910, server 1902 can compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for the split extended reality (XR) process with the client device.

[0139] At 1920, server 1902 may output a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split XR process with the client device (e.g., server 1902 may send indication 1922 to client device 1904). In some aspects, outputting a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency may include sending a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency to the client device or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device. Furthermore, the set of rendering latency may be a set of rendering statistics associated with the latency of the split XR process of the server, the set of encoding latency may be a set of encoding statistics associated with the latency of the split XR process of the server, and the set of network latency may be a set of network statistics associated with the latency of the split XR process of the client device.

[0140] At 1930, server 1902 may, based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second set of offsets between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. In some aspects, calculating the first offset between the first rendering start time configured for the first TWT and the first TWT start time may include calculating the difference between the first rendering start time and the first TWT start time. Furthermore, calculating the second set of offsets between the second set of rendering start times configured for the second TWT and the second set of TWT start times may include: calculating the difference between an initial second rendering start time and an initial second TWT start time; and calculating the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time. In addition, calculating the second offset set between the second set of rendering start times for the second TWT configuration and the second set of TWT start times may include: calculating the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0141] At 1940, server 1902 can estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration based on the calculation of the first offset and the second offset set.

[0142] Furthermore, at 1940, server 1902 may calculate the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. In some aspects, selecting the first TWT configuration or the second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination that the difference is less than the m2r2p threshold. Furthermore, the m2r2p threshold may be associated with the power level at the client device or the battery level at the client device.

[0143] At 1950, server 1902 can estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set.

[0144] Furthermore, at 1950, server 1902 may calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold. In some aspects, selecting the first TWT configuration or the second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination that the difference is less than the server processing threshold. Furthermore, the processing threshold may be associated with the power level at the client device or the battery level at the client device.

[0145] At 1960, server 1902 may adjust the server processing threshold if the power level at the client device is less than a power threshold or if the battery level at the client device is less than a battery threshold; and send an instruction to the client device on the adjusted server processing threshold if the power level at the client device is less than the power threshold or if the battery level at the client device is less than the battery threshold.

[0146] At 1970, server 1902 may select the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set.

[0147] At point 1980, server 1902 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. In some aspects, outputting the second indication of the selection of the first TWT configuration or the second TWT configuration may include sending the second indication of the selection of the first TWT configuration or the second TWT configuration to a modem at a client device (e.g., server 1902 may send indication 1982 to client device 1904). Furthermore, outputting the second indication of the selection of the first TWT configuration or the second TWT configuration may include storing the second indication of the selection of the first TWT configuration or the second TWT configuration in a modem at the server (e.g., server 1902 may store indication 1984 in memory 1906).

[0148] Furthermore, at 1980, server 1902 may communicate with the client device via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0149] Figure 20 This is a flowchart 2000 of an example method for display processing according to one or more technologies disclosed herein. The method may be performed by a server, telephone, smartphone, client device, headset, HMD, AR glasses, DPU (or other display processor), CPU (or other central processing unit), DPU driver, DDIC, GPU (or other graphics processor), means for display processing, wireless communication device, and / or an apparatus capable of performing such a combination. Figures 1 to 19 The example uses any device for frame processing to perform.

[0150] At 2002, the server can compute at least one of the following sets of rendering latency, encoding latency, or network latency used in the split extended reality (XR) process with the client device, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1910, server 1902 can compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for the split extended reality (XR) process with the client device. Furthermore, step 2002 can be performed by… Figure 1 The display processor 127 in the middle is executed.

[0151] At 2004, the server may output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set used for the split XR process with the client device, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19As described in 1920, server 1902 may output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set used for the split XR process with the client device. Furthermore, step 2004 may be... Figure 1 The display processor 127 in the display process executes the operation. In some aspects, outputting a first indication of at least one of the rendering latency set, encoding latency set, or network latency set may include sending a first indication of at least one of the rendering latency set, encoding latency set, or network latency set to a client device or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between a client device and a server, or (2) at least one split XR application running on the client device. Additionally, the rendering latency set may be a set of rendering statistics associated with the latency of the split XR process on the server, the encoding latency set may be a set of encoding statistics associated with the latency of the split XR process on the server, and the network latency set may be a set of network statistics associated with the latency of the split XR process on the client device.

[0152] In 2014, the server could select either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set, such as in combination. Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1970, server 1902 may select either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set. Furthermore, step 2014 may be... Figure 1 The display processor 127 in the middle is executed.

[0153] At 2016, the server can output a second indication of the selection of the first TWT configuration or the second TWT configuration, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1980, server 1902 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. Furthermore, step 2016 may be... Figure 1 The display processor 127 executes the operation. In some aspects, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include sending a second indication of the selection of a first TWT configuration or a second TWT configuration to a modem at the client device. Furthermore, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include storing the second indication of the selection of a first TWT configuration or a second TWT configuration in a modem at the server.

[0154] Furthermore, in 2016, the server could communicate with the client device via a split XR process based on a selection of a first TWT configuration or a second TWT configuration, such as in combination with... Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1980, server 1902 can communicate with client devices via a split XR process based on a selection of a first TWT configuration or a second TWT configuration. Furthermore, step 2016 can be performed by... Figure 1 The display processor 127 in the middle is executed.

[0155] Figure 21 This is a flowchart 2100 of an example method for display processing according to one or more technologies disclosed herein. The method may be performed by a server, telephone, smartphone, client device, headset, HMD, AR glasses, DPU (or other display processor), CPU (or other central processing unit), DPU driver, DDIC, GPU (or other graphics processor), means for display processing, wireless communication device, and / or an apparatus capable of performing such a combination. Figures 1 to 19 The example uses any device for frame processing to perform.

[0156] At 2102, the server may compute at least one of the following sets of rendering latency, encoding latency, or network latency used in the split extended reality (XR) process with the client device, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1910, server 1902 can compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for the split extended reality (XR) process with the client device. Furthermore, step 2102 can be performed by… Figure 1 The display processor 127 in the middle is executed.

[0157] At 2104, the server may output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set used for the split XR process with the client device, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1920, server 1902 may output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set used for the split XR process with the client device. Furthermore, step 2104 may be... Figure 1The display processor 127 in the display process executes the operation. In some aspects, outputting a first indication of at least one of the rendering latency set, encoding latency set, or network latency set may include sending a first indication of at least one of the rendering latency set, encoding latency set, or network latency set to a client device or at least one computing device. Furthermore, the split XR process may include at least one of: (1) a split XR pipeline between a client device and a server, or (2) at least one split XR application running on the client device. Additionally, the rendering latency set may be a set of rendering statistics associated with the latency of the split XR process on the server, the encoding latency set may be a set of encoding statistics associated with the latency of the split XR process on the server, and the network latency set may be a set of network statistics associated with the latency of the split XR process on the client device.

[0158] At 2106, the server may, based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows, as combined Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1930, server 1902 may calculate, based on the first instruction, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. Furthermore, step 2106 may be... Figure 1The display processor 127 performs the operation. In some aspects, calculating the first offset between the first rendering start time and the first TWT start time for the first TWT configuration may include calculating the difference between the first rendering start time and the first TWT start time. Furthermore, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the difference between the initial second rendering start time and the initial second TWT start time; and calculating the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time. Additionally, calculating the second offset set between the second set of rendering start times and the second set of TWT start times for the second TWT configuration may include: calculating the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0159] At 2108, the server can estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration based on this calculation of the first offset and the second offset set, as combined with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1940, server 1902 can estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration based on this calculation of the first offset and the second offset set. Furthermore, step 2108 can be... Figure 1 The display processor 127 in the middle is executed.

[0160] Furthermore, at 2108, the server can calculate the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold. In some aspects, selecting the first TWT configuration or the second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination that the difference is less than the m2r2p threshold, such as in combination with... Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1940, server 1902 can calculate the difference between the first m2r2p latency configured in the first TWT and the second m2r2p latency configured in the second TWT; and determine whether the difference is less than an m2r2p threshold. Furthermore, step 2108 can be performed by... Figure 1 The display processor 127 in the system executes this. Furthermore, the m2r2p threshold can be associated with the power level at the client device or the battery level at the client device.

[0161] At 2110, the server can estimate the first server processing time for the first TWT configuration and the second server processing time for the second TWT configuration based on this calculation of the first offset and the second offset set, such as in combination. Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1950, server 1902 can estimate the first server processing time for the first TWT configuration and the second server processing time for the second TWT configuration based on this calculation of the first offset and the second offset set. Furthermore, step 2110 can be... Figure 1 The display processor 127 in the middle is executed.

[0162] Furthermore, at 2110, the server can calculate the difference between the first server processing time configured in the first TWT and the second server processing time configured in the second TWT; and determine whether the difference is less than a server processing threshold, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1950, server 1902 can calculate the difference between the first server processing time configured with the first TWT and the second server processing time configured with the second TWT; and determine whether the difference is less than a server processing threshold. Furthermore, step 2110 can be performed by... Figure 1 The display processor 127 in the system performs the operation. In some aspects, selecting a first TWT configuration or a second TWT configuration may include selecting the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold. Furthermore, the processing threshold may be associated with the power level at the client device or the battery level at the client device.

[0163] At 2112, the server can adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication to the client device of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold, as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1960, server 1902 can adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and send an indication to the client device of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. Furthermore, step 2112 can be performed by... Figure 1The display processor 127 in the middle is executed.

[0164] At 2114, the server can select either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set, such as in combination. Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1970, server 1902 may select either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set. Furthermore, step 2114 may be... Figure 1 The display processor 127 in the middle is executed.

[0165] At 2116, the server may output a second indication of the selection of the first TWT configuration or the second TWT configuration, such as in combination with Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1980, server 1902 may output a second indication of the selection of the first TWT configuration or the second TWT configuration. Furthermore, step 2116 may be... Figure 1 The display processor 127 executes the operation. In some aspects, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include sending a second indication of the selection of a first TWT configuration or a second TWT configuration to a modem at the client device. Furthermore, outputting a second indication of the selection of a first TWT configuration or a second TWT configuration may include storing the second indication of the selection of a first TWT configuration or a second TWT configuration in a modem at the server.

[0166] Furthermore, at 2116, the server can communicate with the client device via a split XR process based on the selection of a first TWT configuration or a second TWT configuration, such as in combination with... Figures 1 to 19 The examples described in the document. For example, as... Figure 19 As described in 1980, server 1902 can communicate with client devices via a split XR process based on a selection of a first TWT configuration or a second TWT configuration. Furthermore, step 2116 can be... Figure 1 The display processor 127 in the middle is executed.

[0167] In each configuration, a method or apparatus for display processing is provided. This apparatus may be a client device, headset, HMD, AR glasses, server, telephone, smartphone, DPU (or other display processor), CPU (or other central processing unit), DPU driver, DDIC, GPU (or other graphics processing unit), means for display processing, wireless communication device, and / or some other processor capable of performing display processing. In various aspects, the apparatus may be display processor 127 within device 104, or may be some other hardware within device 104 or another device. The apparatus (e.g., display processor 127) may include components for obtaining a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a server. The apparatus (e.g., display processor 127) may further include components for calculating, based on the first indication, (1) a first offset between a first rendering start time and a first TWT start time for a first target wake-up time (TWT) configuration, and (2) a second set of offsets between a second set of rendering start times and a second set of TWT start times for a second TWT configuration, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. The apparatus (e.g., display processor 127) may further include components for selecting either the first TWT configuration or the second TWT configuration for the split XR process with the server based on the calculation of the first offset and the second set of offsets. The apparatus (e.g., display processor 127) may further include components for outputting a second indication of the selection of the first TWT configuration or the second TWT configuration. The apparatus (e.g., display processor 127) may further include components for estimating a first motion-to-render-to-photon (m2r2p) delay for the first TWT configuration and a second m2r2p delay for the second TWT configuration based on the calculation of the first offset and the second offset set. The apparatus (e.g., display processor 127) may further include components for: calculating the difference between the first m2r2p delay of the first TWT configuration and the second m2r2p delay of the second TWT configuration; and determining whether the difference is less than an m2r2p threshold. The apparatus (e.g., display processor 127) may further include components for: adjusting the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery charge at the client device is less than a battery threshold; and sending an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery charge at the client device is less than the battery threshold.The apparatus (e.g., display processor 127) may further include components for estimating a first server processing time for the first TWT configuration and a second server processing time for the second TWT configuration based on the calculation of the first offset and the second offset set. The apparatus (e.g., display processor 127) may further include components for: calculating the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determining whether the difference is less than a server processing threshold. The apparatus (e.g., display processor 127) may further include components for: adjusting the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and sending an indication of the adjusted server processing threshold to the server when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. The apparatus (e.g., display processor 127) may further include components for communicating with the server via the split XR process based on the selection of the first TWT configuration or the second TWT configuration. The apparatus (e.g., display processor 127) may further include components for calculating at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a client device. The apparatus (e.g., display processor 127) may further include components for outputting a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency for the split XR process with the client device. The apparatus (e.g., display processor 127) may further include components for calculating, based on the first indication, (1) a first offset between a first rendering start time and a first TWT start time configured for a first target wake-up time (TWT), and (2) a second offset set between a second set of rendering start times and a second set of TWT start times configured for a second TWT, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows. The apparatus (e.g., display processor 127) may further include components for selecting either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set. The apparatus (e.g., display processor 127) may further include components for outputting a second indication of the selection of the first TWT configuration or the second TWT configuration. The apparatus (e.g., display processor 127) may further include components for estimating a first motion-to-render-to-photon (m2r2p) delay of the first TWT configuration and a second m2r2p delay of the second TWT configuration based on the calculation of the first offset and the second offset set.The apparatus (e.g., display processor 127) may further include components for: calculating the difference between the first m2r2p latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration; and determining whether the difference is less than an m2r2p threshold. The apparatus (e.g., display processor 127) may further include components for estimating the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set. The apparatus (e.g., display processor 127) may further include components for: calculating the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determining whether the difference is less than a server processing threshold. The apparatus (e.g., display processor 127) may also include components for: adjusting the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and sending an indication to the client device of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery level at the client device is less than the battery threshold. The apparatus (e.g., display processor 127) may also include components for communicating with the client device via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0168] The subject matter described herein can be implemented to achieve one or more benefits or advantages. For example, the described display processing technology can be implemented by client devices, headsets, HMDs, AR glasses, servers, telephones, smartphones, DPUs (or other display processors), CPUs (or other central processing units), DPU drivers, DDICs, GPUs (or other graphics processing units), means for display processing, wireless communication devices, or some other processor capable of performing display processing. This can also be achieved at a lower cost compared to other display processing technologies. Furthermore, the display processing technology of this invention can improve or accelerate data processing or execution. In addition, the display processing technology of this invention can improve resource or data utilization and / or resource efficiency. Additionally, aspects of this disclosure can utilize the dual TWT window technology to improve memory bandwidth efficiency and / or increase processing speed at client devices, servers, GPUs, DPUs, and / or CPUs.

[0169] It should be understood that the specific order or hierarchy of the boxes in the disclosed process / flowcharts is merely an example of the exemplary method. It should be understood that the specific order or hierarchy of the boxes in the process / flowcharts may be rearranged based on design preferences. Furthermore, some boxes may be combined or omitted. The appended method claims provide elements of various boxes in a sample order, but are not intended to limit one to the given specific order or hierarchy.

[0170] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims, wherein references to elements in the singular form, unless specifically stated otherwise, are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0171] Unless otherwise specified, the term "some" refers to one or more, and the term "or" may be interpreted as "and / or" unless otherwise specified in the context. Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, which may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, wherein any such combination may contain one or more members of A, B, or C. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. The terms “module,” “mechanism,” “element,” “device,” etc., cannot replace the word “component.” Therefore, no claim element will be construed as a functional component unless the element is explicitly recited using the phrase “component for…”.

[0172] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then such functionality, processing unit, technique, or other module may be stored on or transmitted on a computer-readable medium as one or more instructions or code.

[0173] According to this disclosure, unless otherwise specified in the context, the term "or" may be understood as "and / or". Additionally, while phrases such as "one or more" or "at least one" may be used for some features disclosed herein but not others, features not using such language may be understood to have such implied meaning unless otherwise specified in the context.

[0174] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term “processing unit” is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then the functionality, processing unit, technique, or other module described herein may be stored on or transmitted on a computer-readable medium as one or more instructions or code. A computer-readable medium may include computer data storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices. As used herein, disks and optical discs include: compact optical discs (CDs), laser optical discs, optical discs, digital multifunction optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0175] The code can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), arithmetic logic units (ALUs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the term "processor" can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0176] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in any hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0177] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.

[0178] Aspect 1 is an apparatus for performing display processing at a client device, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory, and configured, at least in part, based on information stored in the at least one memory, to: obtain a first indication of at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a server; calculate, based on the first indication, (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second TWT start time, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; select, based on the calculation of the first offset and the second offset set, either the first TWT configuration or the second TWT configuration for the split XR process with the server; and output a second indication of the selection of the first TWT configuration or the second TWT configuration.

[0179] Aspect 2 is the apparatus according to aspect 1, wherein, in order to calculate the first offset between the first rendering start time configured for the first TWT and the first TWT start time, the at least one processor is configured individually or in any combination to: calculate the difference between the first rendering start time and the first TWT start time; and wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the difference between the initial second rendering start time and the initial second TWT start time; and calculate the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0180] Aspect 3 is the apparatus according to aspect 2, wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between the at least one subsequent second rendering start time and the at least one subsequent second TWT start time.

[0181] Aspect 4 is an apparatus according to any one of aspects 1 to 3, wherein the at least one processor is configured individually or in any combination to estimate, based on the calculation of the first offset and the second offset set, a first motion-to-render-to-photon (m2r2p) delay of the first TWT configuration and a second m2r2p delay of the second TWT configuration.

[0182] Aspect 5 is the apparatus according to aspect 4, wherein the at least one processor is configured individually or in any combination to: calculate the difference between the first m2r2p delay of the first TWT configuration and the second m2r2p delay of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold.

[0183] Aspect 6 is the apparatus according to aspect 5, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the m2r2p threshold.

[0184] Aspect 7 is the apparatus according to any one of aspects 5 to 6, wherein the m2r2p threshold is associated with the power level at the client device or the battery charge at the client device.

[0185] Aspect 8 is the apparatus according to aspect 7, wherein the at least one processor is further configured, individually or in any combination, to: adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery charge at the client device is less than a battery threshold; and to send an indication of the adjusted m2r2p threshold to the server when the power level at the client device is less than the power threshold or when the battery charge at the client device is less than the battery threshold.

[0186] Aspect 9 is an apparatus according to any one of aspects 1 to 8, wherein the at least one processor is configured individually or in any combination to estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set.

[0187] Aspect 10 is the apparatus according to aspect 9, wherein the at least one processor is further configured, individually or in any combination, to: calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; and determine whether the difference is less than a server processing threshold.

[0188] Aspect 11 is the apparatus according to aspect 10, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold.

[0189] Aspect 12 is an apparatus according to any one of aspects 10 to 11, wherein the processing threshold is associated with a power level at the client device or a battery charge at the client device.

[0190] Aspect 13 is the apparatus according to aspect 12, wherein the at least one processor is further configured, individually or in any combination, to: adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery charge at the client device is less than a battery threshold; and to send an indication to the server of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery charge at the client device is less than the battery threshold.

[0191] Aspect 14 is an apparatus according to any one of aspects 1 to 13, wherein the at least one processor is also configured, individually or in any combination, to communicate with the server via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0192] Aspect 15 is an apparatus according to any one of aspects 1 to 14, wherein, in order to obtain the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set, the at least one processor is configured individually or in any combination to receive from the server or at least one computing device the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set.

[0193] Aspect 16 is an apparatus according to any one of aspects 1 to 15, wherein the split XR process includes at least one of the following: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device.

[0194] Aspect 17 is an apparatus according to any one of Aspects 1 to 16, wherein the rendering latency set is a set of rendering statistics associated with the latency of the split XR process of the server, wherein the encoding latency set is a set of encoding statistics associated with the latency of the split XR process of the server, and wherein the network latency set is a set of network statistics associated with the latency of the split XR process of the server.

[0195] Aspect 18 is an apparatus according to any one of aspects 1 to 17, the apparatus further comprising at least one of an antenna or a transceiver coupled to the at least one processor, wherein, in order to output a second indication of the selection of the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: transmit the second indication of the selection of the first TWT configuration or the second TWT configuration to a modem at the server via the antenna or at least one of the transceivers; or store the second indication of the selection of the first TWT configuration or the second TWT configuration in a modem at the client device.

[0196] Aspect 19 is an apparatus for performing display processing at a server, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory and, at least in part based on information stored in the at least one memory, the at least one processor being configured individually or in any combination to: compute at least one of a set of rendering latency, a set of encoding latency, or a set of network latency for a split extended reality (XR) process with a client device; output a first indication of at least one of the set of rendering latency, the set of encoding latency, or the set of network latency for the split XR process with the client device; and based on the first indication... To calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; to select either the first TWT configuration or the second TWT configuration for the split XR process with the client device based on the calculation of the first offset and the second offset set; and to output a second indication of the selection of the first TWT configuration or the second TWT configuration.

[0197] Aspect 20 is the apparatus according to aspect 19, wherein, in order to calculate the first offset between the first rendering start time configured for the first TWT and the first TWT start time, the at least one processor is configured individually or in any combination to: calculate the difference between the first rendering start time and the first TWT start time; and wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the difference between the initial second rendering start time and the initial second TWT start time; and calculate the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

[0198] Aspect 21 is the apparatus according to aspect 20, wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between the at least one subsequent second rendering start time and the at least one subsequent second TWT start time.

[0199] Aspect 22 is an apparatus according to any one of aspects 19 to 21, wherein the at least one processor is further configured, individually or in any combination, to: estimate a first motion-to-render-to-photon (m2r2p) delay of the first TWT configuration and a second m2r2p delay of the second TWT configuration based on the calculation of the first offset and the second offset set; calculate the difference between the first m2r2p delay of the first TWT configuration and the second m2r2p delay of the second TWT configuration; and determine whether the difference is less than an m2r2p threshold.

[0200] Aspect 23 is the apparatus according to aspect 22, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the m2r2p threshold; or wherein the m2r2p threshold is associated with a power level at the client device or a battery charge at the client device.

[0201] Aspect 24 is an apparatus according to any one of aspects 19 to 23, wherein the at least one processor is further configured individually or in any combination to: estimate a first server processing time of the first TWT configuration and a second server processing time of the second TWT configuration based on the calculation of the first offset and the second offset set; calculate the difference between the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration; determine whether the difference is less than a server processing threshold; wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold; or wherein the processing threshold is associated with a power level at the client device or a battery level at the client device.

[0202] Aspect 25 is the apparatus according to aspect 24, wherein the at least one processor is further configured, individually or in any combination, to: adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery charge at the client device is less than a battery threshold; and to send an indication to the client device of the adjusted server processing threshold when the power level at the client device is less than the power threshold or when the battery charge at the client device is less than the battery threshold.

[0203] Aspect 26 is an apparatus according to any one of aspects 19 to 25, wherein the at least one processor is also configured, individually or in any combination, to communicate with the client device via the split XR process based on the selection of the first TWT configuration or the second TWT configuration.

[0204] Aspect 27 is an apparatus according to any one of aspects 19 to 26, wherein, in order to output the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set, the at least one processor is configured individually or in any combination to send the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set to the client device or at least one computing device; wherein the split XR process includes at least one of: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device; or wherein the rendering latency set is a set of rendering statistics associated with the latency of the split XR process of the server, wherein the encoding latency set is a set of encoding statistics associated with the latency of the split XR process of the server, and wherein the network latency set is a set of network statistics associated with the latency of the split XR process of the client device.

[0205] Aspect 28 is an apparatus according to any one of aspects 19 to 27, the apparatus further comprising at least one of an antenna or transceiver coupled to the at least one processor, wherein, in order to output a second indication of the selection of the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: transmit the second indication of the selection of the first TWT configuration or the second TWT configuration to a modem at the client device via the antenna or at least one of the transceivers; or store the second indication of the selection of the first TWT configuration or the second TWT configuration in a modem at the server.

[0206] Aspect 29 is a method for implementing the display processing of any one of aspects 1 to 28.

[0207] Aspect 30 is an apparatus for display processing, the apparatus including components for implementing any one of aspects 1 to 28.

[0208] Aspect 31 is a computer-readable medium (e.g., a non-transitory computer-readable medium) storing computer-executable code that, when executed by at least one processor, causes the at least one processor to implement any one of aspects 1 to 28.

Claims

1. An apparatus for performing display processing at a client device, the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, and configured individually or in any combination, based at least in part on information stored in the at least one memory, to: Obtain a first indication of at least one of the set of rendering latency, set of encoding latency, or set of network latency used in the split extended reality (XR) process with the server; Based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; The first TWT configuration or the second TWT configuration for the split XR process with the server is selected based on the calculation of the first offset and the second offset set. as well as Output a second indication of the selection of the first TWT configuration or the second TWT configuration.

2. The apparatus of claim 1, wherein, in order to calculate the first offset between the first rendering start time configured for the first TWT and the first TWT start time, the at least one processor is configured individually or in any combination to: calculate the difference between the first rendering start time and the first TWT start time; and In order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the difference between the initial second rendering start time and the initial second TWT start time; And calculate the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

3. The apparatus of claim 2, wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between the at least one subsequent second rendering start time and the at least one subsequent second TWT start time.

4. The apparatus of claim 1, wherein the at least one processor is further configured, alone or in any combination, to: The calculations based on the first offset and the second offset set are used to estimate the first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration.

5. The apparatus of claim 4, wherein the at least one processor is further configured, alone or in any combination, to: Calculate the difference between the first m2r2p latency configured in the first TWT configuration and the second m2r2p latency configured in the second TWT configuration; and Determine whether the difference is less than the m2r2p threshold.

6. The apparatus of claim 5, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the m2r2p threshold.

7. The apparatus of claim 5, wherein the m2r2p threshold is associated with the power level at the client device or the battery charge at the client device.

8. The apparatus of claim 7, wherein the at least one processor is further configured, alone or in any combination, to: Adjust the m2r2p threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and If the power level at the client device is less than the power threshold, or if the battery level at the client device is less than the battery threshold, an indication for adjusting the m2r2p threshold is sent to the server.

9. The apparatus of claim 1, wherein the at least one processor is further configured, alone or in any combination, to: The first server processing time for the first TWT configuration and the second server processing time for the second TWT configuration are estimated based on the calculations performed on the first offset and the second offset set.

10. The apparatus of claim 9, wherein the at least one processor is further configured, alone or in any combination, to: Calculate the difference between the processing time of the first server in the first TWT configuration and the processing time of the second server in the second TWT configuration; and Determine whether the difference is less than the server processing threshold.

11. The apparatus of claim 10, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold.

12. The apparatus of claim 10, wherein the processing threshold is associated with a power level at the client device or a battery charge at the client device.

13. The apparatus of claim 12, wherein the at least one processor is further configured, alone or in any combination, to: Adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and If the power level at the client device is less than the power threshold, or if the battery level at the client device is less than the battery threshold, an instruction for adjusting the server processing threshold is sent to the server.

14. The apparatus of claim 1, wherein the at least one processor is further configured, alone or in any combination, to: Based on the selection of the first TWT configuration or the second TWT configuration, communication is made with the server via the split XR process.

15. The apparatus of claim 1, wherein, in order to obtain the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set, the at least one processor is configured individually or in any combination to: receive from the server or at least one computing device the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set.

16. The apparatus of claim 1, wherein the split XR process comprises at least one of the following: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device.

17. The apparatus of claim 1, wherein the rendering latency set is a set of rendering statistics associated with the latency of the split XR process of the server, wherein the encoding latency set is a set of encoding statistics associated with the latency of the split XR process of the server, and wherein the network latency set is a set of network statistics associated with the latency of the split XR process of the server.

18. The apparatus of claim 1, further comprising: At least one of the antennas or transceivers coupled to the at least one processor, wherein, in order to output the second indication of the selection of the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: The second indication of the selection of the first TWT configuration or the second TWT configuration is transmitted to the modem at the server via at least one of the antennas or the transceiver; or The second indication of the selection of the first TWT configuration or the second TWT configuration is stored in the modem at the client device.

19. An apparatus for performing display processing at a server, the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, and configured individually or in any combination, based at least in part on information stored in the at least one memory, to: The computation is used for at least one of the following sets of rendering latency, encoding latency, or network latency in the split extended reality (XR) process with the client device; Output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set for the split XR process with the client device; Based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; The first TWT configuration or the second TWT configuration for the split XR process with the client device is selected based on the calculation of the first offset and the second offset set. as well as Output a second indication of the selection of the first TWT configuration or the second TWT configuration.

20. The apparatus of claim 19, wherein, in order to calculate the first offset between the first rendering start time configured for the first TWT and the first TWT start time, the at least one processor is configured individually or in any combination to: calculate the difference between the first rendering start time and the first TWT start time; and In order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the difference between the initial second rendering start time and the initial second TWT start time; And calculate the difference between at least one subsequent second rendering start time and at least one subsequent second TWT start time.

21. The apparatus of claim 20, wherein, in order to calculate the second offset set between the second set of rendering start times configured for the second TWT and the second set of TWT start times, the at least one processor is configured individually or in any combination to: calculate the sum of the difference between the initial second rendering start time and the initial second TWT start time and the difference between the at least one subsequent second rendering start time and the at least one subsequent second TWT start time.

22. The apparatus of claim 19, wherein the at least one processor is further configured, alone or in any combination, to: The first motion-to-render-to-photon (m2r2p) latency of the first TWT configuration and the second m2r2p latency of the second TWT configuration are estimated based on the calculations of the first offset and the second offset set. Calculate the difference between the first m2r2p latency configured in the first TWT configuration and the second m2r2p latency configured in the second TWT configuration; and Determine whether the difference is less than the m2r2p threshold.

23. The apparatus of claim 22, wherein, in order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the m2r2p threshold; or The m2r2p threshold is associated with the power level or battery charge at the client device.

24. The apparatus of claim 19, wherein the at least one processor is further configured, alone or in any combination, to: The calculations on the first offset and the second offset set are used to estimate the first server processing time of the first TWT configuration and the second server processing time of the second TWT configuration. Calculate the difference between the first server processing time configured in the first TWT configuration and the second server processing time configured in the second TWT configuration; Determine whether the difference is less than the server processing threshold; In order to select the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to select the first TWT configuration or the second TWT configuration based on the determination of whether the difference is less than the server processing threshold; or The processing threshold is associated with the power level or battery level at the client device.

25. The apparatus of claim 24, wherein the at least one processor is further configured, alone or in any combination, to: Adjust the server processing threshold when the power level at the client device is less than a power threshold or when the battery level at the client device is less than a battery threshold; and If the power level at the client device is less than the power threshold, or if the battery level at the client device is less than the battery threshold, an instruction for adjusting the server processing threshold is sent to the client device.

26. The apparatus of claim 19, wherein the at least one processor is further configured, alone or in any combination, to: Based on the selection of the first TWT configuration or the second TWT configuration, communication is made with the client device via the split XR process.

27. The apparatus of claim 19, wherein, in order to output the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set, the at least one processor is configured individually or in any combination to send the first indication for at least one of the rendering latency set, the encoding latency set, or the network latency set to the client device or at least one computing device; The split XR process includes at least one of the following: (1) a split XR pipeline between the client device and the server, or (2) at least one split XR application running on the client device; or The rendering latency set is a set of rendering statistics associated with the latency of the split XR process of the server, the encoding latency set is a set of encoding statistics associated with the latency of the split XR process of the server, and the network latency set is a set of network statistics associated with the latency of the split XR process of the client device.

28. The apparatus of claim 19, further comprising: At least one of the antennas or transceivers coupled to the at least one processor, wherein, in order to output the second indication of the selection of the first TWT configuration or the second TWT configuration, the at least one processor is configured individually or in any combination to: The second indication of the selection of the first TWT configuration or the second TWT configuration is sent to the modem at the client device via at least one of the antennas or the transceivers; or The second indication of the selection of the first TWT configuration or the second TWT configuration is stored in the modem at the server.

29. A method for performing display processing at a client device, the method comprising: Obtain a first indication of at least one of the set of rendering latency, set of encoding latency, or set of network latency used in the split extended reality (XR) process with the server; Based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; The first TWT configuration or the second TWT configuration for the split XR process with the server is selected based on the calculation of the first offset and the second offset set. as well as Output a second indication of the selection of the first TWT configuration or the second TWT configuration.

30. A method for performing display processing at a server, the method comprising: The computation is used for at least one of the following sets of rendering latency, encoding latency, or network latency in the split extended reality (XR) process with the client device; Output a first indication of at least one of the rendering latency set, the encoding latency set, or the network latency set for the split XR process with the client device; Based on the first instruction, calculate (1) a first offset between a first rendering start time configured for a first target wake-up time (TWT) and a first TWT start time, and (2) a second offset set between a second set of rendering start times configured for a second TWT and a second set of TWT start times, wherein the first TWT configuration includes one TWT window and the second TWT configuration includes two TWT windows; The first TWT configuration or the second TWT configuration for the split XR process with the client device is selected based on the calculation of the first offset and the second offset set. as well as Output a second indication of the selection of the first TWT configuration or the second TWT configuration.

Images