Reprojection optimization based on contextual brightness

Description

1 / 69 CONTEXTUAL BRIGHTNESS-BASED REPROJECTION OPTIMIZATION CROSS-REFERENCE TO RELATED DEPOSIT REQUESTS

[0001] This application claims the benefit and priority of the provisional application. US Serial No. 63 / 498,800, entitled REPROJECTION OPTIMIZATION FOR SPLIT ARCHITECTURES and filed on April 27, 2023, and non-provisional patent application US Serial No. 18 / 521,414, entitled REPROJECTION OPTIMIZATION BASED ON CONTEXTUAL BRIGHTNESS and filed on November 28, 2023, are expressly incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] This disclosure relates generally to processing systems and, more particularly, to one or more techniques for content reprojection onto split architectures. INTRODUCTION

[0003] Computing devices frequently perform graphics and / or display processing (e.g., using a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices may include, for example, computer workstations, mobile phones such as smartphones, embedded systems, personal computers, tablet computers, and video game consoles. GPUs are configured to execute a graphics processing pipeline that includes one or more processing stages, which operate together to execute graphics processing commands and output a frame. A central processing unit (CPU) can control the operation of the GPU by issuing one or more graphics processing commands to the GPU.Current CPUs are typically capable of running multiple applications simultaneously, each of which may need to utilize the GPU during execution. A display processor is... Petition 870250094200, dated 10 / 15 / 2025, pp. 277 / 382 2 / 69 configured to convert digital information received from a CPU into analog values, and can issue commands to a display panel in order to display visual content. A device that provides content for visual presentation on a display may use a GPU and / or a display processor.

[0004] A device's GPU may be configured to perform the processes in a graphics processing pipeline. Additionally, a display processor or a display processing unit (DPU) may be configured to perform the display processing processes. However, with the advent of wireless communication and smaller handheld devices, there has been a greater need for improved graphics or display processing. BRIEF SUMMARY

[0005] The following description presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all aspects covered, and is not intended to identify key or critical elements of all aspects, nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as an introduction to the more detailed description that is presented later.

[0006] In one aspect of the disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus may be any apparatus capable of performing display processing (e.g., a GPU, CPU, or DPU). The apparatus may obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene. Additionally, the apparatus may set up a second intensity map based on the first intensity map and at least one frame of coordinates from a user's perspective of a device to display content associated with the scene, wherein the second intensity map is associated with luminance information. Petition 870250094200, dated 10 / 15 / 2025, pp. 278 / 382 3 / 69 for the scene. The device can also determine whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene. Furthermore, the device can identify the section on the display that is associated with at least one region based on the luminance information for that region being within the appropriate luminance range or outside the appropriate luminance range, and where pixel set processing comprises processing the pixel set based on the identification of the section on the display. The device can also filter the luminance information for at least one region in the second intensity map if the luminance information for that region is within the appropriate luminance range.Furthermore, the device can process a set of pixels corresponding to a section on a display that is associated with at least one region based on luminance information for that region being at least within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for that region in the second intensity map based on whether the luminance information for that region is within the appropriate luminance range. The device can also provide an indication of the processed pixel set or the estimated transformation parameter set. The device can also transform the display content that corresponds to at least one region in the second intensity map based on the estimated transformation parameter set.The device can also adjust a display content location corresponding to at least one region on the second intensity map if at least one of the following occurs: (1) a frequency change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold. Petition 870250094200, dated 10 / 15 / 2025, pp. 279 / 382 4 / 69

[0007] Details of one or more examples of the disclosure are presented in the attached drawings and in the description below. Other attributes, objectives, and advantages of the disclosure will become apparent from the description and drawings, as well as from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram illustrating an example content generation system.

[0009] Figure 2 illustrates an example graphics processing unit (GPU).

[0010] Figure 3 is a diagram that illustrates example processing components.

[0011] Figure 4 is a diagram illustrating an example split architecture.

[0012] Figure 5 is a diagram that illustrates example scenes associated with display processing or graphics processing.

[0013] Figure 6 is a diagram that illustrates example scenes associated with display processing or graphics processing.

[0014] Figure 7 is a diagram that illustrates example scenes associated with display processing or graphics processing.

[0015] Figure 8 is a diagram that illustrates an example processing flow.

[0016] Figure 9 is a diagram that illustrates an example processing flow.

[0017] Figure 10 is a diagram that illustrates example scenes associated with display processing or graphics processing.

[0018] Figure 11 is a diagram illustrating an example processing flow.

[0019] Figure 12 is a diagram illustrating an example processing flow.

[0020] Figure 13 is a diagram illustrating a processing flow of Petition 870250094200, dated 10 / 15 / 2025, pages 280 / 382 5 / 69 example.

[0021] Figure 14 is a diagram that illustrates an example processing flow.

[0022] Figure 15 is a communication flowchart illustrating example communications between a GPU, a CPU, and memory.

[0023] Figure 16 is a flowchart of an example display processing method.

[0024] Figure 17 is a flowchart of an example display processing method. DETAILED DESCRIPTION

[0025] Aspects of split architecture may include late-stage reprojection. For example, eye and depth frames may be rendered on the complementary device for each eye (left, right). Furthermore, the device / glass may receive these frames, decode the frames, process the frames, and / or send the frames to the display. Additionally, the user may move during this time, and the rendered objects may be displayed in different places in the scene compared to what the user expects. To account for this user movement and minimize visual discrepancy, the device may distort the eye patches based on the latest available pose information. In some respects, light / brightness-sensitive rendering may enhance the visual appeal of virtual content and may make the scene more immersive.Furthermore, virtual objects can have varying complexities that may be visible to the user under certain lighting conditions (e.g., lighting conditions assumed during rendering). However, when the brightness of the scene changes substantially compared to the rendering time, these finer details in the virtual object may not be discernible to the user. For example, a reprojection pipeline may not be aware of these relative brightness changes, which can lead to computational overshoot. Aspects of the present disclosure can reduce computational complexity in such scenarios (e.g., Petition 870250094200, dated 10 / 15 / 2025, pages 281 / 382 6 / 69 example, when the scene brightness changes substantially compared to the rendered time). By reducing computational complexity when the scene brightness changes substantially compared to the rendered time, the aspects presented in the present invention can reduce memory / interconnect bandwidth. Furthermore, by reducing computational complexity when the scene brightness changes substantially compared to the rendered time, the aspects presented in the present invention can potentially allow for an overall reduction in power consumption.

[0026] Aspects presented in the present invention may include several benefits or advantages. For example, aspects presented in the present invention may reduce computational complexity in such scenarios (e.g., when scene brightness changes substantially compared to the rendered time). By reducing computational complexity when scene brightness changes substantially compared to the rendered time, aspects presented in the present invention may reduce memory / interconnect bandwidth. Furthermore, by reducing computational complexity when scene brightness changes substantially compared to the rendered time, aspects presented in the present invention may potentially allow for an overall reduction in power consumption.

[0027] Various aspects of systems, apparatus, computer program products and methods are described more fully later in this document, with reference to the accompanying drawings. This disclosure can, however, be incorporated in many different forms and should not be interpreted as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure is thorough and complete, and will convey to those skilled in the art the entirety of the scope of this disclosure. Based on the teachings of the present invention, those skilled in the art will recognize that the scope of this disclosure is intended to cover any aspect of systems, apparatus, computer program products Petition 870250094200, dated 10 / 15 / 2025, pp. 282 / 382 7 / 69 of computer and methods disclosed in the present invention, whether implemented independently of, or in combination with, other aspects of the disclosure. For example, an apparatus may be implemented, or a method may be practiced using any number of the aspects set forth in the present invention. Furthermore, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using another structure, functionality, or structure and functionality in addition to or different from the various aspects of the disclosure set forth in the present invention. Any aspect disclosed in the present invention may be incorporated by one or more elements of a claim.

[0028] Although several aspects are described in the present invention, many variations and permutations of these aspects fall within the scope of this disclosure. While some potential benefits and advantages of the aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to particular benefits, uses, or purposes. Instead, aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, networks, and transmission protocols, some of which are illustrated by way of example in the figures and description below. The detailed description and drawings are merely illustrative of this disclosure, rather than limiting, the scope of this disclosure being defined by the appended claims and equivalents thereof.

[0029] Several aspects are presented with reference to various devices and methods. These devices and methods are described in the detailed description below, and illustrated in the attached drawings by various blocks, components, circuits, processes, algorithms, and the like (collectively referred to as elements). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements will be implemented in the form of hardware or software will depend on the particular application and the design constraints imposed on the system as a whole.

[0030] By way of example, an element, or any portion of an element, Petition 870250094200, dated 10 / 15 / 2025, pages 283 / 382 8 / 69, or any combination of elements, can be implemented in the form of a processing system, which includes one or more processors (which can also be called processing units).Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gate logic, distinct hardware circuits, and other suitable hardware components configured to perform the various functionalities described throughout this disclosure.One or more processors in the processing system can execute software. Software can be broadly interpreted as meaning instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, execution threads, procedures, functions, etc., whether these are called software, firmware, middleware, microcode, hardware description language, or another term. The term application can refer to software. As described in the present invention, one or more techniques can refer to an application, that is, software, that is configured to perform one or more functions. In these examples, the application can be stored in memory, for example, in the memory integrated into a processor chip, in system memory, or in any other memory.The hardware described in the present invention, such as a processor, can be configured to run the application. For example, the application could be... Petition 870250094200, dated 10 / 15 / 2025, pages 284 / 382 9 / 69 described as including code that, when executed by the hardware, causes the hardware to perform one or more techniques described in the present invention. As an example, the hardware may access the code from memory and execute the code accessed from memory to perform one or more techniques described in the present invention. In some examples, components are identified in the present disclosure. In these examples, the components may be hardware, software, or a combination thereof. The components may be separate components or subcomponents of a single component.

[0031] Consequently, in one or more examples described in the present invention, the described functions can be implemented in hardware, software, or any combination thereof. If implemented in software, the functions can be stored or encoded as one or more instructions or code in a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any available media that can be accessed by a computer.By way of example, and not limitation, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the aforementioned types of computer-readable media, or any other media that can be used to store computer-executable code in the form of instructions or data structures that can be accessed by a computer.

[0032] In general, the present disclosure describes techniques for obtaining a graphics processing pipeline in a single device or in multiple devices, improving the rendering of graphic content and / or reducing the load on a processing unit, that is, any processing unit configured to perform one or more techniques described in the present invention, Petition 870250094200, dated 10 / 15 / 2025, pages 285 / 382 10 / 69 as a GPU. For example, this disclosure describes techniques for graphics processing on any device that uses graphics processing. Other example benefits are described throughout this disclosure.

[0033] As used in the present invention, instances of the term content may refer to graphic content, image, and vice versa. This is true regardless of whether the terms are being used as adjectives, nouns, or other grammatical classes. In some examples, as used in the present invention, the term graphic content may refer to content produced through one or more processes of a graphics processing pipeline. In some examples, as used in the present invention, the term graphic content may refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used in the present invention, the term graphic content may refer to content produced by a graphics processing unit.

[0034] In some examples, as used in the present invention, the term display content may refer to content generated by a processing unit configured to perform display processing. In some examples, as used in the present invention, the term display content may refer to content generated by a display processing unit. Graphic content can be processed to become display content. For example, a graphics processing unit may output graphic content, such as a frame, to a buffer (which may be called a frame buffer). A display processing unit may read the graphic content, such as one or more frames from the buffer, and perform one or more display processing techniques on it in order to generate display content. For example, a display processing unit may be configured to perform composition on one or more rendered layers in order to generate a frame.As another example, a display processing unit might be configured to compose, merge, or otherwise combine two or more layers into a single one. Petition 870250094200, dated 10 / 15 / 2025, pp. 286 / 382 11 / 69 frame. A display processing unit may be configured to perform scaling, for example upscaling or downscaling, on a frame. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have already been merged together to form the frame, that is, the frame includes two or more layers, and the frame that includes two or more layers may be subsequently merged.

[0035] Figure 1 is a block diagram illustrating an example content generation system 100, configured to implement one or more techniques of the present disclosure. The content generation system 100 includes a device 104. The device 104 may include one or more components or circuits that perform various functions described in the present invention. In some examples, one or more components of the device 104 may be components of a SOC. The device 104 may include one or more components configured to perform one or more techniques of the present disclosure. In the example shown, the device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, the device 104 may include several components, for example, a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131.The reference to display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays. Display 131 may include a first display and a second display. The first display may be a display for the left eye and the second display may be a display for the right eye. In some examples, the first and second displays may receive different frames for presentation on them. In other examples, the first and second displays may receive the same frames for presentation on them. In further examples, the results of graphic processing may not be displayed on the device; for example, the first and second displays may not receive any frames for presentation on them. Instead, the frames or the results of graphic processing may be transferred to another device. Petition 870250094200, dated 10 / 15 / 2025, pages 287 / 382 12 / 69 In some respects, this could be called split rendering.

[0036] Processing unit 120 may include internal memory 121. The processing unit 120 may be configured to perform graphics processing, as in a graphics processing pipeline 107. The content encoder / decoder 122 may include internal memory 123. In some examples, the device 104 may include a display processor, such as the display processor 127, to perform one or more display processing techniques on one or more frames generated by the processing unit 120, before presentation by one or more displays 131. The display processor 127 may be configured to perform display processing. For example, the display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by the processing unit 120. The one or more displays 131 may be configured to display or otherwise present frames processed by the display processor 127.In some examples, the one or more displays 131 may include one or more of the following: a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.

[0037] External memory to the processing unit 120 and the content encoder / decoder 122, such as system memory 124, may be accessible to the processing unit 120 and the content encoder / decoder 122. For example, the processing unit 120 and the content encoder / decoder 122 may be configured to read and / or write to external memory, such as system memory 124. The processing unit 120 and the content encoder / decoder 122 may be communicatively coupled to system memory 124 via a bus. In some examples, the processing unit 120 and the Petition 870250094200, dated 10 / 15 / 2025, pp. 288 / 382 13 / 69 content encoder / decoder 122 can be communicatively coupled to each other via the bus or a different connection.

[0038] The content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store the received encoded or decoded graphic content. The content encoder / decoder 122 can be configured to receive encoded or decoded graphic content, for example, from system memory 124 and / or communication interface 126, in the form of encoded pixel data. The content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0039] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile storage or memory devices. In some examples, internal memory 121 or system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media or optical storage media, or any other type of memory.

[0040] Internal memory 121 or system memory 124 may be a non-transient storage medium according to some examples. The term non-transient may indicate that the storage medium is not incorporated into a carrier wave or a propagated signal. However, the term non-transient should not be interpreted as meaning that internal memory 121 or system memory 124 is not removable or that its contents are static. As an example, system memory 124 may be removed from device 104 and moved to another device. As another example, system memory 124 may not be removable from device 104.

[0041] Processing unit 120 can be a central processing unit (CPU), a graphics processing unit (GPU), a GPU Petition 870250094200, dated 10 / 15 / 2025, pp. 289 / 382 14 / 69 general-purpose processing unit (GPGPU) or any other processing unit that can be configured to perform graphics processing. In some instances, the processing unit 120 may be integrated into a device motherboard 104. In some instances, the processing unit 120 may be present in a graphics card that is installed in a port on a device motherboard 104, or it may otherwise be incorporated into a peripheral device configured to interoperate with device 104. The processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, another equivalent set of integrated or distinct logic circuits, or any combination thereof.If the techniques are implemented partially in software, the processing unit 120 may store instructions for the software in a suitable computer-readable non-transient storage medium, for example, internal memory 121, and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered as being one or more processors.

[0042] The content encoder / decoder 122 may be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into a device motherboard 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic, software, hardware, firmware, another set of equivalent integrated or discrete logic circuits, or Petition 870250094200, dated 10 / 15 / 2025, pp. 290 / 382 15 / 69 any combination thereof. If the techniques are implemented partially in software, the content encoder / decoder 122 may store instructions for the software in a suitable non-transient computer-readable storage medium, for example, internal memory 123, and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing, including hardware, software, a combination of hardware and software, etc., may be considered as one or more processors.

[0043] In some respects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any reception function described in the present invention with respect to the device 104. Additionally, the receiver 128 may be configured to receive information, for example, eye or head position information, rendering commands, or location information, from another device. The transmitter 130 may be configured to perform any transmission function described in the present invention with respect to the device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a content request. The receiver 128 and the transmitter 130 may be combined into a transceiver 132.In these examples, transceiver 132 may be configured to perform any reception and / or transmission function described in the present invention in relation to device 104.

[0044] Referring again to Figure 1, in certain respects, processing unit 120 may include a reprojection component 198 configured to obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene. The reprojection component 198 may also be configured to set up a second intensity map based on the first intensity map and at least one frame. Petition 870250094200, dated 10 / 15 / 2025, pp. 291 / 382 16 / 69 of coordinates from a user's perspective on a device to display scene-associated content, where the second intensity map is associated with luminance information for the scene. The reprojection component 198 can also be configured to determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the scene-associated display content. The reprojection component 198 can also be configured to identify the section on the display that is associated with at least one region based on the luminance information for that region being within or outside the suitable luminance range, and where the pixel set processing comprises processing the pixel set based on the identification of the section on the display.The reprojection component 198 can also be configured to filter luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range. The reprojection component 198 can also be configured to process a set of pixels corresponding to a section in a display that is associated with at least one region based on whether the luminance information for at least one region is within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range.The reprojection component 198 can also be configured to output an indication of the processed pixel set or the estimated transformation parameter set. The reprojection component 198 can also be configured to transform the display content corresponding to at least one region in the second intensity map based on the set of... Petition 870250094200, dated 10 / 15 / 2025, pages 292 / 382 17 / 69 estimated transformation parameters. The reprojection component 198 can also be configured to adjust a location of the display content corresponding to at least one region in the second intensity map if at least one of the following is true: (1) a frequency change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold.

[0045] Although the following description may focus on display processing, the concepts described in the present invention may be applicable to other similar processing techniques.

[0046] As described in the present invention, a device, such as device 104, may refer to any device, apparatus or system configured to perform one or more techniques described in the present invention. For example, a device may be a server, a base station, a user device, a client device, a station, an access point, a computer, for example a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation or a mainframe computer, an end product, an appliance, a telephone, a smartphone, a server, a video game platform or console, a handheld device, for example a portable video game device or a personal digital assistant (PDA), a wearable computing device, for example a smartwatch, an augmented reality device or a virtual reality device,a non-wearable device, a display or display device, a television, a television signal decoder, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an automotive onboard computer, any mobile device, any device configured to generate graphic content, or any device configured to perform one or more techniques described in, Petition 870250094200, dated 10 / 15 / 2025, pp. 293 / 382 18 / 69 present invention. The processes in the present invention can be described as being carried out by a particular component (e.g., a GPU) but, in further embodiments, can be carried out using other components (e.g., a CPU), consistent with the disclosed embodiments.

[0047] GPUs can process multiple types of data or data packets in a GPU pipeline. For example, in some aspects, a GPU can process two types of data or data packets, for example, context register packets and draw call data. A context register packet can be a set of global state information, for example, information regarding a global register, shading program, or constant data, which can regulate how a graphics context will be processed. For example, context register packets may include information regarding a color format. In some aspects of context register packets, there may be a bit that indicates which workload belongs to a context register. Furthermore, there may be multiple functions or programs running simultaneously and / or in parallel.For example, functions or programming can describe a certain operation, such as color mode or color format. Consequently, a context register can define multiple states of a GPU.

[0048] Context states can be used to determine how an individual processing unit operates, for example a vertex fetcher (VFD), a vertex shader (VS), a shader processor, or a geometry processor, and / or in which mode the processing unit operates. For this, GPUs can use context registers and programming data. In some respects, a GPU can generate a workload in the pipeline, for example a vertex or pixel workload, based on the context register definition of a mode or state. Certain processing units, for example a VFD, can use these states to determine certain functions, for example, how a vertex is assembled. Because these modes or states can change, GPUs can Petition 870250094200, dated 10 / 15 / 2025, pp. 294 / 382 19 / 69 need to change the corresponding context. Additionally, the workload that corresponds to the mode or state may follow the changing mode or state.

[0049] Figure 2 illustrates an example GPU 200 according to one or more techniques of the present disclosure. As shown in Figure 2, the GPU 200 includes command processor (CP) 210, draw call packets 212, VFD 220, VS 222, vertex cache (VPC) 224, triangle setup engine (TSE) 226, rasterizer (RAS rasterizer) 228, Z process engine (ZPE) 230, pixel interpolator (PI) 232, fragment shader (FS fragment shader) 234, render backend (RB) 236, level 1 cache (L1) (cluster cache (CCHE)) 237, level 2 cache (L2) (cluster cache (UCHE)) 238 and system memory 240.Although Figure 2 shows that GPU 200 includes processing units 220 to 238, GPU 200 may include several additional processing units. Furthermore, processing units 220 to 238 are merely an example, and any combination or order of processing units may be used by GPUs according to this disclosure. GPU 200 also includes command buffer 250, context register packets 260, and context states 261.

[0050] As shown in Figure 2, a GPU can use a CP, for example CP 210, or hardware accelerator to analyze a command buffer and obtain context register packets, for example, context register packets 260, and / or draw call data packets, for example, draw call packets 212. The CP 210 can then send the context register packets 260 or the draw call packets 212 through separate paths to the processing blocks or units on the GPU. Additionally, the command buffer 250 can switch between different context register and draw call states. For example, a command buffer can be structured as follows: context register of context N, draw call(s) of context N, context register of context N+1, and draw call(s) of context N+1. Petition 870250094200, dated 10 / 15 / 2025, pages 295 / 382 20 / 69

[0051] GPUs can render images in several different ways. In some cases, GPUs can render an image using tile rendering and / or tile rendering. In tile rendering GPUs, an image can be divided or separated into different sections or tiles. After the image is divided, each section or tile can be rendered separately. Tile rendering GPUs can divide computer graphics images into a grid format, so that each portion of the grid, i.e., a tile, is rendered separately. In some aspects, during a binning pass, an image can be divided into different bins or tiles. In some aspects, during the binning pass, a visibility stream can be constructed where visible primitives or draw calls can be identified. In contrast to tile rendering, direct rendering does not divide the frame into smaller bins or tiles.Instead, in direct rendering the entire frame is rendered at once. Additionally, some types of GPUs may allow both tile rendering and direct rendering.

[0052] Instructions executed by a CPU (e.g., software instructions) or a display processor may cause the CPU or display processor to search for and / or generate a composition strategy to compose a frame based on dynamic priority and runtime statistics associated with one or more composition strategy groups. A frame to be displayed by a physical display device, such as a display panel, may include a plurality of layers. Furthermore, the composition of the frame may be based on combining the plurality of layers in the frame (e.g., based on a frame buffer). After the plurality of layers is combined in the frame, the frame may be provided to the display panel for display on it. The process of combining each of the plurality of layers in the frame may be called composition, frame composition, a composition procedure, a composition process, or similar terms.

[0053] A frame composition procedure or composition strategy may correspond to a technique for composing different layers of the plurality of Petition 870250094200, dated 10 / 15 / 2025, pp. 296 / 382 21 / 69 layers in a single frame. The plurality of layers can be stored in doubled data rate (DDR) memory. Each layer in the plurality of layers can additionally correspond to a separate buffer. A compositor or hardware composer (HWC), associated with a block or function, can determine an input for each layer / buffer and perform the frame composition procedure to generate an output indicative of a composite frame. That is, the input can be the layers and the output can be a frame composition procedure to compose the frame to be displayed in the display panel.

[0054] Figure 3 is a block diagram 300 illustrating an example display structure that includes the processing unit 120, the system memory 124, the display processor 127 and the display(s) 131, as can be identified in relation to the device 104.

[0055] A GPU is generally included in devices that provide content for visual presentation on a display. For example, processing unit 120 may include a GPU 310 configured to render graphic data for display on a computing device (e.g., device 104), which may be a computer workstation, a mobile phone, a smartphone or other smart device, an embedded system, a personal computer, a tablet computer, a video game console, and the like. The operations of the GPU 310 may be controlled based on one or more graphics processing commands provided by a CPU 315. The CPU 315 may be configured to run multiple applications simultaneously. In some cases, each of the multiple applications running simultaneously may utilize the GPU 310 simultaneously.Processing techniques can be performed via the 120 processing unit, which emits a frame through physical or wireless communication channels.

[0056] System memory 124, which can be executed by processing unit 120, may include user space 320 and kernel space 325. User space 320 (sometimes called application space) may include software application(s) and / or application framework(s). For example, the Petition 870250094200, dated 10 / 15 / 2025, pp. 297 / 382 22 / 69 Software application(s) may include operating systems, media applications, graphics applications, workspace applications, etc. Application framework(s) may include frameworks used by one or more software applications, such as libraries, services (e.g., display services, input services, etc.), application program interfaces (APIs), etc. The kernel space 325 may additionally include a display driver 330. The display driver 330 may be configured to control the display processor 127. For example, the display driver 330 may cause the display processor 127 to compose a frame and transmit the frame data to a display.

[0057] The display processor 127 includes a display control block 335 and a display interface 340. The display processor 127 can be configured to handle functions of the display(s) 131 (for example, based on input received from the display driver 330). The display control block 335 can additionally be configured to output image frames to the display(s) 131 via the display interface 340. In some examples, the display control block 335 can also, or alternatively, perform post-processing of image data provided based on the execution of system memory 124 by the processing unit 120.

[0058] The display interface 340 can be configured to cause the display(s) 131 to display image frames. The display interface 340 can output image data to the display(s) 131 according to an interface protocol, such as MIPI DSI (MIPI DSI - Mobile Industry Processor Interface, Display Serial Interface). That is, the display(s) 131 can be configured according to MIPI DSI standards. The MIPI DSI standard supports a video mode and a command mode. In examples where the display(s) 131 is / are operating in video mode, the display processor 127 can continuously update the graphic content of the display(s) 131. For example, all graphic content can be updated per update cycle (e.g., line by line). In the examples where display(s) 131 is / are operating in command mode, the processor Petition 870250094200, dated 10 / 15 / 2025, pages 298 / 382 23 / 69 display 127 can record the graphic content of a frame in a 350 buffer.

[0059] In some of these examples, display processor 127 may not continuously update the graphic content of display(s) 131. Instead, display processor 127 may use a vertical synchronization pulse (Vsync) to coordinate the rendering and consumption of graphic content in buffer 350. For example, when a Vsync pulse is generated, display processor 127 may output new graphic content to buffer 350. Thus, the generation of the Vsync pulse may indicate that the current graphic content has been rendered in buffer 350.

[0060] The frames are displayed on display(s) 131 based on a display controller 345, a display client 355, and buffer 350. The display controller 345 can receive image data from the display interface 340 and store the received image data in buffer 350. In some examples, the display controller 345 can output the image data stored in buffer 350 to the display client 355. In this way, buffer 350 can represent local memory for display(s) 131. In some examples, the display controller 345 can output the image data received from the display interface 340 directly to the display client 355.

[0061] The display client 355 may be associated with a touch panel that detects interactions between a user and the display(s) 131. As the user interacts with the display(s) 131, one or more sensors on the touch panel may send signals to the display controller 345 indicating which of the one or more sensors have sensor activity, a duration of sensor activity, pressure applied to the one or more sensors, etc. The display controller 345 may use the sensor outputs to determine a manner in which the user interacted with the display(s) 131. The display(s) 131 may additionally be associated with / include other devices, such as a camera, a microphone, and / or a speaker, that operate in connection with the display client 355.

[0062] Some device processing techniques 104 can be performed over three stages (e.g., stage 1: a stage of Petition 870250094200, dated 10 / 15 / 2025, pp. 299 / 382 24 / 69 rendering; stage 2: a composition stage; and stage 3: a display / transfer stage). However, other processing techniques may combine the composition stage and the display / transfer stage into a single stage, so that the processing technique can be executed based on two total stages (e.g., stage 1: rendering stage; and stage 2: composition / display / transfer stage). During the rendering stage, the GPU 310 can process a content buffer based on the execution of an application that generates content on a pixel-by-pixel basis. During the composition and display stage(s), pixel elements can be assembled to form a frame that is transferred to a physical display subsystem / panel (e.g., displays 131) that displays the frame.

[0063] Instructions executed by a CPU (e.g., software instructions) or a display processor may cause the CPU or display processor to search for and / or generate a composition strategy to compose a frame based on dynamic priority and runtime statistics associated with one or more composition strategy groups. A frame to be displayed by a physical display device, such as a display panel, may include a plurality of layers. Furthermore, the composition of the frame may be based on combining the plurality of layers in the frame (e.g., based on a frame buffer). After the plurality of layers is combined in the frame, the frame may be provided to the display panel for display on it. The process of combining each of the plurality of layers in the frame may be called composition, frame composition, a composition procedure, a composition process, or similar terms.

[0064] A frame composition procedure or composition strategy may correspond to a technique for composing different layers of a plurality of layers into a single frame. The plurality of layers may be stored in doubled data rate (DDR) memory. Each layer of the plurality of layers may additionally correspond to a separate buffer. A Petition 870250094200, dated 10 / 15 / 2025, pp. 300 / 382 25 / 69 A compositor or hardware compositor (HWC), associated with a block or function, can determine an input from each layer / buffer and perform the frame composition procedure to generate an output indicative of a composite frame. That is, the input can be the layers and the output can be a frame composition procedure to compose the frame to be displayed on the display panel.

[0065] Some aspects of display processing may utilize different types of mask layers, for example, a format mask layer. A mask layer is a layer that can represent a portion of a display or display panel. For example, an area of ​​a mask layer may correspond to an area of ​​a display, but the entire mask layer may depict a portion of the content that is actually displayed on the display or panel.For example, a mask layer might include an upper and a lower portion of a display area, but the middle portion of the mask layer might be empty. In some examples, there might be multiple mask layers to represent different portions of a display area. Furthermore, for certain portions of a display area, the contents of different mask layers might overlap each other. Consequently, a mask layer might represent a portion of a display area that may or may not overlap other mask layers.

[0066] Figure 4 is a diagram 400 illustrating an example of a split architecture. More specifically, Figure 4 depicts a general split architecture 402, such as a split augmented reality (AR) architecture or a split extended reality (XR) architecture.As shown in Figure 4, the diagram 400 including general division architecture 402 includes a series of different steps, such as: rendering 412, encoding 414, packaging 416, unpacking 422, decoding 424, processing / transformation 426, and display 428. Figure 4 shows that a complementary system-on-a-chip (SoC) 410 can perform the rendering 412, encoding 414, and packaging 416 steps. Furthermore, a glass / device SoC 420 can perform the unpacking 422 and decoding 424 steps. Petition 870250094200, dated 10 / 15 / 2025, pages 301 / 382 26 / 69 processing / transformation 426 and display 428. Furthermore, Figure 4 shows a first time (t1), a second time (t2), and a third time (t3). The first time (t1) can occur during the rendering step 412, the second time (t2) can occur during the processing / transformation step 426, and a third time (t3) can occur during the display step 428. Additionally, there may be a WiFi connection (e.g., Wi-Fi 430) between the processing steps on the complementary SoC 410 and the processing steps on the glass / device SoC 420. Figure 4 also depicts different eyes and depths for a display device in the split AR architecture. For example, diagram 400 shows a right eye (RE), a right depth (RD), a left eye (LE), and a left depth (LD).As shown in Figure 4, the steps for right eye (RE), right depth (RD), left eye (LE), and left depth (LD) can occur during the processing steps in the complementary SoC 410 and the processing steps in the glass / device SoC 420.

[0067] As shown in Figure 4, the complementary SoC 410 can first perform the rendering step 412. This step can occur during the first time (t1). After the rendering step 412, the complementary SoC 410 can perform the encoding step 414. In some cases, the data / content associated with images / frames can be encoded during encoding 414. After the encoding step 414, the data / content can then go through a packaging step 416 (e.g., a packaging process or Real-Time Transport Protocol (RTP) packaging process). During the packaging process, the data / content can be converted into one or more frames.The frames can then be transmitted from the complementary SoC 410 to the glass / device SoC 420. As shown in Figure 4, this transmission can be performed via Wi-Fi 430. In some cases, the frames can be transmitted via a network protocol (e.g., a User Datagram Protocol (UDP), an Internet Protocol (IP) (UDP / IP), and a network protocol). In the glass / device SoC 420, the frames can be... Petition 870250094200, dated 10 / 15 / 2025, pages 302 / 382 27 / 69 received (e.g., received via a UDP / IP network protocol). The frames may also go through an unpacking step 422 or unpacking process (e.g., a Real-Time Transport Protocol (RTP) unpacking process), which may convert the data packets into data / content. After unpacking, the data / content may be decoded during the decoding step 424. After decoding, the glass / device SoC 420 may undergo the processing / transformation step 426. Finally, the decoded data / content may be sent to a display 428 (e.g., a headset or HMD) for display of the data / content.

[0068] As depicted in Figure 4, aspects of the split architecture may also include a late-stage reprojection.For example, as shown in Figure 4, eye and depth frames can be rendered on the complementary device for each eye (left eye and / or right eye) in certain time cases (e.g., at time t1). Furthermore, as shown in Figure 4, the device / glass (e.g., glass SoC / device 420) can receive these frames and unpack them in the unpacking step 422, decode the frames in the decoding step 424, process the frames in the processing / transformation step 426 (e.g., at time t2), and / or send the frames to the display in the display step 428 (e.g., at time t3). Additionally, the user can move or adjust a position during this time, and the rendered objects can be displayed in different places in the scene compared to what the user expects.As further shown in Figure 4, in order to account for this user movement and minimize visual discrepancy, the device may distort the eye patches based on the latest available pose information. For example, glass SoC / device 420 may undergo process / transformation step 426 (e.g., at time t2).

[0069] In some respects, certain types of rendering (e.g., light / brightness-aware rendering) can enhance visual content. For example, light / brightness-sensitive rendering can enhance the visual appeal of Petition 870250094200, dated 10 / 15 / 2025, pages 303 / 382 28 / 69 virtual content and can make a scene more immersive. Furthermore, virtual objects can have varying complexities that may be visible to the user under certain lighting conditions (e.g., lighting conditions assumed during rendering). However, when the brightness of the scene changes substantially (e.g., changes compared to a rendered time), some finer details in the virtual object may not be discernible to the user. For example, a reprojection pipeline may not be aware of any changes in relative brightness, which can lead to computation overload. Based on the above, it may be beneficial to reduce computational complexity in such scenarios. For example, it may be beneficial to reduce computational complexity in split rendering architectures during changes in relative brightness.In addition, it can be beneficial to reduce the amount of computation during these scenarios (e.g., changes in relative brightness).

[0070] Aspects of the present disclosure can reduce computational complexity in certain scenarios. For example, aspects presented in the present invention can reduce computational complexity in split-rendering architectures during scenarios where the brightness of a scene changes substantially compared to a rendered time. By reducing the computational complexity of the split-rendering architecture (e.g., when the scene brightness changes substantially compared to the rendered time), aspects presented in the present invention can reduce bandwidth in memory or interconnection. Furthermore, by reducing computational complexity when the scene brightness changes (e.g., changes substantially) compared to the rendered time, aspects presented in the present invention can potentially allow for an overall reduction in power consumption.As such, the aspects presented in the present invention can reduce computational complexity in devices with split rendering architectures and / or reduce overall power consumption in devices with split rendering architectures. Petition 870250094200, dated 10 / 15 / 2025, pages 304 / 382 29 / 69

[0071] Aspects presented in the present invention can utilize a method to reduce computational complexity in devices in split rendering architectures during certain scenarios (e.g., when the scene brightness changes substantially compared to the rendered time). For example, aspects presented in the present invention can obtain a brightness map (e.g., a brightness map obtained from the camera sensors). This brightness map can help determine the brightness in a scene, which can help identify when a scene's brightness will change substantially. The brightness map obtained from the camera sensors can be transformed based on the eye / display coordinate frame in order to generate an updated brightness map (e.g., brightness map b1).The updated brightness map can be further processed by an application engine to determine threshold limits (e.g., b2 threshold limits) based on the type of content being generated. After this, a distortion engine can sample the brightness map (b1 brightness map or b2 threshold limits) that spans a portion of a grid (e.g., a grid that the distortion engine is currently processing). In some respects, if the brightness level is within a threshold limit for that region, the distortion engine can proceed to process all pixels in the grid according to a chosen distortion procedure (e.g., w1 distortion procedure). In some respects, if the brightness level is outside a threshold limit, the distortion engine can process a set of pixels (e.g., primary pixels) according to the chosen distortion procedure (e.g., w1 distortion procedure).For the remaining pixels in the grid, the distortion mechanism can interpolate the values ​​of the secondary pixels based on the primary pixels. In some cases, there may be potentially large changes in brightness intensity in the scene. In these cases, the distortion mechanism can avoid processing all samples in the chosen grid, thus saving a significant amount of computation and / or energy consumption. Petition 870250094200, dated 10 / 15 / 2025, pages 305 / 382 30 / 69

[0072] The aspects presented in the present invention relate to optimizing reprojection based on contextual brightness. In some cases, virtual objects may have varying complexities that are visible to a user during rendering under certain lighting conditions. One problem is that the brightness of a scene can change substantially compared to the rendered time. The reprojection pipeline may not be aware of these relative brightness changes, which can lead to computation overload. The aspects presented in the present invention can utilize a brightness map obtained from camera sensors, which can be transformed based on an eye / display coordinate reference to generate a new brightness map (b1). The map can further be processed by an application mechanism in order to determine threshold limits (b2) based on the type of content being generated.Furthermore, the GPU / distortion engine can sample the brightness map (b1 or b2) encompassing the portion of a grid that the GPU / distortion engine is currently processing. Depending on the brightness level and whether the brightness level is within certain threshold limits for a region, the GPU / distortion engine can decide to process all pixels or a subset of pixels according to the chosen distortion procedure (w1). Aspects presented in the present invention can reduce computational complexity and / or reduce bandwidth for a memory / interconnect, which can result in an overall reduction in power consumption in the device.

[0073] Figure 5 is a 500 diagram and a 510 diagram, respectively, illustrating examples of scenes associated with display processing and / or graphics processing. More specifically, Figure 5 depicts an example of an environmental scene 502 in diagram 500 and an example of an environmental scene 512 in diagram 510. As shown in Figure 5, diagram 500 depicts an environmental scene 502 that includes objects 504 (e.g., trees and buildings). Similarly, diagram 510 depicts an environmental scene 512 that includes objects 514 (e.g., trees and buildings). However, unlike diagram 500, the environmental scene 512 in diagram 510 includes an eye / display aligned luminance map. As Petition 870250094200, dated 10 / 15 / 2025, pp. 306 / 382 31 / 69 shown in Figure 5, the eye / display aligned luminance map in environmental scene 512 includes a variety of different luminance measurements. These luminance measurements measure the luminance levels at various points in environmental scene 512. Furthermore, the luminance measurements of the eye / display aligned luminance map can allow aspects presented in the present invention to determine or measure changes in brightness intensity in the scene.

[0074] Figure 6 is a diagram 600 and a diagram 610, respectively, illustrating examples of scenes associated with display processing and / or graphics processing. More specifically, Figure 6 depicts an example of an environmental scene 602 in diagram 600 and an example of an environmental scene 612 in diagram 610. As shown in Figure 6, environmental scene 602 includes a virtual object 604 (e.g., a building) and a light 606. Similarly, environmental scene 612 includes a virtual object 614 (e.g., a building) and a light 616. As shown in Figure 6, diagram 600 depicts an environmental scene 602 where a light 606 (e.g., an ambient light) is on at a time when virtual object 604 is rendered. Consequently, virtual object 604 is visible. Furthermore, diagram 610 depicts the environmental scene 612 where the light 616 (e.g., an ambient light) is switched off at a point in the display. As such, the virtual object 614 is invisible.In fact, as shown in diagram 610, since light 616 is off at the time of display, virtual object 614 may not be displayed for an immersive experience (i.e., the virtual object is invisible).

[0075] Figure 7 is a diagram 700 and a diagram 710, respectively, illustrating examples of scenes associated with display processing and / or graphics processing. More specifically, Figure 7 depicts example scene 702 in diagram 700 and example scene 712 in diagram 710. Each of these scenes includes several similar objects. For example, scene 702 includes sun 704 and door 706, while scene 712 includes sun 704, door 706, and virtual object 708. As shown in Figure 7, diagram 700 depicts scene 702 which includes a sun 704, where door 706 is closed at render time. Furthermore, the diagram. Petition 870250094200, dated 10 / 15 / 2025, pages 307 / 382 32 / 69 Figure 710 depicts scene 712 which includes sun 704, door 706 and virtual object 708, where door 706 is open at the time of display. In diagram 710, the finer details of the design may not be distinguishable due to a bright light from sun 704.

[0076] Figure 8 is a diagram 800 which illustrates an example of a processing flow. More specifically, Figure 8 depicts a process flow 802 for reprojection optimization processes according to aspects of the present disclosure. As shown in Figure 8, diagram 800 includes sensors 810 which include camera sensors 812, luminance map 820, computer vision processor 830, eye / display aligned luminance map 840, GPU 850, input frame 860, output distorted frame 870 and display 880. As depicted in Figure 8, the sensors 810 which include camera sensors 812 can send luminance map 820 to the computer vision processor 830.Based on this, the computer vision processor 830 can configure or determine an eye / display aligned luminance map (e.g., eye / display aligned luminance map 840). After that, the computer vision processor 830 can then send the eye / display aligned luminance map 840 to the GPU 850. The GPU 850 can also obtain the eye / display aligned luminance map 840 and at least one input frame (e.g., input frame 860) in order to produce a distorted output frame (e.g., distorted output frame). The GPU 850 can then send the distorted output frame 870 to the display 880. In turn, the display 880 can display the distorted output frame 870. The process flow 802 in diagram 800 depicts the reprojection optimization processes according to aspects presented in the present invention.

[0077] Figure 9 is a 900 diagram illustrating another example of a processing flow. More specifically, Figure 9 depicts a 902 process flow for reprojection optimization processes according to aspects of the present disclosure. As shown in Figure 9, the 900 diagram includes a sensor intensity map 910, a coordinate transformation process 920, a Petition 870250094200, dated 10 / 15 / 2025, pages 308 / 382 33 / 69 eye / display coordinate aligned brightness map 930, a display grid 940, a GPU / distortion engine 950, an input frame 960, and a display resolution grid 970. As depicted in Figure 9, a sensor intensity map 910 can be sent to a coordinate transformation process 920, which can result in an eye / display coordinate aligned brightness map 930. The eye / display coordinate aligned brightness map 930 can include different regions in a display grid that are covered by the eye / display coordinate aligned brightness map 930. As shown in Figure 9, a region (e.g., grid region) in the display grid 940 covered by the brightness map (e.g., eye / display coordinate aligned brightness map 930) can include different types of pixels (e.g., primary pixels (p) and secondary pixels (s)). In Figure 9, the 940 display grid is shown in an enlarged view.The brightness map aligned to eye / display coordinates 930 and the input structure 960 can be obtained by the GPU / distortion engine 950. If the luminance in the grid region (e.g., region in the display grid 940) is within a threshold range (e.g., a luminance threshold), the GPU / distortion engine 950 can distort all pixels in the corresponding display grid region (e.g., region in the display resolution grid 970). If the luminance in the grid region (e.g., region in the display grid 940) is outside a threshold range (e.g., a luminance threshold), the GPU / distortion engine 950 can distort a subset of the pixels (e.g., the primary pixels) and / or interpolate other pixels (e.g., the secondary pixels) in the display grid region (e.g., region in the display resolution grid 970).If the luminance in the grid region (e.g., region in the display grid 940) is in an indistinguishable range (e.g., less than a minimum luminance threshold and greater than a maximum luminance threshold), the GPU / warp mechanism 950 may refrain from warping (i.e., not warping) the pixels in the corresponding display grid region (e.g., region in the display resolution grid). Petition 870250094200, dated 10 / 15 / 2025, pp. 309 / 382 34 / 69 970). This distortion process by the GPU / 950 distortion mechanism can result in the 970 display resolution grid.

[0078] Aspects presented in the present invention may include several benefits or advantages. For example, aspects presented in the present invention may reduce the computational complexity of split rendering architectures in certain scenarios (e.g., when scene brightness changes substantially compared to the rendered time). By reducing the computational complexity of the split rendering architecture (e.g., when scene brightness changes substantially compared to the rendered time), aspects presented in the present invention may reduce bandwidth in memory or interconnection. Furthermore, by reducing computational complexity when scene brightness changes (e.g., changes substantially) compared to the rendered time, aspects presented in the present invention may potentially allow for an overall reduction in power consumption.Consequently, the aspects presented in the present invention can reduce computational complexity in devices with split rendering architectures and / or reduce overall power consumption in devices with split rendering architectures.

[0079] In some respects, a certain layer in the split rendering architecture can be used to display important information (e.g., a notification, etc.). For example, a particular layer in the display devices (e.g., a layer fixed relative to the head) can display certain types of notifications or messages. Additionally, for a user of a display device in split rendering architectures, the light intensity of the display device may vary when the user is in certain types of environments (e.g., outdoor environments). Furthermore, the light intensity of the display device may vary when the user is in indoor environments where the lighting may change frequently (e.g., an environment with a light turned off and then on repeatedly). In such cases with variable light intensity, it may be Petition 870250094200, dated 10 / 15 / 2025, pp. 310 / 382 35 / 69 beneficial to automatically handle varying light intensity so that the user can view display devices more easily. For example, it may be beneficial to establish a method for automatically estimating the transformation parameters of a display device and providing this information.

[0080] Aspects of the present disclosure enable users of display devices to view the devices more easily during certain lighting environments (e.g., bright environments or environments with varying light intensity). For example, aspects presented in the present invention can provide a method for automatically estimating the transformation parameters of a display device and providing this information (e.g., sending this information to an application). That is, aspects presented in the present invention can utilize a spatial content reprojection based on light intensity.For example, aspects presented in the present invention can utilize an application that can decide to send certain parameters (e.g., transformation parameters) to a GPU or warp mechanism. Aspects presented in the present invention can also filter the parameters (e.g., transformation parameters) and / or prevent transformation from occurring. Aspects presented in the present invention can apply this process to several different layers for a split rendering architecture. For example, although this method can be applied to different layers, aspects presented in the present invention can utilize fixed content relative to the head for the split rendering architecture.

[0081] Aspects presented in the present invention can obtain an intensity map (e.g., intensity map r1) from certain types of sensors (e.g., camera sensors).The intensity map can be transformed based on eye / display coordinate frames in order to generate an updated intensity map (e.g., intensity map e1). The updated intensity map (e.g., intensity map e1) can be compared to an application threshold and / or a user-defined threshold. This threshold comparison can lead to distributed regions / blobs that are scattered across the map (e.g., the map of...). Petition 870250094200, dated 10 / 15 / 2025, pp. 311 / 382 36 / 69 intensity). Furthermore, the aspects presented in the present invention can traverse the map (e.g., the intensity map) to filter the largest area that is visually suitable for displaying content. The aspects presented in the present invention can also determine or calculate a grid-aligned box that can encapsulate the region / blob. Additionally, the aspects presented in the present invention can calculate (i.e., solve) certain types of parameters (e.g., scaling and translation parameters). In some cases, the aspects presented in the present invention can estimate additional parameters using different types of methods, such as homography. Furthermore, the aspects presented in the present invention can send / pass the calculated / estimated parameters from the previously mentioned step to an application mechanism.

[0082] In addition, the application engine can make a number of determinations or calculations. For example, the application engine can determine whether to move the content to a more visually ideal portion of the display. Similarly, the application engine can decide not to move the content to a more visually ideal portion of the display. Furthermore, if certain updates are occurring frequently or a transformation amount is large (e.g., greater than a threshold), the application engine can adjust (i.e., smooth) the changes. The application engine can then send the new parameters to a warp engine. In turn, the GPU / warp engine can warp the content (e.g., fixed content relative to the head) according to the parameters received from the application engine and / or generate certain data (e.g., end-of-frame data). This data can then be sent to the display engine.Furthermore, in some cases, the aspects presented in the present invention can send information (e.g., encapsulated blob information) to the application mechanism. In turn, the application mechanism can determine whether to proceed to the next steps (e.g., calculate or solve the aforementioned parameters, as mentioned in the previously mentioned steps). Petition 870250094200, dated 10 / 15 / 2025, pages 312 / 382 37 / 69

[0083] Figure 10 is a diagram 1000 and a diagram 1010, respectively, illustrating examples of scenes associated with display processing and / or graphics processing. More specifically, Figure 10 depicts an example scene on screen 1002 in diagram 1000 and an example scene on screen 1012 in diagram 1010. Each of the scenes in diagram 1000 and diagram 1010 depicts example application decision scenes. Diagram 1000 includes screen 1002 displaying virtual object 1004. Similarly, diagram 1010 includes screen 1012 displaying virtual object 1014. As shown in Figure 10, diagram 1000 depicts a scene where an application might decide to move virtual content (e.g., virtual object 1004) to another portion of screen 1002.Furthermore, diagram 1010 depicts a scene with variable luminance (e.g., fast variable luminance), where an application can decide to reduce the frequency of movement of virtual content (e.g., virtual object 1014) to another part of the screen 1012. By reducing the frequency of moving virtual content (e.g., virtual object 1014) to another part of the screen (e.g., screen 1012), the aspects presented in the present invention can reduce the amount of calculations and / or energy consumption in a display device.

[0084] Figure 11 is a diagram 1100 that illustrates an example of a processing flow. More specifically, Figure 11 depicts process flow 1102 (e.g., a high-level process flow) for the reprojection of spatial content according to aspects of the present disclosure. As shown in Figure 11, diagram 1100 includes sensors 1110 which include camera sensors 1112, luminance map 1120, computer vision processor 1130, estimated parameters 1140, user / application threshold map 1142, CPU 1150, decision 1152, input frame 1160, GPU or distortion mechanism 1170, output distorted frame 1172, and display 1180. As depicted in Figure 11, sensors 1110 which include camera sensors 1112 can send luminance map 1120 to the computer vision processor 1130. The computer vision processor 1130 can also receive an input frame 1160.Based on this, the 1130 computer vision processor can determine or estimate parameters (e.g., Petition 870250094200, dated 10 / 15 / 2025, pp. 313 / 382 38 / 69 estimated parameters 1140). Furthermore, the computer vision processor 1130 can send estimated parameters 1140 to the CPU 1150. The CPU 1150 can also receive a user / application threshold map 1142. Based on this, the CPU 1150 can make a decision 1152 and then transmit an indication of the decision 1152 to the GPU / distortion mechanism 1170. The GPU / distortion mechanism 1170 can also receive the input frame 1160. Based on this, the GPU / distortion mechanism 1170 can configure the distorted output frame 1172 and then transmit the distorted output frame 1172 to the display 1180. Then, the display 1180 can display the distorted output frame 1172. The process flow 1102 in diagram 1100 depicts the spatial content reprojection processes of in accordance with aspects presented in the present invention.

[0085] Figure 12 is a diagram 1200 that illustrates an example of a processing flow. More specifically, Figure 12 depicts a process flow 1202 for the reprojection of spatial content according to aspects of the present disclosure. As shown in Figure 12, the diagram 1200 includes a sensor intensity map 1210 (e.g., a camera sensor intensity map), a coordinate transformation step 1220 (e.g., a step to transform coordinates), a luminance map aligned to eye / display coordinates 1230, a threshold and mapping / filtering function 1240, and a filtered intensity map 1250. Figure 12 also shows a previously generated light intensity map 1242 and a user / application-suitable threshold map 1244.As depicted in Figure 12, the sensor intensity map 1210 can be sent to the coordinate transformation step 1220, which can then be sent to the eye / display coordinate-aligned luminosity map 1230. Figure 12 illustrates that the eye / display coordinate-aligned luminosity map 1230 can include varying light intensities, which are represented by different shaded grids (e.g., unshaded grids, grids with similar shading, grids with medium shading, and grids with heavy shading). Petition 870250094200, dated 10 / 15 / 2025, pp. 314 / 382 39 / 69 The result of the eye / display coordinate-aligned luminosity map 1230 can be communicated to the threshold and mapping / filtering function 1240. The output of the threshold and mapping / filtering function 1240 can be the filtered intensity map 1250. In some respects, the threshold and mapping / filtering function 1240 can receive a previously generated light intensity map 1242 and / or a user / application-suitable threshold map 1244. In turn, this can be used to create the filtered intensity map 1250.

[0086] Figure 13 is a diagram 1300 illustrating an example of a processing flow. More specifically, Figure 13 depicts a process flow 1302 for spatial content reprojection according to aspects of the present disclosure, which may be a continuation of the process flow 1202 shown in Figure 12.As shown in Figure 13, diagram 1300 includes an input frame 1310 containing information / data 1312, a filtered intensity map 1250 (e.g., the filtered intensity map 1250 from Figure 12), a computer vision processor 1320, and estimated parameters 1330. As depicted in Figure 13, the input frame 1310 (with information / data 1312) and the filtered intensity map 1250 (e.g., the filtered intensity map 1250 from the previous step in diagram 1200) can be sent to the computer vision processor 1320. In some cases, using homography mapping, the computer vision processor 1320 can estimate or solve a number of different parameters (e.g., the scaling factor, translation parameters, and / or other parameters). As shown in Figure 13, this estimate or calculation of the computer vision processor 1320 can result in estimated parameters 1330.As further shown in Figure 13, these estimated parameters 1330 can then be sent to a subsequent pipeline processing step.

[0087] Figure 14 is a diagram 1400 that illustrates an example of processing flow. More specifically, Figure 14 depicts a process flow 1402 for spatial content reprojection according to aspects of the present. Petition 870250094200, dated 10 / 15 / 2025, pages 315 / 382 40 / 69 disclosure, which may be a continuation of process flow 1302 shown in Figure 13. As shown in Figure 14, diagram 1400 includes input frame 1310 containing information / data 1312 (e.g., input frame 1310 containing information / data 1312 from Figure 13), estimated parameters 1330 (e.g., estimated parameters 1330 from Figure 13), application computing mechanism 1430 (e.g., a CPU), a decision step 1440, GPU / distortion mechanism 1450, and output distorted frame 1460 containing information / data 1462. As depicted in Figure 14, certain parameters (e.g., estimated parameters 1330 from the previous step in diagram 1300) may be sent to the application computing mechanism 1430 (e.g., CPU). Based on the estimated parameters 1330, the application computing mechanism 1430 (e.g., CPU) can make a decision in the decision step 1440.This decision from decision step 1440 (e.g., a 'yes' decision or a 'no' decision) can be sent to the GPU / distortion engine 1450, along with the input frame 1310 which includes information / data 1312 (e.g., the input frame 1310 which includes information / data 1312 from the previous step in diagram 1300). Based on this, the GPU / distortion engine 1450 can set up the distorted output frame 1460 which includes information / data 1462. The GPU / distortion engine 1450 can then transmit the distorted output frame 1460 which includes information / data 1462. For example, the GPU / distortion engine 1450 can send the distorted output frame 1460 which includes information / data 1462 to a display.

[0088] Aspects presented in the present invention may include several benefits or advantages. For example, aspects of the present disclosure allow users of display devices to view the devices more easily during certain lighting environments (e.g., bright environments or environments with varying light intensity). Aspects presented in the present invention may provide a method for automatically estimating the transformation parameters of a display device and providing this information (e.g., Petition 870250094200, dated 10 / 15 / 2025, pages 316 / 382 41 / 69 send this information to an application). In fact, the aspects presented in the present invention can utilize a spatial content reprojection based on light intensity. For example, the aspects presented in the present invention can utilize an application that can decide to send certain parameters (e.g., transformation parameters) to a GPU or distortion mechanism. Furthermore, the aspects presented in the present invention can also filter the parameters (e.g., transformation parameters) and / or prevent transformation from occurring. Aspects presented in the present invention can apply this process to several different layers for a split rendering architecture. For example, although this method can be applied to different layers, the aspects presented in the present invention can utilize fixed content relative to the head for the split rendering architecture.

[0089] Figure 15 is a 1500 communication flow diagram of data processing or graphics processing according to one or more techniques of this disclosure. As shown in Figure 15, the 1500 diagram includes example communications between GPU 1502 (e.g., a GPU, a cache in a GPU, a GPU component, another graphics processor, a CPU, a CPU component, or another central processing unit), CPU 1504 (e.g., a CPU, a cache in a CPU, a CPU component, another central processing unit, a GPU, a GPU component, or another graphics processor), and memory 1506 (e.g., system memory, graphics memory, or memory or cache in a GPU), according to one or more techniques of this disclosure.

[0090] In 1510, the GPU 1502 can obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, where the first intensity map corresponds to a camera associated with the scene. In some respects, obtaining the first intensity map may encompass obtaining the first intensity map from a set of camera sensors for the camera associated with the scene.

[0091] In 1520, GPU 1502 can configure a second intensity map. Petition 870250094200, dated 10 / 15 / 2025, pp. 317 / 382 42 / 69 based on the first intensity map and at least one coordinate frame from a device user's perspective to display scene-associated content, where the second intensity map is associated with luminance information for the scene. In some respects, the configuration of the second intensity map may involve transforming the first intensity map to generate the second intensity map based on the first intensity map and at least one coordinate frame from the device user's perspective to the scene-associated display content.

[0092] In 1530, GPU 1502 can determine whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene. The suitable luminance range can be greater than a minimum luminance threshold and less than a maximum luminance threshold. In addition, at least one of the minimum luminance threshold or the maximum luminance threshold can be an application-defined threshold or a user-defined threshold.In some respects, determining whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range may involve transmitting a first indication of the luminance information for at least one region; and receiving a second indication of whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range for the display content. Furthermore, receiving the second indication may involve receiving the second indication from a graphics processing unit (GPU) or an application engine in a central processing unit (CPU).

[0093] In 1540, the GPU 1502 can identify the section on the display that is associated with at least one region based on luminance information for at least one region being within the appropriate luminance range or outside the appropriate luminance range and where pixel set processing comprises processing the pixel set based on the section identification in Petition 870250094200, dated 10 / 15 / 2025, pages 318 / 382 43 / 69 display.

[0094] In 1550, GPU 1502 can filter luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range. In some respects, if the luminance information for at least one region is within the appropriate luminance range, the luminance information for at least one region may be visually appropriate to the display content associated with the scene. Furthermore, if the luminance information for at least one region is within the appropriate luminance range, a grid-aligned section in the second intensity map may be associated with at least one region.

[0095] In 1560, the 1502 GPU can process a set of pixels corresponding to a section on a display that is associated with at least one region based on luminance information for the at least one region being at least one within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for the at least one region in the second intensity map based on whether the luminance information for the at least one region is within the appropriate luminance range. In some respects, processing the set of pixels corresponding to the section on the display may involve transforming the entire set of pixels corresponding to the section on the display based on luminance information for the at least one region being within the appropriate luminance range.Furthermore, the GPU (e.g., GPU 1502) can transmit to the display the transformed set of pixels corresponding to the section on the display. In some cases, the transformation of the set of pixels corresponding to the section on the display may involve distorting the set of pixels corresponding to the section on the display. Additionally, a portion of the transformed set of pixels may be associated with at least one of the following: Petition 870250094200, dated 10 / 15 / 2025, pp. 319 / 382 44 / 69 movement of the device user's head towards the display content, movement of the device user's body, or the device user's surrounding environment. In some respects, processing the pixel set corresponding to the section on the display may involve transforming a subset of pixels into the pixel set corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range. Furthermore, the GPU (e.g., GPU 1502) may transmit the transformed subset of pixels corresponding to the section on the display to the display.In some cases, the GPU (e.g., GPU 1502) may estimate a color value from a remaining amount of the pixel set, where the remaining amount of the pixel set is equal to an amount of the pixel set that does not include the subset of pixels, where the estimated color value of the remaining amount of the pixel set is interpolated based on the subset of pixels. Furthermore, the transformation of the subset of pixels corresponding to the section on the display may involve distorting the subset of pixels corresponding to the section on the display. Additionally, an amount of the transformed subset of pixels may be associated with at least one of: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment.In some respects, processing the pixel set corresponding to the section on the display may involve refraining from transforming a subset of pixels into the pixel set corresponding to the section on the display based on luminance information for at least one region being outside the proper luminance range and being within the indistinguishable luminance range, where the indistinguishable luminance range is less than a minimum luminance threshold and greater than a maximum luminance threshold, and where the indistinguishable luminance range corresponds to a time for the user to distinguish the display content that is greater than a distinguishable time limit.

[0096] In 1570, the 1502 GPU can output an indication of the pixel set. Petition 870250094200, dated 10 / 15 / 2025, pp. 320 / 382 45 / 69 processed or estimated transformation parameter set. In some respects, the output of the processed pixel set or estimated transformation parameter set indication may comprise transmitting, to at least one of: a distortion mechanism or an application mechanism in a central processing unit (CPU), the processed pixel set or estimated transformation parameter set indication. For example, GPU 1502 may transmit indication 1572 to CPU 1504. Furthermore, the output of the processed pixel set or estimated transformation parameter set indication may comprise storing, in primary memory or a cache, the processed pixel set or estimated transformation parameter set indication. For example, GPU 1502 may store indication 1574 in memory 1506.Furthermore, the transformation parameter set may include at least one of the following: a scaling parameter set or a translation parameter set.

[0097] In 1580, GPU 1502 can transform the display content corresponding to at least one region in the second intensity map based on the estimated transformation parameter set. In some respects, after the transformation, the GPU (e.g., GPU 1502) can transmit the transformed display content corresponding to at least one region in the second intensity map.

[0098] Furthermore, in 1580, GPU 1502 can adjust a display content location corresponding to at least one region in the second intensity map if at least one of the following is true: (1) a frequency change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold. In some respects, the GPU (e.g., GPU 1502) can estimate a set of updated transformation parameters associated with luminance information for at least one region based on the adjusted location of the content. Petition 870250094200, dated 10 / 15 / 2025, pages 321 / 382 46 / 69 display corresponding to at least one region. The GPU (e.g., GPU 1502) can also output the estimated updated transformation parameter set associated with luminance information for at least one region.

[0099] Figure 16 is a 1600 flowchart of an example method of data processing or graphics processing according to one or more techniques of this disclosure. The method can be performed by a GPU (for example, a GPU, a cache in a GPU, a GPU component, another graphics processor, a CPU, a CPU component or another central processing unit), a CPU (for example, a CPU, a cache in a CPU, a CPU component, another central processing unit, a GPU, a GPU component or another graphics processor), a display driver integrated circuit (DDIC), an apparatus for data or graphics processing, a wireless communication device and / or any apparatus that can perform data or graphics processing, as used in connection with the examples in Figures 1 to 15.

[0100] In 1602, the GPU can obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, where the first intensity map corresponds to a camera associated with the scene, as described in relation to the examples in Figures 1 to 15. For example, as described in 1510 of Figure 15, GPU 1502 can obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, where the first intensity map corresponds to a camera associated with the scene. Additionally, step 1602 can be performed by processing unit 120 in Figure 1. In some respects, obtaining the first intensity map may involve obtaining the first intensity map from a set of camera sensors for the camera associated with the scene.

[0101] In 1604, the GPU can set up a second intensity map based on the first intensity map and at least one coordinate frame from a user's perspective on a device to display content. Petition 870250094200, dated 10 / 15 / 2025, pages 322 / 382 47 / 69 associated with the scene, where the second intensity map is associated with luminance information for the scene, as described in relation to the examples in Figures 1 to 15. For example, as described in 1520 of Figure 15, GPU 1502 can configure a second intensity map based on the first intensity map and at least one coordinate frame from a device user's perspective to display scene-associated content, where the second intensity map is associated with luminance information for the scene. Additionally, step 1604 can be performed by processing unit 120 in Figure 1. In some respects, the configuration of the second intensity map may involve transforming the first intensity map to generate the second intensity map based on the first intensity map and at least one coordinate frame from the device user's perspective to the scene-associated display content.

[0102] In 1606, the GPU can determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the display content associated with the scene, as described in relation to the examples in Figures 1 to 15. For example, as described in 1530 of Figure 15, GPU 1502 can determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the display content associated with the scene. Additionally, step 1606 can be performed by processing unit 120 in Figure 1. The suitable luminance range can be greater than a minimum luminance threshold and less than a maximum luminance threshold. Furthermore, at least one of the minimum luminance threshold or the maximum luminance threshold can be an application-defined threshold or a user-defined threshold.In some respects, determining whether the luminance information for at least one region on the second intensity map is within the appropriate luminance range may involve transmitting an initial indication of the luminance information for at least one region, and receiving a second. Petition 870250094200, dated 10 / 15 / 2025, pp. 323 / 382 48 / 69 indicates whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range for the display content. Additionally, receiving the second indication may include receiving the second indication from a graphics processing unit (GPU) or an application engine in a central processing unit (CPU).

[0103] In 1612, the GPU can process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region being at least one within the proper luminance range, outside the proper luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the proper luminance range, as described in relation to the examples in Figures 1 to 15.For example, as described in 1560 of Figure 15, GPU 1502 can process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region being at least one within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range. Additionally, step 1612 can be performed by processing unit 120 in Figure 1.In some respects, processing the set of pixels corresponding to the section on the display may involve transforming the entire set of pixels corresponding to the section on the display based on luminance information so that at least one region is within the appropriate luminance range. Additionally, the GPU (e.g., GPU 1502). Petition 870250094200, dated 10 / 15 / 2025, pp. 324 / 382 49 / 69 can transmit to the display the transformed set of pixels corresponding to the section on the display. In some cases, the transformation of the set of pixels corresponding to the section on the display may involve distorting the set of pixels corresponding to the section on the display. Furthermore, a portion of the transformed set of pixels may be associated with at least one of the following: movement of the device user's head toward the display content, movement of the device user's body, or the user's surrounding environment. In some respects, the processing of the set of pixels corresponding to the section on the display may involve transforming a subset of pixels within the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range.Furthermore, the GPU (e.g., GPU 1502) can transmit to the display the transformed subset of pixels corresponding to the section on the display. In some cases, the GPU (e.g., GPU 1502) can estimate a color value from a remaining amount of the pixel set, where the remaining amount of the pixel set is equal to an amount of the pixel set that does not include the subset of pixels, where the estimated color value of the remaining amount of the pixel set is interpolated based on the subset of pixels. Additionally, the transformation of the subset of pixels corresponding to the section on the display may involve distorting the subset of pixels corresponding to the section on the display. Furthermore, an amount of the transformed subset of pixels may be associated with at least one of: movement of the device user's head towards the display content, movement of the device user's body, or the device user's surrounding environment.In some respects, processing the set of pixels corresponding to the section on the display may involve refraining from transforming a subset of pixels into the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range and being within the range. Petition 870250094200, dated 10 / 15 / 2025, pages 325 / 382 50 / 69 indistinguishable luminance range, where the indistinguishable luminance range is less than a minimum luminance threshold and greater than a maximum luminance threshold, and where the indistinguishable luminance range corresponds to a time for the user to distinguish the display content that is greater than a distinguishable time threshold.

[0104] In 1614, the GPU can output an indication of the set of pixels processed or the set of transformation parameters estimated, as described in relation to the examples in Figures 1 to 15. For example, as described in 1570 of Figure 15, GPU 1502 can output an indication of the set of pixels processed or the set of transformation parameters estimated. Additionally, step 1614 can be performed by processing unit 120 in Figure 1.In some respects, the output of the indication of the processed pixel set or the estimated transformation parameter set may comprise transmitting, to at least one of: a distortion mechanism or an application mechanism in a central processing unit (CPU), the indication of the processed pixel set or the estimated transformation parameter set. For example, GPU 1502 may transmit the indication 1572 to CPU 1504. Furthermore, the output of the indication of the processed pixel set or the estimated transformation parameter set may comprise storing, in primary memory or a cache, the indication of the processed pixel set or the estimated transformation parameter set. For example, GPU 1502 may store the indication 1574 in memory 1506.Furthermore, the transformation parameter set may include at least one of the following: a scaling parameter set or a translation parameter set.

[0105] Figure 17 is a flowchart 1700 of an example method of data processing or graphics processing according to one or more techniques of the present disclosure. The method can be performed by a GPU (e.g., a GPU, a cache on a GPU, a GPU component, another graphics processor), a CPU, a CPU component, or another processor. Petition 870250094200, dated 10 / 15 / 2025, pp. 326 / 382 51 / 69 central), a CPU (for example, a CPU, a cache in a CPU, a CPU component, another central processor, a GPU, a GPU component, or another graphics processor), a display driver integrated circuit (DDIC), an apparatus for data or graphics processing, a wireless communication device, and / or any apparatus that can perform data or graphics processing, as used in relation to the examples in Figures 1 to 15.

[0106] In 1702, the GPU can obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene, as described in relation to the examples in Figures 1 to 15.For example, as described in 1510 of Figure 15, GPU 1502 can obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, where the first intensity map corresponds to a camera associated with the scene. Additionally, step 1702 can be performed by processing unit 120 in Figure 1. In some respects, obtaining the first intensity map may involve obtaining the first intensity map from a set of camera sensors for the camera associated with the scene.

[0107] In 1704, the GPU can configure a second intensity map based on the first intensity map and at least one coordinate frame from a user's perspective on a device to display scene-associated content, where the second intensity map is associated with luminance information for the scene, as described in relation to the examples in Figures 1 to 15. For example, as described in 1520 of Figure 15, GPU 1502 can configure a second intensity map based on the first intensity map and at least one coordinate frame from a user's perspective on a device to display scene-associated content, where the second intensity map is associated with luminance information for the scene. Additionally, step 1704 can be performed by the unit of Petition 870250094200, dated 10 / 15 / 2025, pp. 327 / 382 52 / 69 processing 120 in Figure 1. In some respects, the configuration of the second intensity map may involve transforming the first intensity map to generate the second intensity map based on the first intensity map and at least one coordinate frame from the device user's perspective to the display content associated with the scene.

[0108] In 1706, the GPU can determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the display content associated with the scene, as described in relation to the examples in Figures 1 to 15. For example, as described in 1530 of Figure 15, GPU 1502 can determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the display content associated with the scene. Additionally, step 1706 can be performed by processing unit 120 in Figure 1. The suitable luminance range can be greater than a minimum luminance threshold and less than a maximum luminance threshold. Furthermore, at least one of the minimum luminance threshold or the maximum luminance threshold can be an application-defined threshold or a user-defined threshold.In some respects, determining whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range may involve transmitting a first indication of the luminance information for at least one region; and receiving a second indication of whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range for the display content. Furthermore, receiving the second indication may involve receiving the second indication from a graphics processing unit (GPU) or an application engine in a central processing unit (CPU).

[0109] In 1708, the GPU can identify the section on the display that is associated with at least one region based on luminance information for that region being within or outside the appropriate luminance range. Petition 870250094200, dated 10 / 15 / 2025, pages 328 / 382 53 / 69 of adequate luminance and where pixel set processing may comprise processing the pixel set based on section identification on the display, as described in relation to the examples in Figures 1 to 15. For example, as described in 1540 of Figure 15, GPU 1502 may identify the section on the display that is associated with at least one region based on luminance information for at least one region being within the adequate luminance range or outside the adequate luminance range, and where pixel set processing may comprise processing the pixel set based on section identification on the display. Additionally, step 1708 may be performed by processing unit 120 in Figure 1.

[0110] In 1710, the GPU can filter luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range, as described in relation to the examples in Figures 1 to 15. For example, as described in 1550 of Figure 15, GPU 1502 can filter luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range. Additionally, step 1710 can be performed by processing unit 120 in Figure 1. In some respects, if the luminance information for at least one region is within the appropriate luminance range, the luminance information for at least one region may be visually appropriate to the display content associated with the scene.Furthermore, if the luminance information for at least one region is within the appropriate luminance range, a grid-aligned section in the second intensity map may be associated with at least one region.

[0111] In 1712, the GPU can process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region, with at least one being within the appropriate luminance range, outside of Petition 870250094200, dated 10 / 15 / 2025, pp. 329 / 382 54 / 69 adequate luminance range or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the adequate luminance range, as described in relation to the examples in Figures 1 to 15.For example, as described in 1560 of Figure 15, GPU 1502 can process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region being at least one within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range. Additionally, step 1712 can be performed by processing unit 120 in Figure 1.In some respects, processing the sectional pixel set on the display may involve transforming the entire sectional pixel set based on luminance information so that at least one region is within the appropriate luminance range. Additionally, the GPU (e.g., GPU 1502) may transmit the transformed sectional pixel set to the display. In some cases, the transformation of the sectional pixel set may involve distorting the sectional pixel set. Furthermore, a portion of the transformed pixel set may be associated with at least one of: the user's head movement toward the display content, the user's body movement, or the user's surrounding environment.In some respects, processing the set of pixels corresponding to the section on the display may involve transforming one. Petition 870250094200, dated 10 / 15 / 2025, pp. 330 / 382 55 / 69 subset of pixels in the pixel set corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range. Additionally, the GPU (e.g., GPU 1502) can transmit the transformed subset of pixels corresponding to the section on the display to the display. In some cases, the GPU (e.g., GPU 1502) can estimate a color value from a remaining amount of the pixel set, where the remaining amount of the pixel set is equal to an amount of the pixel set that does not include the subset of pixels, where the estimated color value of the remaining amount of the pixel set is interpolated based on the subset of pixels. Furthermore, the transformation of the subset of pixels corresponding to the section on the display may involve distorting the subset of pixels corresponding to the section on the display.Furthermore, a portion of the transformed subset of pixels may be associated with at least one of the following: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment. In some respects, processing the section-matched pixel set on the display may involve refraining from transforming a subset of pixels into the section-matched pixel set on the display based on luminance information for at least one region being outside the appropriate luminance range and within the indistinguishable luminance range, where the indistinguishable luminance range is less than a minimum luminance threshold and greater than a maximum luminance threshold, and where the indistinguishable luminance range corresponds to a time for the user to distinguish the display content that is greater than a distinguishable time threshold.

[0112] In 1714, the GPU can output an indication of the set of pixels processed or the set of transformation parameters estimated, as described in relation to the examples in Figures 1 to 15. For example, as described in 1570 of Figure 15, the GPU 1502 can output an indication of the set of pixels processed or the set of transformation parameters estimated. Petition 870250094200, dated 10 / 15 / 2025, pp. 331 / 382 56 / 69 Additionally, step 1714 can be performed by processing unit 120 in Figure 1. In some respects, the output of the indication of the processed pixel set or the estimated transformation parameter set may involve transmitting, to at least one of: a distortion mechanism or an application mechanism in a central processing unit (CPU), the indication of the processed pixel set or the estimated transformation parameter set. For example, GPU 1502 may transmit indication 1572 to CPU 1504. Furthermore, the output of the indication of the processed pixel set or the estimated transformation parameter set may involve storing, in a primary memory or a cache, the indication of the processed pixel set or the estimated transformation parameter set. For example, GPU 1502 may store indication 1574 in memory 1506.Furthermore, the transformation parameter set may include at least one of the following: a scaling parameter set or a translation parameter set.

[0113] In 1716, the GPU can transform the display content corresponding to at least one region in the second intensity map based on the estimated transformation parameter set, as described in relation to the examples in Figures 1 to 15. For example, as described in 1580 of Figure 15, GPU 1502 can transform the display content corresponding to at least one region in the second intensity map based on the estimated transformation parameter set. Additionally, step 1716 can be performed by processing unit 120 in Figure 1. In some respects, after the transformation, the GPU (e.g., GPU 1502) can transmit the transformed display content corresponding to at least one region in the second intensity map.

[0114] Furthermore, in 1716, the GPU can adjust a display content location corresponding to at least one region in the second intensity map if at least one of the following: (1) a frequency change in a set of display coordinates for the corresponding display content Petition 870250094200, dated 10 / 15 / 2025, pages 332 / 382 57 / 69 at least one region is less than a frequency threshold, or (2) a display content size corresponding to at least one region is greater than a size threshold, as described in relation to the examples in Figures 1 to 15. For example, as described in 1580 of Figure 15, GPU 1502 can adjust a display content location corresponding to at least one region in the second intensity map if at least one of the following: (1) a change frequency in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a display content size corresponding to at least one region is greater than a size threshold. Additionally, step 1716 can be performed by processing unit 120 in Figure 1.In some respects, the GPU (e.g., GPU 1502) can estimate a set of updated transformation parameters associated with luminance information for at least one region based on the adjusted location of the display content corresponding to at least one region. The GPU (e.g., GPU 1502) can also output the estimated updated set of transformation parameters associated with luminance information for at least one region.

[0115] In configurations, a method or apparatus for data or graphics processing is provided. The apparatus may be a GPU (or other graphics processor), a CPU (or other central processing unit), a DDIC, a graphics processing apparatus, and / or some other processor that can perform data or graphics processing. In certain respects, the apparatus may be the processing unit 120 within the device 104, or it may be some other hardware within the device 104 or another device. The apparatus, for example, processing unit 120, may include means for obtaining a first intensity map associated with luminance information for a scene that includes a plurality of frames, where the first intensity map corresponds to a camera associated with the scene. The apparatus, for example, processing unit 120, may also include means for Petition 870250094200, dated 10 / 15 / 2025, pages 333 / 382 58 / 69 configure a second intensity map based on the first intensity map and at least one coordinate frame from a user's perspective of a device to display scene-associated content, where the second intensity map is associated with luminance information for the scene. The device, for example, processing unit 120, may also include means to determine whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the scene-associated display content.The device, for example, processing unit 120, may also include means for processing a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region, with at least one region being within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or means for estimating a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range. The device, for example, processing unit 120, may also include means for emitting an indication of the processed set of pixels or the estimated set of transformation parameters.The device, for example, processing unit 120, may also include means to identify the section on the display that is associated with at least one region based on luminance information for that region being within the appropriate luminance range or outside the appropriate luminance range. The device, for example, processing unit 120, may also include means to filter the luminance information for that region in the second intensity map if the luminance information for that region is within the appropriate luminance range. The device, for example, processing unit 120, may also include means to adjust. Petition 870250094200, dated 10 / 15 / 2025, pp. 334 / 382 59 / 69 a location of the display content corresponding to at least one region in the second intensity map if at least one of the following: (1) a frequency change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold. The device, for example, processing unit 120, may also include means for transforming the display content corresponding to at least one region in the second intensity map based on the set of estimated transformation parameters.

[0116] The matter described in the present invention can be implemented to obtain one or more benefits or advantages. For example, the described graphics processing or display techniques can be used by a GPU, a CPU, a central processing unit, or some other processor capable of performing graphics or display processing to implement the content reprojection techniques described in the present invention. This can also be done at a low cost compared to other graphics processing or display techniques. Furthermore, the graphics processing or display techniques of the present invention can improve or accelerate data execution or processing. Additionally, the graphics processing or display techniques of the present invention can improve resource or data utilization and / or resource efficiency.Additionally, aspects of this disclosure may utilize content reprojection techniques to improve memory bandwidth efficiency and / or increase processing speed in a cache, GPU, CPU, or DPU.

[0117] It is understood that the specific order or hierarchy of blocks in the disclosed processes / flowcharts is an illustration of example approaches. Based on design preferences, it should be understood that the specific order or hierarchy of blocks in the processes / flowcharts may be rearranged. In addition, some blocks may be combined or omitted. The method claims in Petition 870250094200, dated 10 / 15 / 2025, pp. 335 / 382 Annexes 60 / 69 present elements from the various blocks in a sample order, and are not intended to be limited to the specific order or hierarchy presented.

[0118] The above description is provided to enable any person skilled in the art to practice the various aspects described in the present invention. Various modifications of these aspects will be readily apparent to those skilled in the art, and the generic principles defined in the present invention may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown in the present invention, but should be considered to be in harmony with the complete scope and consistent with the language of the claims, wherein a reference to an element in the singular is not intended to mean one and only one, except where specifically so indicated, but rather one or more. The word exemplifier is used in the present invention to mean serving as an example, an instance, or an illustration.Any aspect described in the present invention as exemplary should not necessarily be interpreted as preferential or advantageous in relation to other aspects.

[0119] Except where specifically indicated otherwise, the term "some" refers to one or more and the term "or" may be interpreted as "and / or" where the context does not indicate otherwise. Combinations such as at least one of A, B or C, one or more of A, B or C, at least one of A, B and C, one or more of A, B and C and A, B, C or any combination thereof include any combination of A, B and / or C and may include multiples of A, multiples of B or multiples of C. Specifically, combinations such as at least one of A, B or C, one or more of A, B or C, at least one of A, B and C, one or more of A, B and C and A, B and C or any combination thereof may be only A, only B, only C, A and B, A and C, B and C or A and B and C, where any of these combinations may contain one or more members from A, B or C.All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or may become known to those skilled in the art are expressly incorporated in the present invention. Petition 870250094200, dated 10 / 15 / 2025, pages 336 / 382 61 / 69 are by way of reference and are intended to be covered by the claims. Furthermore, nothing disclosed in the present invention is intended to be exclusive to the public, regardless of whether such disclosure is explicitly mentioned in the claims. The words module, mechanism, element, device and the like may not be a substitute for the word means. Thus, no claimed element should be interpreted as a means plus function unless the element is expressly mentioned using the phrase means for.

[0120] In one or more examples, the functions described in the present invention may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term processing unit has been used throughout this disclosure, such processing units may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technique described in the present invention, or other module, is implemented in software, the function, processing unit, technique described in the present invention, or other module, may be stored or transmitted in the form of one or more instructions or codes in a computer-readable medium.

[0121] According to this disclosure, the term "or" can be interpreted as "and / or" where the context does not indicate otherwise. Additionally, although phrases such as "one or more" or "at least one" or similar may have been used for some attributes disclosed in the present invention but not for others, the attributes for which such language was not used can be interpreted as having such an implicit meaning where the context does not indicate otherwise.

[0122] In one or more examples, the functions described in the present invention may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term processing unit has been used throughout this disclosure, such processing units may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, Petition 870250094200, dated 10 / 15 / 2025, pp. 337 / 382 62 / 69 technique described in the present invention, or other module, is implemented in software, the function, the processing unit, the technique described in the present invention, or other module, may be stored or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media may include computer data storage media or communication media, including any means that facilitates the transfer of a computer program from one place to another. In this way, computer-readable media may generally correspond to (1) tangible computer-readable storage media that are non-transient or (2) a communication medium, such as a signal or a carrier wave.Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. By way of example, and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices. As used in the present invention, disks (disk and disc) include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks generally reproduce data magnetically, while discs reproduce data optically by means of lasers. Combinations of the above items should also be included within the scope of computer-readable media.A computer program product may include a computer-readable medium.

[0123] The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), arithmetic logic units (ALUs), field-programmable logic arrays (FPGAs), or other equivalent set of integrated or distinct logic circuits. Consequently, the term processor, as used in the present invention, may refer to any of these. Petition 870250094200, dated 10 / 15 / 2025, pages 338 / 382 63 / 69 aforementioned structures or any other structure suitable for implementing the techniques described in the present invention. Furthermore, the techniques can be fully implemented in one or more circuits or logic elements.

[0124] The techniques of the present disclosure can be implemented in a wide variety of devices or appliances, including a wireless handset, an integrated circuit (IC), or a set of ICs, for example, a chipset. Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Instead, as described above, various units can be combined into any hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware. Consequently, the term processor, as used in the present invention, can refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described in the present invention.Furthermore, the techniques can be fully implemented in one or more circuits or logic elements.

[0125] The following aspects are merely illustrative and may be combined with other aspects or teachings described in the present invention, without limitation.

[0126] Aspect 1 is a display processing apparatus, which includes at least one memory and at least one processor coupled to at least one memory and, based at least in part on the information stored in at least one memory, the at least one processor, individually or in any combination, is configured to: obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene; configure a second intensity map based on the first intensity map and at least one frame of coordinates from a user's perspective of a device to display content associated with the scene, wherein the Petition 870250094200, dated 10 / 15 / 2025, pp. 339 / 38264 / 69 second intensity map is associated with luminance information for the scene; determine if the luminance information for at least one region in the second intensity map is within a suitable luminance range for the display content associated with the scene; process a set of pixels corresponding to a section in a display that is associated with at least one region based on the luminance information for at least one region being within the suitable luminance range, outside the suitable luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the suitable luminance range;and provide an indication of the set of pixels processed or the set of transformation parameters estimated.

[0127] Aspect 2 is the apparatus of aspect 1, in which to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: transform the entire set of pixels corresponding to the section on the display based on luminance information for at least one region being within the appropriate luminance range.

[0128] Aspect 3 is the apparatus of aspect 2, in which at least one processor, individually or in any combination, is additionally configured to: transmit to the display the transformed set of pixels corresponding to the section on the display.

[0129] Aspect 4 is the device of any of aspects 2 to 3, in which to transform the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: distort the set of pixels corresponding to the section on the display.

[0130] Aspect 5 is the apparatus of any of aspects 2 to 4, in which a quantity of the transformed set of pixels is associated with at least Petition 870250094200, dated 10 / 15 / 2025, pages 340 / 382 65 / 69 one of the following: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment.

[0131] Aspect 6 is the apparatus of any of aspects 1 to 5, in which to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: transform a subset of pixels into the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range.

[0132] Aspect 7 is the apparatus of aspect 6, in which at least one processor, individually or in any combination, is additionally configured to: transmit to the display the transformed subset of pixels corresponding to the section on the display.

[0133] Aspect 8 is the apparatus of any of aspects 6 to 7, in which at least one processor, individually or in any combination, is additionally configured to: estimate a color value from a remaining amount of the pixel set, wherein the remaining amount of the pixel set is equal to an amount of the pixel set that does not include the pixel subset, wherein the estimated color value of the remaining amount of the pixel set is interpolated based on the pixel subset.

[0134] Aspect 9 is the device of any of aspects 6 to 8, in which to transform the subset of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: distort the subset of pixels corresponding to the section on the display.

[0135] Aspect 10 is the apparatus of any of aspects 6 to 9, in which a quantity of the transformed subset of pixels is associated with at least one of: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment. Petition 870250094200, dated 10 / 15 / 2025, pp. 341 / 382 66 / 69

[0136] Aspect 11 is the apparatus of any of aspects 1 to 10, in which, to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: refrain from transforming a subset of pixels into the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the proper luminance range and being within the indistinguishable luminance range, wherein the indistinguishable luminance range is less than a minimum luminance threshold and greater than a maximum luminance threshold, and wherein the indistinguishable luminance range corresponds to a time for the user to distinguish the display content that is greater than a distinguishable time threshold.

[0137] Aspect 12 is the apparatus of any of aspects 1 to 11, in which at least one processor, individually or in any combination, is further configured to: identify the section on the display that is associated with at least one region based on luminance information for the at least one region being within the proper luminance range or outside the proper luminance range, and in which, to process the pixel set, at least one processor is configured to process the pixel set based on the identification of the section on the display.

[0138] Aspect 13 is the apparatus of any of aspects 1 to 12, in which to determine whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range, at least one processor, individually or in any combination, is configured to: transmit a first indication of the luminance information for at least one region; and receive a second indication of whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range for the display content.

[0139] Aspect 14 is the apparatus of aspect 13, in which to receive the second indication, at least one processor, individually or in any Petition 870250094200, dated 10 / 15 / 2025, pages 342 / 382 The 67 / 69 combination is configured to: receive the second indication from a graphics processing unit (GPU) or an application engine in a central processing unit (CPU).

[0140] Aspect 15 is the device of any of aspects 1 to 14, in which to configure the second intensity map, at least one processor, individually or in any combination, is configured to: transform the first intensity map to generate the second intensity map based on the first intensity map and at least one coordinate frame from the device user's perspective to the display content associated with the scene.

[0141] Aspect 16 is the apparatus of any of aspects 1 to 15, which further comprises at least one of an antenna or a transceiver coupled to at least one processor, wherein to obtain the first intensity map, the at least one processor, individually or in any combination, is configured to: obtain, by means of at least one of the antenna or transceiver, the first intensity map of a set of camera sensors for the camera associated with the scene.

[0142] Aspect 17 is any apparatus of aspects 1 to 16, in which the suitable luminance range is greater than a minimum luminance threshold and less than a maximum luminance threshold, and in which at least one of the minimum luminance threshold or the maximum luminance threshold is an application-defined threshold or a user-defined threshold.

[0143] Aspect 18 is the apparatus of any of aspects 1 to 17, in which at least one processor, individually or in any combination, is additionally configured to: filter luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range.

[0144] Aspect 19 is the apparatus of aspect 18, wherein, if the luminance information for at least one region is within the appropriate luminance range, the luminance information for at least one region Petition 870250094200, dated 10 / 15 / 2025, pages 343 / 382 68 / 69 are visually conducive to the display content associated with the scene.

[0145] Aspect 20 is the apparatus of aspect 18, wherein, if the luminance information for at least one region is within the appropriate luminance range, a grid-aligned section in the second intensity map is associated with at least one region.

[0146] Aspect 21 is the apparatus of any of aspects 1 to 20, in which at least one processor, individually or in any combination, is further configured to: adjust a location of the display content corresponding to at least one region in the second intensity map if at least one of: (1) a frequency of change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold.

[0147] Aspect 22 is the apparatus of aspect 21, wherein at least one processor, individually or in any combination, is further configured to: estimate a set of updated transformation parameters associated with luminance information for at least one region based on the adjusted location of the display content corresponding to at least one region; and output the estimated set of updated transformation parameters associated with the luminance information for at least one region.

[0148] Aspect 23 is the apparatus of any of aspects 1 to 22, wherein at least one processor, individually or in any combination, is further configured to: transform the display content corresponding to at least one region into the second intensity map based on the estimated set of transformation parameters.

[0149] Aspect 24 is the apparatus of aspect 23, in which at least one processor, individually or in any combination, is additionally configured to: transmit the transformed display content corresponding to at least one region in the second intensity map. Petition 870250094200, dated 10 / 15 / 2025, pp. 344 / 382 69 / 69

[0150] Aspect 25 is the apparatus of any of aspects 1 to 24, in which the transformation parameter set includes at least one of: a scaling parameter set or a translation parameter set.

[0151] Aspect 26 is the apparatus of any of aspects 1 to 25, in which, to give the indication of the processed pixel set or the estimated transformation parameter set, at least one processor, individually or in any combination, is configured to: transmit, to at least one of: a distortion mechanism or an application mechanism in a central processing unit (CPU), the indication of the processed pixel set or the estimated transformation parameter set.

[0152] Aspect 27 is the apparatus of any of aspects 1 to 26, in which, to issue the indication of the processed pixel set or the estimated transformation parameter set, at least one processor, individually or in any combination, is configured to: store, in a first memory or a cache, the indication of the processed pixel set or the estimated transformation parameter set.

[0153] Aspect 28 is a display processing method for implementing any of aspects 1 through 27.

[0154] Aspect 29 is a display processing device that includes means for implementing any of aspects 1 through 27.

[0155] Aspect 30 is a computer-readable medium (i.e., a non-transient computer-readable medium) that stores computer-executable code (i.e., code for display processing), the code when executed by at least one processor causes at least one processor to implement any of aspects 1 to 27. Petition 870250094200, dated 10 / 15 / 2025, pages 345 / 382

Claims

1 / 10 CLAIMS 1. Apparatus for display processing characterized by comprising: at least one memory; and at least one processor coupled to at least one memory and wherein, based at least in part on the information stored in the at least one memory, the at least one processor, individually or in any combination, is configured to: obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene; configure a second intensity map based on the first intensity map and at least one coordinate frame from a user's perspective of a device to display content associated with the scene, wherein the second intensity map is associated with luminance information for the scene;To determine whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene; to process a set of pixels corresponding to a section in a display that is associated with at least one region based on the luminance information for that region being at least within the suitable luminance range, outside the suitable luminance range, or within an indistinguishable luminance range; or to estimate a set of transformation parameters associated with luminance information for that region in the second intensity map based on whether the luminance information for that region is within the suitable luminance range; and to issue an indication of the processed pixel set or the estimated transformation parameter set. Petition 870250094200, October 15, 2025, pp. 373 / 382 2 / 10; 2. Apparatus, according to claim 1, characterized in that, to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: transform the entire set of pixels corresponding to the section on the display based on luminance information to at least one region being within the appropriate luminance range.

3. Device according to claim 2, characterized in that at least one processor, individually or in any combination, is further configured to: transmit to the display the transformed set of pixels corresponding to the section on the display.

4. Device according to claim 2, characterized in that, to transform the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: distort the set of pixels corresponding to the section on the display.

5. Device according to claim 2, characterized in that a quantity of the transformed set of pixels is associated with at least one of: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment.

6. Apparatus, according to claim 1, characterized in that, to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: transform a subset of pixels into the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range.

7. Device according to claim 6, characterized in that at least one processor, individually or in any combination, is further configured to: Petition 870250094200, dated 10 / 15 / 2025, pp. 374 / 382 3 / 10 transmit to the display the transformed subset of pixels corresponding to the section on the display.

8. Apparatus, according to claim 6, characterized in that at least one processor, individually or in any combination, is further configured to: estimate a color value of a remaining quantity of the pixel set, wherein the remaining quantity of the pixel set is equal to a quantity of the pixel set that does not include the subset of pixels, wherein the estimated color value of the remaining quantity of the pixel set is interpolated based on the subset of pixels.

9. Device according to claim 6, characterized in that, to transform the subset of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: distort the subset of pixels corresponding to the section on the display.

10. Device according to claim 6, characterized in that a quantity of the transformed subset of pixels is associated with at least one of: movement of the device user's head toward the display content, movement of the device user's body, or the device user's surrounding environment.

11. Device according to claim 1, characterized in that, to process the set of pixels corresponding to the section on the display, at least one processor, individually or in any combination, is configured to: refrain from transforming a subset of pixels into the set of pixels corresponding to the section on the display based on luminance information for at least one region being outside the appropriate luminance range and being within the indistinguishable luminance range, wherein the indistinguishable luminance range is less than a minimum luminance threshold and greater than a maximum luminance threshold, and wherein the indistinguishable luminance range corresponds to a time for the user to distinguish the display content that is greater than a distinguishable time threshold.

12. Apparatus, according to claim 1, characterized in that at least one processor, individually or in any combination, is further configured to: identify the section on the display that is associated with at least one region based on luminance information for the at least one region being within the appropriate luminance range or outside the appropriate luminance range, and in that, to process the set of pixels, the at least one processor is configured to process the set of pixels based on the identification of the section on the display.

13. Apparatus, according to claim 1, characterized in that, to determine whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range, at least one processor, individually or in any combination, is configured to: transmit a first indication of the luminance information for at least one region; and receive a second indication of whether the luminance information for at least one region in the second intensity map is within the appropriate luminance range for the display content.

14. Device according to claim 13, characterized in that, to receive the second indication, at least one processor, individually or in any combination, is configured to: receive the second indication from a graphics processing unit (GPU) or an application engine in a central processing unit (CPU).

15. Device according to claim 1, characterized in that, to configure the second intensity map, at least one processor, individually or in any combination, is configured to: transform the first intensity map to generate the second intensity map based on the first intensity map and at least one coordinate frame from the perspective of the device user to the display content associated with the scene.

16. Apparatus, according to claim 1, characterized by further comprising at least one antenna or transceiver coupled to at least one processor, wherein to obtain the first intensity map, the at least one processor, individually or in any combination, is configured to: obtain, via at least one of the antennas or transceivers, the first intensity map from a set of camera sensors to the camera associated with the scene.

17. Apparatus according to claim 1, characterized in that the suitable luminance range is greater than a minimum luminance threshold and less than a maximum luminance threshold, and wherein at least one of the minimum luminance threshold or the maximum luminance threshold is an application-defined threshold or a user-defined threshold.

18. Device according to claim 1, characterized in that at least one processor, individually or in any combination, is further configured to: filter the luminance information for at least one region in the second intensity map if the luminance information for at least one region is within the appropriate luminance range.

19. Apparatus according to claim 18, characterized in that, if the luminance information for at least one region is within the appropriate luminance range, the luminance information for at least one region is visually conducive to the display content associated with the scene.

20. Apparatus according to claim 18, characterized in that, if the luminance information for at least one region is within the appropriate luminance range, a grid-aligned section on the second intensity map is associated with at least one region. Petition 870250094200, dated 10 / 15 / 2025, pp. 377 / 382 6 / 10 21. Device according to claim 1, characterized in that at least one processor, individually or in any combination, is further configured to: adjust a location of the display content corresponding to at least one region in the second intensity map if at least one of: (1) a frequency change in a set of display coordinates for the display content corresponding to at least one region is less than a frequency threshold, or (2) a size of the display content corresponding to at least one region is greater than a size threshold.

22. Apparatus according to claim 21, characterized in that at least one processor, individually or in any combination, is further configured to: estimate a set of updated transformation parameters associated with luminance information for at least one region based on the adjusted location of the display content corresponding to at least one region; and output the estimated set of updated transformation parameters associated with luminance information for at least one region.

23. Device according to claim 1, characterized in that at least one processor, individually or in any combination, is further configured to: transform the display content corresponding to at least one region into the second intensity map based on the set of estimated transformation parameters.

24. Device, according to claim 23, characterized in that at least one processor, individually or in any combination, is additionally configured to: transmit the transformed display content corresponding to at least one region in the second intensity map. Petition 870250094200, dated 10 / 15 / 2025, pp. 378 / 382 7 / 10 25. Apparatus, according to claim 1, characterized in that the set of transformation parameters includes at least one of: a set of scaling parameters or a set of translation parameters.

26. Apparatus, according to claim 1, characterized in that, to emit the indication of the processed pixel set or the estimated transformation parameter set, at least one processor, individually or in any combination, is configured to: transmit, to at least one of: a distortion mechanism or an application mechanism in a central processing unit (CPU), the indication of the processed pixel set or the estimated transformation parameter set.

27. Device according to claim 1, characterized in that, to display the indication of the set of processed pixels or the set of estimated transformation parameters, at least one processor, individually or in any combination, is configured to: store, in a first memory or a cache, the indication of the set of processed pixels or the set of estimated transformation parameters.

28. Display processing method characterized by comprising: obtaining a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene; configuring a second intensity map based on the first intensity map and at least one frame of coordinates from a user's perspective of a device to display content associated with the scene, wherein the second intensity map is associated with luminance information for the scene; determining whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene; Petition 870250094200, dated 10 / 15 / 2025, p.379 / 382 8 / 10 process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region, with at least one being within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range; and output an indication of the processed pixel set or the estimated transformation parameter set.

29. Apparatus for display processing characterized by comprising: means for obtaining a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene; means for configuring a second intensity map based on the first intensity map and at least one frame of coordinates from a user's perspective of a device to display content associated with the scene, wherein the second intensity map is associated with luminance information for the scene; means for determining whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene;means to process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region, with at least one region being within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or means to estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range; and means to issue an indication of the processed pixel set or the estimated transformation parameter set.

30. A computer-readable medium characterized by storing computer-executable instructions for display processing, wherein the instructions, when executed by at least one processor, cause at least one processor to: obtain a first intensity map associated with luminance information for a scene that includes a plurality of frames, wherein the first intensity map corresponds to a camera associated with the scene; set up a second intensity map based on the first intensity map and at least one frame of coordinates from a user's perspective on a device to display content associated with the scene, wherein the second intensity map is associated with luminance information for the scene; determine whether the luminance information for at least one region in the second intensity map is within a luminance range suitable for the display content associated with the scene;To process a set of pixels corresponding to a section in a display that is associated with at least one region based on luminance information for at least one region, with at least one region being within the appropriate luminance range, outside the appropriate luminance range, or within an indistinguishable luminance range; or to estimate a set of transformation parameters associated with luminance information for at least one region in the second intensity map based on whether the luminance information for at least one region is within the appropriate luminance range; and to issue an indication of the processed pixel set or the estimated transformation parameter set. Petition 870250094200, dated 10 / 15 / 2025, pp. 381 / 382 10 / 10; 31. Product, process, system, kit, means or use, characterized by comprising one or more elements described in the descriptive report, claims, drawings, sequence listing, or summary of this application, when applicable. Petition 870250094200, dated 10 / 15 / 2025, pp. 382 / 382

Images