Display of images with different dynamic ranges

Abstract

The present disclosure relates to methods and apparatus for graphics processing, such as generating a smooth transition between images of different dynamic ranges (e.g., Standard Dynamic Range (SDR) images and High Dynamic Range (HDR) images). An exemplary method generally comprises the step of using a high frame rate during a transition period to allow the characteristics of the images to change incrementally. The characteristics may include, in particular, brightness (i.e., luminance), color gamut, and tone mapping. For example, during a transition period, a subset of HDR images is displayed at a second frame rate (e.g., 120 Hz) higher than the frame rate based on the HDR images (e.g., 30 Hz). At the same time, the brightness level (as well as other modes) of the display panel is incrementally adjusted from the SDR brightness level to the HDR brightness level during the transition time period across the subset of HDR images.

Description

Technology Field

[0001] The present disclosure generally relates to processing systems, and more specifically to one or more techniques for graphics processing. Background Technology

[0002] High Dynamic Range (HDR) video has a greater dynamic range than Standard Dynamic Range (SDR) video. SDR video describes images, renderings, or videos using conventional gamma curves that present a dynamic range considered to be the standard. HDR video encoding standards allow for higher maximum luminance (and / or lower minimum darkness or maximum blackness) than SDR video. For example, HDR video uses 10-bit color depth or more, whereas SDR video typically uses 8 bits for non-professional content and 10 bits for professional content. In addition to indicating the ratio between maximum and minimum luminance, the term "HDR video" may also refer to a wide color gamut. As such, HDR video or digital content often contains information regarding a wide dynamic range and a large color volume.

[0003] Modern display panels capable of providing higher luminance to properly present HDR video or other digital content may be referred to as HDR displays. HDR displays can generate greater maximum brightness than SDR displays. For example, the brightness or luminance of a display panel may be measured in the unit "nit." While HDR displays can reach the upper hundreds to thousands of nits, SDR displays often only have the lower hundreds of nits. When HDR displays can play both SDR and HDR videos, sudden transitions between displaying SDR and HDR video occur due to differences in dynamic ranges, causing disruptive abrupt changes in brightness and color gamut to attract the viewer's attention. It is desirable to avoid or minimize these sudden transitions. means of solving the problem

[0004] The following is a simplified overview of one or more modes to provide a basic understanding of such modes. This overview is not intended to be a comprehensive overview of all modes considered, nor to identify the core elements of all modes, nor to describe the scope of any or all modes. Its sole purpose is to present some concepts of one or more modes in a simplified form as an introduction to the more detailed descriptions to be provided later.

[0005] In an embodiment of the present disclosure, a method, a computer-readable medium, an apparatus, and a computing device are provided.

[0006] In certain embodiments, an exemplary device is operable to switch between displaying a first plurality of images containing content having a first dynamic range and displaying a second plurality of images containing at least some content having a second dynamic range different from the first dynamic range. The device generally includes a display; a memory; and a processor coupled to the memory. The memory and the processor are configured to display a first plurality of images on the display at a first frame rate during a first time period at a first brightness level of the display panel. The memory and the processor are further configured to display a first subset of a second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images containing at least some content having a second dynamic range different from the first dynamic range. The brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period over the subset of the second plurality of images. The memory and processor are additionally configured to display a second subset of a second plurality of images on the display at a third frame rate lower than the second frame rate during a third time period following a second time period at a second brightness level of the display panel.

[0007] In certain embodiments, an exemplary method for switching between displaying a first plurality of images having content having a first dynamic range on a display panel and displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range is provided. The method includes the step of displaying a first plurality of images on a display at a first frame rate during a first time period at a first brightness level of the display panel. The method includes the step of displaying a first subset of a second plurality of images on a display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images having at least some content having a second dynamic range different from the first dynamic range. The brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period over the subset of the second plurality of images. The method further includes the step of displaying a second subset of a second plurality of images on a display at a third frame rate lower than the second frame rate during a third time period following a second time period at a second brightness level of the display panel.

[0008] Specific embodiments of the present disclosure provide a computing device for switching between displaying a first plurality of images having content having a first dynamic range on a display panel and displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range. The computing device includes means for displaying the first plurality of images on the display at a first frame rate during a first time period at a first brightness level of the display panel. The computing device includes means for displaying a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images having at least some content having a second dynamic range different from the first dynamic range. The brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period over the subset of the second plurality of images. The computing device further includes means for displaying a second subset of a second plurality of images on a display at a third frame rate lower than the second frame rate during a third time period following a second time period at a second brightness level of the display panel.

[0009] Certain aspects of the present disclosure provide a non-transient computer-readable medium storing instructions, the instructions which, when executed by a computing device as described herein, cause the computing device to switch from displaying a first plurality of images having content having a first dynamic range on a display panel to displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range. For example, the non-transient computer-readable medium stores instructions which, when executed by a computing device, cause the computing device to display a first plurality of images on a display at a first frame rate during a first time period at a first brightness level of the display panel. The display panel refresh interval of the display panel corresponds to the time duration of the display cycle of the display panel. The display panel may be configured to refresh each display cycle. A non-transient computer-readable medium stores instructions that, when executed by a computing device, further cause the computing device to display a first subset of a second plurality of images on a display at a second frame rate higher than a first frame rate during a second time period following a first time period, based on the fact that the second plurality of images include at least some content having a second dynamic range different from a first dynamic range. The brightness level of the display panel is incrementally adjusted from a first brightness level to a second brightness level over the second subset of the second plurality of images during the second time period. A non-transient computer-readable medium stores instructions that, when executed by a computing device, further cause the computing device to display a second subset of the second plurality of images on a display at a second frame rate lower than a second frame rate during a third time period following a second time period at the second brightness level of the display panel.

[0010] Details of one or more examples of the present disclosure are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of the present disclosure will become apparent from the description and drawings, and from the claims. Brief explanation of the drawing

[0011] FIG. 1 is a block diagram illustrating an exemplary computational system according to specific embodiments of the present disclosure. FIG. 2 illustrates exemplary brightness profiles of different layers for transitions between Standard Dynamic Range (SDR) and High Dynamic Range (HDR) content according to specific embodiments of the present disclosure. FIG. 3 illustrates exemplary tone mapping curves for use in transitions from SDR content to HDR content according to specific embodiments of the present disclosure. FIG. 4 illustrates a flowchart of exemplary operations for displaying images of different dynamic ranges according to specific embodiments of the present disclosure. FIG. 5 illustrates a flowchart of exemplary operations for displaying images of different dynamic ranges according to specific embodiments of the present disclosure. Same numbers represent the same elements. Specific details for implementing the invention

[0012] The present disclosure provides techniques for smooth transitions between images of different dynamic ranges. For example, High Dynamic Range (HDR) images may require substantially brighter luminance output from a display than Standard Dynamic Range (SDR) images. When HDR images and SDR images are displayed sequentially, sudden changes in brightness and / or color gamut may cause users anxiety or sudden attention. To allow viewers to naturally and smoothly adapt to the high brightness and / or color saturation of HDR images, the present disclosure provides techniques for processing and generating transition content that reduces the impact on experience continuity. The disclosed techniques allow observers to perceive HDR images without being conscious of glitches, jamming, or the sensation of switching.

[0013] In exemplary embodiments, the present disclosure provides methods, devices, and systems for switching between displaying a first plurality of images having content having a first dynamic range on a display panel and displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range. The techniques include displaying a first number of images (e.g., SDR images) on a display panel at a first frame rate during a first time period at a first brightness level of the display panel. The techniques include displaying a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images having at least some content having a second dynamic range different from the first dynamic range (e.g., HDR images). In some cases, the techniques may incrementally adjust the brightness level of the display panel from a first brightness level to a second brightness level during a second time period over a subset of the second plurality of images. The techniques further include displaying a second subset of a second plurality of images on a display at a third frame rate lower than the second frame rate during a third time period following a second time period at a second brightness level of the display panel. Details of these embodiments of the techniques are further discussed below.

[0014] For example, when a display panel switches from displaying SDR images to displaying HDR images, the display panel may increase the frame rate during the transition time period. For example, the transition frame rate may be higher (e.g., much higher) than the intrinsic frame rates of the SDR images or HDR images. In one example, if the HDR images have a frame rate of 30 Hz, the transition frame rate may be 120 Hz. If the transition period is 1 second (or 1000 ms), there will be 120 frames between the SDR and HDR images for the transition. To achieve a smooth transition, the brightness, color gamut, and / or other visual characteristics of the transition frames may change incrementally (e.g., linearly, non-linearly, etc.). In some cases, the transition time period may be longer or shorter, or the frame rate may vary based on the SDR or HDR images.

[0015] Various aspects of systems, devices, computer program products, and methods are described more fully below regarding the transition between visual content of different dynamic ranges. However, the present disclosure may be embodied in many different forms and should not be interpreted as being limited to any specific structure or function presented throughout the present disclosure. Rather, these aspects are provided to make the present disclosure thorough and complete and to fully convey the scope of the present disclosure to those skilled in the art. Based on the teachings in this specification, those skilled in the art will understand that the scope of the present disclosure is intended to cover any aspect of the systems, devices, computer program products, and methods disclosed herein, whether embodied independently of or in combination with other aspects of the present disclosure. For example, a device may be embodied or a method may be practiced using any number of aspects presented herein. Furthermore, the scope of the present disclosure is intended to cover such a device or method practiced using other structures, functions, or structures and functions in addition to or in addition to the various aspects of the present disclosure presented herein. Any embodiment disclosed in this specification may be implemented by one or more elements of the claim.

[0016] Although various embodiments are described herein, many variations and permutations of these embodiments fall within the scope of this disclosure. While some potential benefits and advantages of the embodiments of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to specific benefits, uses, or purposes. Rather, the embodiments of this disclosure are intended to be broadly applicable to different graphic technologies, system configurations, etc., some of which are illustrated by example in the drawings and in the following description. The detailed description and drawings are merely illustrative and not limiting of this disclosure, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0017] Various embodiments are presented with reference to various devices and methods. These devices and methods are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as “elements”). These elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or as software depends on design constraints imposed on the system as a whole and on the specific application.

[0018] For example, an element, or any part of an element, or any combination of elements may be implemented as a “processing system” comprising one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-chip (SOCs), baseband processors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. One or more processors in the processing system may execute software. Software may be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. The term application may also refer to software. As described herein, one or more techniques may refer to an application, i.e., software, configured to perform one or more functions. In such examples, the application may be stored in memory, for example, on-chip memory of a processor, system memory, or any other memory. Hardware described herein, such as a processor, may also be configured to execute the application.For example, an application may be described as including code that causes the hardware to perform one or more techniques described herein when executed by the hardware. As an example, the hardware may perform one or more techniques described herein by accessing code from memory and executing the code accessed from memory. In some examples, components are identified in this disclosure. In such examples, the components may be hardware, software, or a combination thereof. The components may be separate components or subcomponents of a single component.

[0019] Accordingly, in one or more examples described herein, the described functions may be implemented in hardware, software, or any combination thereof. When implemented in software, the functions may be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media. Storage media may be any available media that can be accessed by a computer. By example, not limiting, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of the types of computer-readable media mentioned above, or any other media that can be used to store computer-executable code in the form of instructions or data structures that can be accessed by a computer.

[0020] Generally, the present disclosure describes techniques for a single device having a plurality of pipelines, or for a single device or a plurality of devices having a graphics processing pipeline, for improving the rendering of graphics content and / or for reducing the load on any processing unit configured to perform one or more of the techniques described herein, such as a processing unit, i.e., a GPU. For example, the present disclosure describes techniques for graphics processing in any device utilizing graphics processing. Other exemplary benefits are described throughout the present disclosure.

[0021] As used herein, instances of the term “content” may be interchangeable with the terms “image” or “images.” That is, “content” may refer to “graphic content” or “image,” and vice versa. This is true regardless of whether the terms are used as adjectives, nouns, or other parts of speech. In some examples, as used herein, the term “graphic content” may refer to content generated by one or more processes of a graphics processing pipeline. In some examples, as used herein, the term “graphic content” may refer to content generated by a processing unit configured to perform graphics processing. In some examples, as used herein, the term “graphic content” may refer to content generated by a graphics processing unit.

[0022] In some examples, as used herein, the term “display content” may refer to content generated by a processing unit configured to perform display processing. In some examples, as used herein, the term “display content” may refer to content generated by a display processing unit (DPU). Graphic content may be processed to become display content. For example, a graphics processing unit may output graphic content, such as a frame, to a buffer (which may be referred to as a frame buffer). A DPU may read graphic content, such as one or more frames, from a buffer and perform one or more display processing techniques on it to generate display content. For example, a DPU may be configured to perform composition on one or more rendered layers to generate a frame. As another example, a DPU may be configured to composite, blend, or otherwise combine two or more layers into a single frame. A DPU may be configured to perform scaling on a frame, such as upscaling or downscaling. In some examples, a frame may refer to a layer. In other examples, a frame may represent two or more layers that have already been blended together to form a frame, that is, the frame contains two or more layers, and a frame containing two or more layers may subsequently be blended.

[0023] FIG. 1 is a block diagram illustrating an exemplary computing system (100) configured to perform one or more of the techniques of the present disclosure. The computing system (100) includes a device (104). The device (104) may include one or more components or circuits for performing the various functions described herein. In some examples, one or more components of the device (104) may be components of an SOC. The device (104) may include one or more components configured to perform one or more of the techniques of the present disclosure. In the illustrated example, the device (104) may include a GPU (120), a CPU (122), and a system memory (124). In some embodiments, the device (104) may include a plurality of optional components, e.g., a communication interface (126), a transceiver (132), a receiver (128), a transmitter (130), a DPU (127), and one or more displays (131).

[0024] References to a display (131) may refer to one or more displays (131). For example, a display (131) may include a single display or a plurality of displays. A display (131) may include a first display and a second display. The first display may be a left-eye display and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation on the first and second displays. In other examples, the first and second displays may receive the same frames for presentation on the first and second displays. In additional examples, the results of the graphics processing may not be displayed on the device, for example, the first and second displays may not receive any frames for presentation on the first and second displays. Instead, the frames or the results of the graphics processing may be transmitted to another device. In some embodiments, this may be referred to as split-rendering.

[0025] The GPU (120) may include internal memory (121). The GPU (120) may be configured to perform graphics processing, as in the graphics processing pipeline (107). The CPU (122) may include internal memory (123). In some examples, the device (104) may include a display processor, such as a DPU (127), to perform one or more display processing techniques on one or more frames generated by the GPU (120) before presentation by one or more displays (131). The DPU (127) may be configured to perform display processing. For example, the DPU (127) may be configured to perform one or more display processing techniques on one or more frames generated by the GPU (120). One or more displays (131) may be configured to display or otherwise present the frames processed by the DPU (127). In some examples, one or more displays (131) may include one or more of an LCD (liquid crystal display), a plasma display, an OLED (organic light emitting diode) display, a projection display device, an augmented reality display device, a virtual reality display device, a head-mounted display, or any other type of display device.

[0026] Memory outside the GPU (120) and CPU (122), such as system memory (124), may be accessible to the GPU (120) and CPU (122). For example, the GPU (120) and CPU (122) may be configured to read from and / or write to external memory, such as system memory (124). The GPU (120) and CPU (122) may be coupled to the system memory (124) so ​​as to communicate via a bus. In some examples, the GPU (120) and CPU (122) may be coupled to each other so as to communicate via a bus or different connection.

[0027] The CPU (122) may be configured to receive graphic content (such as SDR and HDR images of different dynamic ranges or characteristics) from any source, such as system memory (124) and / or a communication interface (126). The system memory (124) may be configured to store the received encoded or decoded graphic content. The CPU (122) may be configured to receive the encoded or decoded graphic content in the form of encoded pixel data, for example, from the system memory (124) and / or the communication interface (126). The CPU (122) may be configured to encode or decode any graphic content.

[0028] The internal memory (121) or system memory (124) may include one or more volatile or non-volatile memories or storage devices. In some examples, the internal memory (121) or system memory (124) may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media or optical storage media, or any other type of memory.

[0029] Internal memory (121) or system memory (124) may be a non-transient storage medium in some examples. The term “non-transient” may indicate that the storage medium is not implemented in a carrier wave or a propagated signal. However, the term “non-transient” should not be interpreted to mean that the internal memory (121) or system memory (124) is not movable or that its contents are static. In one example, the system memory (124) may be removed from the device (104) or moved to another device. In another example, the system memory (124) may not be removable from the device (104).

[0030] The GPU (120) may be a general-purpose GPU (GPGPU) or any other processing unit configured to perform graphics processing. In some examples, the GPU (120) may be integrated into the motherboard of the device (104). In some examples, the GPU (120) may exist on a graphics card installed in a port within the motherboard of the device (104), or otherwise may be integrated into a peripheral device configured to interact with the device (104). The GPU (120) may include one or more processors such as one or more microprocessors, GPUs, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the techniques are partially implemented in software, the GPU (120) may store instructions for the software in a suitable non-transient computer-readable storage medium, such as internal memory (121), or may execute instructions in hardware using one or more processors to perform the techniques of the present disclosure. Any of the foregoing, including hardware, software, and a combination of hardware and software, etc., may be considered as one or more processors.

[0031] The CPU (122) may be any processing unit configured to transmit instructions to the GPU (120) and perform general computational processing (e.g., non-graphic processing). In some examples, the CPU (122) may be integrated into the motherboard of the device (104). The CPU (122) may include one or more processors such as one or more microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the techniques are partially implemented in software, the CPU (122) may store instructions for the software in a suitable non-transient computer-readable storage medium, such as internal memory (123), or may execute instructions in hardware using one or more processors to perform the techniques of the present disclosure. Any of the foregoing, including hardware, software, and combinations of hardware and software, may be considered as one or more processors.

[0032] In some embodiments, the computing system (100) may include an optional communication interface (126). The communication interface (126) may include a receiver (128) and a transmitter (130). The receiver (128) may be configured to perform any receiving function described herein with respect to the device (104). Additionally, the receiver (128) may be configured to receive information, for example, eye or head position information, rendering commands, or location information from another device. The transmitter (130) may be configured to perform any transmitting function described herein with respect to the device (104). For example, the transmitter (130) may be configured to transmit information to another device, which may include a request for content. The receiver (128) and the transmitter (130) may be coupled to a transceiver (132). In such examples, the transceiver (132) may be configured to perform any receiving and / or transmitting functions described herein with respect to the device (104).

[0033] As described in this specification, a device such as device (104) may refer to any device, apparatus, or system configured to perform one or more techniques described in this specification. For example, a device may be a server, base station, user equipment, client device, station, access point, computer, e.g., personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer, end product, device, telephone, smartphone, server, video game platform or console, handheld device, e.g., portable video game device or PDA (personal digital assistant), wearable computing device, e.g., smart watch, augmented reality device, or virtual reality device, non-wearable device, display or display device, television, television set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-car computer, any mobile device, any device configured to generate graphic content, or any device configured to perform one or more techniques described in this specification. Although the processes in this specification may be described as being performed by a specific hardware component (e.g., GPU), in additional embodiments, they may be performed using other hardware components (e.g., CPU), in accordance with the disclosed embodiments.

[0034] GPUs can process multiple types of data or data packets in the GPU pipeline. For example, in some embodiments, the GPU can process two types of data or data packets, such as context register packets and draw call data. A context register packet may be a set of global state information, such as information regarding global registers, shading programs, or constant data, which can control how the graphics context is processed. For example, context register packets may contain information regarding color formats. In some embodiments of context register packets, there may be a bit indicating which workload belongs to the context register. Additionally, there may be multiple functions or programs executed simultaneously and / or in parallel. For example, functions or programs may describe specific behaviors, such as color modes or color formats. Accordingly, context registers can define multiple states of the GPU.

[0035] Context states can be used to determine how individual processing units, such as a vertex fetcher (VFD), vertex shader (VS), shader processor, or geometry processor, function, and / or in which mode a processing unit functions. To do so, GPUs may use context registers and programming data. In some modalities, the GPU may generate workloads in the pipeline, such as vertex or pixel workloads, based on context register definitions of modes or states. Specific processing units, such as a VFD, may use these states to determine specific functions, such as how vertices are assembled. Because these modes or states can change, GPUs may need to change the corresponding contexts. Additionally, workloads corresponding to a mode or state may follow the changing mode or state.

[0036] Exemplary transition between images of different dynamic ranges

[0037] The present disclosure generally relates to processing systems, and more specifically to one or more techniques for graphics processing. For example, as discussed, by introducing transition frames at a frame rate different from the content to be displayed and incrementally changing certain characteristics from a first content in a first dynamic range to a second content in a second dynamic range, the present disclosure provides a smooth transition between images of different dynamic ranges. In some cases, certain characteristics include brightness, color gamut, and / or tone mapping. A large number of transition frames may also achieve a smooth transition effect.

[0038] FIG. 2 illustrates exemplary brightness profiles (200) of different layers for transitions between SDR and HDR content according to specific embodiments of the present disclosure. The brightness profiles (200) include multiple timelines (210 to 270) for each layer in three sections: a first SDR-only section (202) (when only SDR content is displayed), an intermediate SDR and HDR combined section (204) (when SDR and HDR content is displayed), and a second SDR-only section (206) (when only SDR content is displayed). Brightness profiles (200) include an ambient light timeline (210), an auto-brightness timeline (220), a dimming factor timeline (230) for SDR layers, an SDR 10-bit code range timeline (240), an SDR luminance (i.e., brightness) range timeline (250), an HDR 10-bit code range timeline (260), and an HDR luminance (i.e., brightness) range timeline (270).

[0039] Profiles (200) illustrate how a display panel may change its luminance or brightness in response to changes in incoming images (i.e., changes between SDR-only content and SDR and HDR content) and ambient light. For example, timelines 250 and 270 illustrate output luminance for SDR and HDR content, respectively (output luminance may overlap when both are displayed). Thus, when the display panel needs to switch from a first SDR-only section (202) to an SDR and HDR combined section (204), the output luminance needs to be adjusted to provide a smooth transition. For example, an auto-brightness timeline (220) illustrates an example of a luminance output that is dimmed down when ambient light is dimmed in the first SDR-only section (202), ramped up to a maximum level in the SDR and HDR combined section (204), and dimmed back to the previous brightness level in the second SDR-only section (206). The present disclosure provides techniques for creating a smooth transition between sections (202, 204, 206).

[0040] Referring first to timelines 210 and 220, during the SDR and HDR combined section (204), the display panel outputs its maximum brightness for HDR content as shown in the auto-brightness timeline (220). During the first SDR section (202) and the second SDR section (204), the auto-brightness timeline (220) follows the changes in the ambient light timeline (210) where auto-brightness is dimmed when ambient light is dimmed.

[0041] Referring to the dimming factor timeline (230), excessive SDR luminance may be compensated during HDR playback by at least taking into account the ambient light profile (210). For example, if the ambient light decreases, the SDR luminance may decrease accordingly. Compensation of SDR luminance is exemplified in reduced quantization levels for SDR in the SDR 10-bit code range timeline (240) (which shares a common profile with timeline 230 but has different quantization units on the vertical axis).

[0042] In cases where HDR content is not displayed, the SDR luminance range timeline (250) represents the luminance for the SDR layers based on the auto-brightness timeline (220) (a dotted line without luminance override for HDR). As illustrated, the output luminance is a fraction of the total panel output capacity, as indicated by the "panel maximum" mark on the vertical axis. Timeline 250 is used as a reference timeline for comparison with the HDR timelines (260 and 270).

[0043] Returning to the HDR 10-bit code range timeline (260) and the HDR luminance range timeline (270), as illustrated in the HDR luminance range timeline (270), the display panel outputs at maximum luminance when displaying 10-bit HDR content. In some embodiments, the HDR luminance range timeline (270) includes an incremental profile (216) that ramps up brightness from the SDR luminance illustrated in timeline 250 to maximum luminance. The incremental profile (216) may be linear or other curved profiles.

[0044] FIG. 3 illustrates exemplary tone mapping curves for use in transitions from SDR content to HDR content according to specific embodiments of the present disclosure. The horizontal axis represents the content (310), which may represent the input luminance factor on a logarithmic scale across light intensities. The vertical axis represents the output (305), which may represent the output luminance factor on a logarithmic scale across light intensities. The relationship between the two may be manipulated to improve the contrast, saturation, or other aspects of the images being displayed. Line 330 represents a neutral output (i.e., direct output or no additional processing). Tone mapping curves are generally applied to adapt or adjust image data of a native histogram (or intensity distribution across the spectrum) to a specific display panel to simulate the human visual experience. Because image content may have different native histograms and display panels may have different dynamic or luminance ranges, different tone mapping curves are required to avoid pixels being too dark or too bright.

[0045] In the context of the present disclosure, in addition to common functions, tone mapping curves are generated to smooth the transition between SDR and HDR images. There are different optional tone mapping curves (320) that can be implemented. For example, an "S" shaped curve may enhance contrast in the mid-range and progressively compress highlights and shadows. During the transition between SDR and HDR images, the HDR images may progressively change the output (305) over time using one or more of the exemplary tone mapping curves (320). For example, one of the tone mapping curves (320) may be applied to some SDR images to simulate the output luminance range of the HDR images to meet higher peak luminance, instead of using a tone mapping curve specific to the SDR images.

[0046] FIG. 4 is an exemplary flowchart illustrating exemplary operations (400) for graphics processing, such as switching between displaying a first plurality of images containing content having a first dynamic range (e.g., SDR) and displaying a second plurality of images containing at least some content having a second dynamic range (e.g., HDR) different from the first dynamic range. In certain embodiments, the operations (400) may be performed by a CPU or GPU, such as the CPU (122) or GPU (120) of FIG. 1. The operations (400) may be triggered by detecting incoming HDR content while playing SDR content.

[0047] Operations (400) may be initiated, in 405, by displaying a first plurality of images on a display at a first frame rate during a first time period at a first brightness level of the display panel. For example, the first plurality of images are SDR images.

[0048] In 410, based on the fact that the second plurality of images include at least some content having a second dynamic range different from the first dynamic range, a first subset of the second plurality of images is displayed on the display at a second frame rate higher than the first frame rate during a second time period following the first time period. For example, the second dynamic range corresponds to HDR, and the second plurality of images include HDR images. The brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period across the subset of the second plurality of images.

[0049] In 415, during a third time period following a second time period, a second subset of a second plurality of images is displayed on the display at a second brightness level of the display panel at a third frame rate lower than the second frame rate. For example, a second subset of HDR images is displayed at the third frame rate in the third time period. In some cases, the third frame rate is 30 Hz and the second frame rate is 120 Hz.

[0050] In certain embodiments, the second dynamic range corresponds to at least one of a different maximum brightness, a different maximum saturation, or a different maximum contrast ratio compared to the first dynamic range. For example, the second dynamic range is an HDR that has or requires a maximum brightness supported by the display, such as 800 nits. In contrast, SDR images may be displayed at a brightness lower than that of HDR images, such as 100 nits. Similarly, HDR images may have greater saturation or a greater contrast ratio than SDR images.

[0051] In certain embodiments, each of the subsets of the second plurality of images is displayed at different brightness levels of the display panel. For example, for HDR layers, gradient brightness levels are calculated between the existing brightness and the target brightness level, such as the average content brightness to be used for HDR playback. When the transition frame rate is 120 Hz and the transition duration is 1000 ms, for example, 120 gradient brightness levels are generated. The brightness levels range from the initial SDR brightness to the maximum output brightness of the display panel. For example, when the SDR images are displayed at 80 nits and the maximum brightness is 1200 nits, the 120 gradient brightness levels range from 80 nits to 1200 nits in intervals of 9.3 nits.

[0052] In certain embodiments, the number of brightness levels of a display panel used to display a subset of a second plurality of images during a second time period is based on the duration of the second time period. For example, the duration may be 500 ms, 1000 ms, etc. The number of brightness levels may correspond to the product of the frame rate during the second time period and its duration. If the second time period is 1000 ms and the frame rate is 120 Hz, there may be 120 brightness levels that vary from SDR brightness levels to maximum panel luminance for HDR images.

[0053] In certain embodiments, the brightness level of the display is adjusted linearly during a second time period. For example, as illustrated in FIG. 2, the profile (216) of the HDR luminance range timeline (270) exemplifies linear adjustment. In some cases, the adjustment may be implemented using non-linear profiles.

[0054] In certain embodiments, a first tone mapping curve is used to display one or more first images among a subset of a second plurality of images during a second time period, and a second tone mapping curve is used to display one or more second images among a subset of a second plurality of images during a second time period. For example, the first tone mapping curve is used to display SDR images, and the second tone mapping curve is used to transition from SDR images to HDR images. In some cases, multiple tone mapping curves may be used instead of the second tone mapping curve during the second time period. The number of multiple tone mapping curves may vary based on the duration length of the second time period. For example, when the second time period is 1000 ms or longer, assuming each tone mapping curve covers 60 different brightness levels, two or more tone mapping curves may be generated for each low and high brightness level. When the second time period is 500 ms, one tone mapping curve may be generated for both the low brightness level and the high brightness level.

[0055] In certain embodiments, the number of tone mapping curves used to display a subset of a second plurality of images during a second time period is based on the duration of the second time period. For example, during a second time period corresponding to a transition from SDR images to HDR images, two or more tone mapping curves may be used. For example, when the generated gradient brightness levels range from 80 nits to 1200 nits in the previous example, there may be two tone mapping curves, one for brightness levels 80 to 400 nits and the other for brightness levels 401 nits to 1200 nits.

[0056] In certain embodiments, a series of tone mapping curves is used to display a subset of a second plurality of images during a second time period, and the series of tone mapping curves has increasing curve peaks. For example, the curve peaks may correspond to a maximum output luminance that increases incrementally during the transition to displaying HDR images (e.g., ramping up on profile 216 to reach the maximum panel luminance for HDR shown in timeline 270 of FIG. 2). That is, for each frame generated for the transition from SDR to HDR, the tone mapping curve may be adjusted for the changing peak brightness. The more frequently this adjustment is made, the smoother the transition the user may experience.

[0057] In some cases, for HDR content, gradient brightness levels and tone mapping curves may be determined by lab tuning using transition time intervals, HDR content metadata, and display panel brightness curves, and / or based on other information. For SDR content, SDR layers may scale pixel values ​​using a gain factor or a lookup table (LUT), and either the gain factor or the lookup table may operate scaling in the linear light domain.

[0058] In certain embodiments, luminance fluctuations during SDR-HDR-SDR switching may employ various techniques, particularly including pulse width modulation (PWM) and direct current (DC) dimming. PWM dimming uses a blackout period and may not be suitable in low luminance situations (e.g., causing flickering). DC dimming may cause color changes in high luminance situations. The choice between the two techniques may depend on the desired luminance range. A combination of these two luminance control techniques may also be used together for wide luminance control.

[0059] In certain embodiments, the second plurality of images comprises at least a portion of content having a first dynamic range from the first plurality of images. For example, some of the SDR images are displayed, such as during a transition and / or during the display of HDR content, during a second and / or third period. These at least a portion of the content having a first dynamic range is displayed at a first brightness during a first time period based on a first brightness level of the display and pixel values ​​of pixels in the first plurality of images corresponding to at least a portion of the content having a first dynamic range (e.g., SDR). The pixel values ​​of pixels in the second plurality of images (e.g., HDR images) corresponding to at least a portion of the content having a first dynamic range are adjusted by comparison with the pixel values ​​of pixels in the first plurality of images so that at least a portion of the content having a first dynamic range is displayed at a first brightness during a second time period. In some cases, the pixel values ​​of pixels in the second plurality of images are adjusted using at least one of a gain factor or a lookup table (LUT). For example, a gain factor or LUT enables the display panel to ensure that SDR images (to be displayed in a second time period and / or a third time period, such as during the SDR and HDR combined section (204) of FIG. 2) can be DC dimmed (or otherwise adjusted) to maintain SDR brightness to achieve a constant visual effect for the SDR images during the second and / or third time periods, even when the display panel is operating at high brightness to display HDR content. For example, during the first time period, the panel brightness may be set to a lower value X, and the RGB pixel values ​​may be at a normal level for the SDR images.During the second and / or third time period, the panel brightness may be set to a value higher than X, and accordingly, the RGB pixel values ​​for the SDR images may be reduced, so that during the first time period and the second and / or third time period, the SDR images have the same approximate brightness based on the combination of the RGB pixel values ​​and the total display panel brightness.

[0060] FIG. 5 illustrates a flowchart illustrating exemplary operations (500) for graphics processing, such as switching between displaying a first plurality of images containing content having a first dynamic range (e.g., SDR) and displaying a second plurality of images containing at least some content having a second dynamic range (e.g., HDR) different from the first dynamic range. In certain embodiments, the operations (500) may be performed by a CPU or GPU, such as the CPU (122) or GPU (120) of FIG. 1. The operations (500) may be triggered by detecting incoming HDR content while playing SDR content.

[0061] Operations (500) may be initiated, in 505, by displaying a first plurality of images on a display at a first frame rate during a first time period for applying a first tone mapping curve. For example, the first plurality of images are SDR images. The first tone mapping curve may have curve peaks for the SDR images.

[0062] In 510, based on the fact that the second plurality of images include at least some content having a second dynamic range different from the first dynamic range, a first subset of the second plurality of images is displayed on the display at a second frame rate higher than the first frame rate during a second time period following the first time period. For example, the second dynamic range corresponds to HDR, and the second plurality of images include HDR images. The tone mapping curve of the display panel is incrementally adjusted from the first tone mapping curve (i.e., for SDR images) to the second tone mapping curve (i.e., for HDR images) over the second time period across the subset of the second plurality of images.

[0063] In 515, during a third time period following the second time period, a second subset of a second plurality of images is displayed on the display at a third frame rate lower than the second frame rate at a second brightness level of the display panel. For example, a second subset of HDR images is displayed at the third frame rate in the third time period. The second plurality of images use a second tone mapping curve to display the HDR images.

[0064] The processes described above for converting from SDR content to HDR content are equally applicable to converting from HDR content to SDR content. For example, after converting from HDR to SDR, the SDR content may be displayed with the original SDR frame rate, brightness, and other SDR parameters.

[0065] In one configuration, a method or device for graphics processing is provided. The device may be a CPU, a GPU, or some other processor capable of performing graphics processing. In one embodiment, the device may be a GPU (120) within the device (104), or some other hardware within the device (104) or another device.

[0066] The subject matter described herein may be implemented to realize one or more benefits or advantages. For example, the described graphics processing techniques may be used by a GPU, a CPU, or some other processor capable of performing graphics processing to implement the state information techniques described herein. This may also be achieved at a lower cost compared to other graphics processing techniques. Furthermore, the graphics processing techniques described herein may improve or accelerate data processing or execution. Additionally, the graphics processing techniques described herein may improve resource or data utilization and / or resource efficiency.

[0067] According to the present disclosure, the term “or” may be interpreted as “and / or” unless the context otherwise indicates. Additionally, phrases such as “one or more” or “at least one” may be used for some features disclosed herein, but features for which such language is not used may be interpreted as implying such meaning unless the context otherwise indicates.

[0068] In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term “processing unit” has been used throughout this disclosure, such processing units may be implemented in hardware, software, firmware, or any combination thereof. Where any function, processing unit, technique described herein, or other module is implemented in software, the function, processing unit, technique described herein, or other module may be stored or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media may include communication media or computer data storage media comprising any medium that facilitates the transmission of a computer program from one place to another. In this way, computer-readable media may generally correspond to (1) non-transient, tangible computer-readable storage media or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for the implementation of the techniques described in this disclosure. By example, not limiting, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices. As used herein, disk and disc include CD (compact disc), laser disc, optical disc, DVD (digital versatile disc), floppy disc, and Blu-ray disc, wherein disks usually reproduce data magnetically, whereas discs reproduce data optically with lasers.The above combinations must also be included within the scope of computer-readable media. A computer program product may also include computer-readable media.

[0069] The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), arithmetic logic units (ALUs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term “processor” as used herein may refer to any of any other structures suitable for implementing the structures described above or the techniques described herein. Additionally, the techniques may be fully implemented in one or more circuits or logic elements.

[0070] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chip sets). While various components, modules, or units are described in the present disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, implementation by different hardware units is not necessarily required. Rather, as described above, various units may be coupled to any hardware unit, or may be provided by a set of interactive hardware units including one or more processors as described above, together with suitable software and / or firmware.

[0071] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

Claim 1 A device operable to switch between displaying a first plurality of images containing content having a first dynamic range and displaying a second plurality of images containing at least some content having a second dynamic range different from the first dynamic range, comprising: a display; a memory; and a processor coupled to the memory, wherein the memory and the processor: display the first plurality of images on the display at a first frame rate during a first time period at a first brightness level of the display panel; and display a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images containing at least some content having a second dynamic range different from the first dynamic range, wherein the brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period across the subset of the second plurality of images; An operable device configured to display a second subset of the second plurality of images on the display at a third frame rate lower than the second frame rate during a third time period following the second time period at the second brightness level of the display panel. Claim 2 A operable device according to claim 1, wherein the second dynamic range corresponds to at least one of a different maximum brightness, a different maximum saturation, or a different maximum contrast ratio compared to the first dynamic range. Claim 3 In claim 1, an operable device wherein each of the subsets of the second plurality of images is displayed at different brightness levels of the display panel. Claim 4 In claim 1, the number of brightness levels of the display panel used to display a subset of the second plurality of images during the second time period is based on the duration of the second time period, in an operable device. Claim 5 In claim 1, an operable device wherein the brightness level of the display is linearly adjusted during the second time period. Claim 6 An operable device according to claim 1, wherein the first tone mapping curve is used to display one or more first images among a subset of the second plurality of images during the second time period, and the second tone mapping curve is used to display one or more second images among a subset of the second plurality of images during the second time period. Claim 7 In claim 1, the number of tone mapping curves used to display a subset of the second plurality of images during the second time period is based on the duration of the second time period, in an operable device. Claim 8 In claim 1, a series of tone mapping curves is used to display a subset of the second plurality of images during the second time period, and the series of tone mapping curves has increasing curve peaks, an operable device. Claim 9 An operable device according to claim 1, wherein the second plurality of images comprises at least a portion of the content having the first dynamic range from the first plurality of images, the at least portion of the content having the first dynamic range is displayed at a first brightness during the first time period based on the first brightness level of the display and pixel values ​​of pixels within the first plurality of images corresponding to the at least portion of the content having the first dynamic range, and the pixel values ​​of pixels within the second plurality of images corresponding to the at least portion of the content having the first dynamic range are adjusted by comparison with pixel values ​​of pixels within the first plurality of images so that the at least portion of the content having the first dynamic range is displayed at the first brightness during the second time period. Claim 10 In claim 9, the pixel values ​​of the pixels within the second plurality of images are: an operable device adjusted using at least one of a gain factor or a lookup table. Claim 11 A method for switching between displaying a first plurality of images having content having a first dynamic range on a display panel and displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range, the method comprising: displaying the first plurality of images on the display at a first frame rate during a first time period at a first brightness level of the display panel; displaying a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images having at least some content having a second dynamic range different from the first dynamic range, wherein the brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period across the subset of the second plurality of images; and displaying a second of the second plurality of images on the display during a third time period following the second time period at the second brightness level of the display panel. A method comprising the step of displaying a subset at a third frame rate lower than the second frame rate. Claim 12 In claim 11, the method wherein the second dynamic range corresponds to at least one of a different maximum brightness, a different maximum saturation, or a different maximum contrast ratio compared to the first dynamic range. Claim 13 In claim 11, a method wherein each of the subsets of the second plurality of images is displayed at a different brightness level of the display panel. Claim 14 In claim 11, the number of brightness levels of the display panel used to display a subset of the second plurality of images during the second time period is based on the duration of the second time period. Claim 15 In claim 11, the method wherein the brightness level of the display is linearly adjusted during the second time period. Claim 16 A method according to claim 11, wherein a first tone mapping curve is used to display one or more first images among a subset of the second plurality of images during the second time period, and a second tone mapping curve is used to display one or more second images among a subset of the second plurality of images during the second time period. Claim 17 In claim 11, the number of tone mapping curves used to display a subset of the second plurality of images during the second time period is based on the duration of the second time period. Claim 18 In claim 11, a series of tone mapping curves are used to display a subset of the second plurality of images during the second time period, and the series of tone mapping curves have increasing curve peaks. Claim 19 A method according to claim 11, wherein the second plurality of images comprises at least a portion of the content having the first dynamic range from the first plurality of images, and the at least portion of the content having the first dynamic range is displayed at the first brightness during the first time period based on the first brightness level of the display and the pixel values ​​of pixels within the first plurality of images corresponding to the at least portion of the content having the first dynamic range, and the pixel values ​​of pixels within the second plurality of images corresponding to the at least portion of the content having the first dynamic range are adjusted by comparison with the pixel values ​​of pixels within the first plurality of images so that the at least portion of the content having the first dynamic range is displayed at the first brightness during the second time period. Claim 20 In claim 19, the pixel values ​​of the pixels within the second plurality of images are adjusted using at least one of a gain factor or a lookup table. Claim 21 An apparatus for switching between displaying a first plurality of images having content having a first dynamic range on a display panel and displaying a second plurality of images having at least some content having a second dynamic range different from the first dynamic range, the apparatus comprising: means for displaying the first plurality of images on the display at a first frame rate during a first time period at a first brightness level of the display panel; means for displaying a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images having at least some content having a second dynamic range different from the first dynamic range, wherein the brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period across the subset of the second plurality of images; and means for displaying the second plurality of images on the display during a third time period following the second time period at the second brightness level of the display panel. A device comprising means for displaying a second subset at a third frame rate lower than the second frame rate. Claim 22 In claim 21, the device wherein the second dynamic range corresponds to at least one of a different maximum brightness, a different maximum saturation, or a different maximum contrast ratio compared to the first dynamic range. Claim 23 In claim 21, a device wherein each of the subsets of the second plurality of images is displayed at different brightness levels of the display panel. Claim 24 In claim 21, the number of brightness levels of the display panel used to display a subset of the second plurality of images during the second time period is based on the duration of the second time period. Claim 25 In claim 21, the device wherein the brightness level of the display is linearly adjusted during the second time period. Claim 26 A non-transient computer-readable storage medium comprising instructions, wherein, when executed by a computing device, the computing device causes: to display a first plurality of images on a display at a first frame rate during a first time period at a first brightness level of a display panel; to display a first subset of the second plurality of images on the display at a second frame rate higher than the first frame rate during a second time period following the first time period, based on the second plurality of images comprising at least some content having a second dynamic range different from a first dynamic range, wherein the brightness level of the display panel is incrementally adjusted from the first brightness level to the second brightness level during the second time period across the subset of the second plurality of images; and to display a second subset of the second plurality of images on the display at a third frame rate lower than the second frame rate during a third time period following the second time period at the second brightness level of the display panel. Non-transient computer-readable storage media. Claim 27 A non-transient computer-readable storage medium according to claim 26, further comprising instructions that, when executed by the computing device, cause the computing device to use a first tone mapping curve to display one or more first images among a subset of the second plurality of images during the second time period, and to use a second tone mapping curve to display one or more second images among a subset of the second plurality of images during the second time period. Claim 28 A non-transient computer-readable storage medium according to claim 26, further comprising instructions that, when executed by the computing device, cause the computing device to use a number of tone mapping curves to display a subset of the second plurality of images during the second time period based on the duration of the second time period. Claim 29 In claim 26, a non-transient computer-readable storage medium further comprising instructions that, when executed by the computing device, cause the computing device to use a series of tone mapping curves to display a subset of the second plurality of images during the second time period, wherein the series of tone mapping curves have increasing curve peaks. Claim 30 A non-transient computer-readable storage medium according to claim 26, wherein the second plurality of images comprises at least a portion of the content having the first dynamic range from the first plurality of images, the at least portion of the content having the first dynamic range is displayed at the first brightness during the first time period based on the first brightness level of the display and pixel values ​​of pixels within the first plurality of images corresponding to the at least portion of the content having the first dynamic range, and the pixel values ​​of pixels within the second plurality of images corresponding to the at least portion of the content having the first dynamic range are adjusted by comparison with pixel values ​​of pixels within the first plurality of images so that the at least portion of the content having the first dynamic range is displayed at the first brightness during the second time period.

Images