Display processing unit extension for very large aspect ratio automotive displays

Abstract

A display processing unit capability and a display method for very large aspect ratio automotive displays. The method includes assigning a plurality of display processing unit hardware pipelines to a plurality of regions for region refresh in a first sequential manner, assigning a plurality of display processing unit hardware mixers to the plurality of regions for region refresh in a second sequential manner, and performing image signal processing on regions of a plurality of content layers to produce a plurality of processed layers for region refresh.

Description

Technical Field

[0001] This disclosure generally relates to the field of computer processor architecture, and more specifically to the display processing unit (DPU) capabilities for automotive displays with very large aspect ratios. Background Technology

[0002] Information processing systems (e.g., computing platforms) strive to achieve high processing throughput and large main memory capacity. An information processing system may include multiple processing engines, each optimized for a specific task. One processing engine, known as a Display Processing Unit (DPU), is typically used to control displays for various user applications. An evolving user application employs automotive displays with very large aspect ratios, for example, screens with a display size of 8,000 pixels wide by 1,000 pixels high. In many scenarios, it is necessary to expand the DPU's capacity to accommodate automotive displays with very large aspect ratios to maintain the desired Quality of Service (QoS). Summary of the Invention

[0003] To provide a basic understanding of one or more aspects of this disclosure, a summary of those aspects is given below. This summary is not an extensive overview of all intended features of this disclosure, nor is it intended to identify key or essential elements of all aspects of this disclosure, nor to depict the scope of any or all aspects of this disclosure. Its sole purpose is to present some concepts of one or more aspects of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0004] In one aspect, this disclosure provides a display processing unit (DPU) capability for automotive displays with very large aspect ratios. Accordingly, an apparatus includes: a plurality of display processing unit (DPU) hardware pipelines configured to perform one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers; and a plurality of display processing unit (DPU) hardware mixers coupled to the plurality of DPU hardware pipelines, the plurality of DPU hardware mixers being configured to generate one or more final image frames by performing one of the following: blending, mixing, or compositing the one or more content layers.

[0005] In one example, the device also includes a display screen coupled to the plurality of DPU hardware mixers, the display screen being configured to display a main screen and a plurality of auxiliary screens. In another example, the device also includes a cross switch coupled to the plurality of DPU hardware mixers, the cross switch being configured to interconnect the plurality of DPU hardware pipelines with the plurality of DPU hardware mixers.

[0006] In one example, the multiple DPU hardware pipelines are also configured to perform image signal processing on regions of one or more content layers to produce multiple processed layers for region refresh. In one example, the image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

[0007] In one example, the multiple DPU hardware mixers are also configured to synthesize subsets of multiple regions. In one example, the multiple DPU hardware mixers are assigned to the multiple regions. In one example, the multiple DPU hardware mixers are also configured to use multiple processed layers to synthesize multiple composite image frames for region refresh.

[0008] In one example, the multiple DPU hardware mixer is also configured to use time-division multiplexing (TDM) to synthesize a final image frame from the multiple synthesized image frames for use in a region refresh. In another example, the multiple DPU hardware mixer is also configured to use a region refresh to transmit the final image frame to one of a plurality of regions in the display screen.

[0009] Another aspect of this disclosure provides a method comprising: allocating a plurality of display processing unit (DPU) hardware pipelines to a plurality of regions for region refresh in a first order; allocating a plurality of display processing unit (DPU) hardware mixers to the plurality of regions for region refresh in a second order; and performing image signal processing on regions of a plurality of content layers to generate a plurality of processed layers for region refresh.

[0010] In one example, each of the plurality of Display Processing Unit (DPU) hardware pipelines is assigned to process one of the plurality of content layers in each of the plurality of regions. In one example, each of the plurality of Display Processing Unit (DPU) hardware pipelines is assigned to process one content layer within a subset of the plurality of regions. In one example, each of the plurality of Display Processing Unit (DPU) hardware mixers is assigned to synthesize a subset of the plurality of regions. In one example, image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

[0011] In one example, the method further includes using the multiple processed layers to synthesize multiple composite image frames for area refresh. In one example, the method further includes using time division multiplexing (TDM) to synthesize a final image frame from the multiple composite image frames for area refresh. In one example, the composite final image frame is operated on sequentially, one region at a time, using the multiple regions. In one example, the method further includes using area refresh to transmit the final image frame to one of the multiple regions in the display screen.

[0012] Another aspect of this disclosure provides an apparatus comprising: allocating a plurality of display processing unit (DPU) hardware pipelines to a plurality of regions for region refresh in a first sequential manner; allocating a plurality of display processing unit (DPU) hardware mixers to the plurality of regions for region refresh in a second sequential manner; performing image signal processing on regions of a plurality of content layers to generate a plurality of processed layers for region refresh; transmitting a final image frame to one of the plurality of regions in a display screen using region refresh; wherein each of the plurality of display processing unit (DPU) hardware pipelines is allocated to process one of the plurality of content layers in each of the plurality of regions; wherein each of the plurality of display processing unit (DPU) hardware mixers is allocated to synthesize a subset of the regions of the plurality of regions; and wherein the image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

[0013] These and other aspects of this disclosure will be more fully understood upon reading the following detailed description. Other aspects, features, and embodiments of this disclosure will become apparent to those skilled in the art after reading the following description of specific exemplary embodiments of the invention in conjunction with the accompanying drawings. Although features of the invention may be discussed below with respect to certain embodiments and drawings, all embodiments of the invention may include one or more of the advantageous features discussed herein. In other words, while one or more embodiments may be discussed as having certain advantageous features, one or more such features may also be used according to the various embodiments of the invention discussed herein. Similarly, although exemplary embodiments may be discussed below as embodiments of an apparatus, system, or method, it should be understood that such exemplary embodiments may be implemented in various apparatuses, systems, and methods. Attached Figure Description

[0014] Figure 1 An example information processing system is shown.

[0015] Figure 2 An example car display with a very large aspect ratio is shown.

[0016] Figure 3 An example automotive system-on-a-chip (SOC) is shown.

[0017] Figure 4 An example mobile system-on-a-chip (SOC) is shown.

[0018] Figure 5 An example display processing unit (DPU) architecture for automotive displays with very large aspect ratios is illustrated.

[0019] Figure 6 An example time-division multiplexing (TDM) timeline is shown.

[0020] Figure 7 An example flowchart for time-division multiplexing (TDM) area refresh of a display screen is shown. Detailed Implementation

[0021] The detailed description below, taken in conjunction with the accompanying drawings, is intended as a description of various configurations and not as representing only the configurations in which the concepts described herein can be practiced. To provide a thorough understanding of the various concepts, the detailed description includes specific details. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring these concepts.

[0022] While for the purpose of simplification, a method is shown and described as a series of actions, it should be understood and recognized that the method is not restricted by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions, depending on one or more aspects. For example, those skilled in the art will understand and recognize that a method may alternatively be represented as a series of related states or events, such as in a state diagram. Furthermore, not all exemplified actions are required to implement a method according to one or more aspects.

[0023] Figure 1An example information processing system 100 is illustrated. In one example, the information processing system 100 includes multiple processing engines, or processor cores, such as a central processing unit (CPU) 120, a digital signal processor (DSP) 130, a graphics processing unit (GPU) 140, a display processing unit (DPU) 180, etc. In one example, various other functions may be included in the information processing system 100, such as a support system 110, a modem 150, memory 160, cache memory 170, and a video display 190. For example, the multiple processing engines and various other functions may be interconnected via an interconnect data bus 105 to transfer data and control information. For example, memory 160 and / or cache memory 170 may be shared among the CPU 120, GPU 140, and other processing engines. In one example, the CPU 120 may include a first internal memory not shared with other processing engines. In one example, the GPU 140 may include a second internal memory not shared with other processing engines. In one example, any of the multiple processing engines may have internal memory (i.e., dedicated memory) not shared with other processing engines.

[0024] In one example, a user application with a very large aspect ratio display is an automotive display, which has a horizontal size comparable to the car's dashboard. For example, automotive displays differ from other displays (such as mobile platform displays, video displays, etc.) in several unique characteristics: • Compared to the vertical layout of mobile handheld devices, car displays are horizontally laid out; • Compared to the aspect ratio of television displays, car displays have a very wide aspect ratio; • Car displays have very high display resolutions, such as 8k pixel × 1k pixel screens, 8k pixel × 1.5k pixel screens, and 16k pixel × 1k pixel screens; • The car display has a main screen and multiple auxiliary screens (for example, up to 12 screens in total).

[0025] In one example, various System-on-Chip (SoCs) can be used for automotive display applications. In one example, a mobile SoC is a first example SoC optimized for use on a mobile platform. In another example, an automotive SoC is a second example SoC optimized for use on an automotive platform. In one example, a mobile SoC might be preferred for an automotive platform due to its hardware architecture characteristics and extensive software functionality.

[0026] In one example, the Display Processing Unit (DPU) in a System-on-Chip (SOC) (e.g., a mobile SOC or an automotive SOC) is used to manage the automotive display. In one example, automotive display management includes driving the automotive display with graphics data and processing / compositing content pixels of multiple layers (e.g., user interface (UI), video, camera, etc.). In one example, a layer is a component of an image frame.

[0027] In one example, the multiple layers are processed by a hardware pipeline in the DPU, and the automotive display is driven by a hardware mixer in the DPU via physical display interface ports (e.g., Display Serial Interface (DSI), Display Pixels (DP), etc.). In one example, the hardware pipeline consists of processing elements in the DPU that fetch content layers (e.g., RGB and YUV surfaces) from memory, such as user interfaces (UI), games, videos, images, etc. In one example, the hardware pipeline performs color format conversion, scaling, and quality improvement on source surfaces (e.g., layer pixels). In one example, the hardware mixer consists of processing elements in the DPU that blend, mix, or synthesize source surface / content layers to generate a final image frame. For example, the hardware mixer could receive N layers, then blend, mix, or synthesize N layers to produce a composite image frame, process the composite image frame to produce a final image frame, and then transmit the final image frame to the display.

[0028] Figure 2 An example car display 200 with a very large aspect ratio is illustrated. In one example, the example car display may have a main display and multiple secondary displays. In one example, the car display may have a pixel width of 8,000 pixels (8k pixels). In another example, the car display may have a pixel width of 16,000 pixels (16k pixels).

[0029] Figure 3An example automotive system-on-a-chip (SOC) 300 is illustrated. In one example, the example automotive SOC 300 includes a first DPU 310 and a second DPU 320. In one example, the first DPU 310 includes a first plurality of DPU hardware pipelines (e.g., eight DPU hardware pipelines) 311 and a first plurality of DPU hardware mixers (e.g., six DPU hardware mixers) 312. In one example, the second DPU 320 includes a second plurality of DPU hardware pipelines (e.g., eight DPU hardware pipelines) 321 and a second plurality of DPU hardware mixers (e.g., six DPU hardware mixers) 322. In one example, the first DPU 310 includes a first cross switch coupled to the first plurality of DPU hardware mixers 312, and the first cross switch is configured to interconnect the first plurality of DPU hardware pipelines 311 with the first plurality of DPU hardware mixers 312. In one example, the second DPU 320 includes a second cross switch coupled to a second plurality of DPU hardware mixers 322, and the second cross switch is configured to interconnect the second plurality of DPU hardware pipelines 321 with the second plurality of DPU hardware mixers 322.

[0030] Figure 4 An example mobile system-on-chip (SOC) 400 is illustrated. In one example, the example mobile SOC 400 includes a DPU 410. In one example, the DPU 410 includes multiple hardware pipelines (e.g., 10 hardware pipelines) 411 and multiple hardware mixers (e.g., 6 hardware mixers) 412. In one example, the DPU 410 includes a cross switch coupled to the multiple DPU hardware mixers 412, and the cross switch is configured to interconnect the multiple DPU hardware pipelines 411 with the multiple DPU hardware mixers 412.

[0031] In one example, the DPU ingests multiple content layers, processes them, composites image frames, and transmits the image frames to the display. In one example, the DPU's processing and transmission capabilities may be limited by several design factors. In one example, the first design factor concerns content layer width. For example, if a single content layer is wider than 2k pixels or 4k pixels, the DPU requires dual hardware pipelines (e.g., a total of 8 to 10 hardware pipelines) for processing. In one example, a typical automotive user application may involve up to 30 content layers. In one example, the second design factor concerns display resolution width. For example, if the display resolution width is wider than 4k pixels or 2k pixels, the DPU requires dual hardware mixers for processing. In one example, the third design factor concerns physical display link bandwidth. For example, if the display link bandwidth is greater than 8k60p (approximately 48Gbps in uncompressed format), the DPU requires dual DP / DSI interface ports to send data to the display.

[0032] In one example, an automotive display might require a very large physical width and a very large content layer width. For instance, an 8k pixel wide automotive display would require 2 to 4 hardware pipelines to handle an 8k pixel content layer width. Similarly, a 16k pixel wide automotive display would require 4 to 8 hardware pipelines to handle a 16k pixel content layer width.

[0033] In one example, an 8k pixel wide automotive display requires four hardware mixers to synthesize the final target image frames and drive the data stream to the main display. In another example, a mobile SoC could have only six hardware mixers, with only two limited-function mixers remaining for other auxiliary displays. In one example, an 8k pixel wide automotive display requires two DP interface ports to send data to the display. In another example, a 16k pixel wide automotive display requires four DP interface ports to send data to the display.

[0034] In one example, neither the mobile SoC nor the automotive SoC has a sufficient number of hardware pipelines and hardware mixers to support an automotive display with a single 8k pixel (or larger) wide main display and 5 to 12 auxiliary displays. In one example, the mobile SoC may have 10 hardware pipelines, and a main display with 5 layers exhausts all available hardware pipelines. In one example, if there are more than 5 layers in the main display, layer processing will need to be handled by another processing engine (e.g., the GPU). In one example, layer processing not being performed by the DPU results in a regression in visual appearance and performance. For example, offloading display processing to another processing engine may degrade desired Quality of Service (QoS), such as latency, jitter, etc.

[0035] In one example, an automotive SoC might have eight hardware pipelines for a single display and could only support four layers for a main display with an 8k pixel width. In another example, if the main display has more than four layers, layer processing might need to be handled by another processing engine (e.g., a GPU). In yet another example, layer processing not being performed by the DPU results in a regression in visual appearance and QoS performance.

[0036] In one example, if the primary display requires almost all of the hardware pipeline for layer processing, the secondary display cannot be processed by the DPU but by another processing engine (e.g., the GPU), which may result in a regression in visual appearance and performance.

[0037] In one example, an 8-kilopixel wide display requires four hardware mixers, and a 16-kilopixel wide display requires eight hardware mixers, which exceeds the available hardware mixer resources in a mobile SoC. In another example, the number of hardware pipelines and hardware mixers is insufficient to meet the physical display requirements of a very large aspect ratio automotive display.

[0038] In one example, the car's display refreshes at a frame rate. In one example, the frame rate specifies the periodicity of a full image frame refresh cycle. In one example, the frame rate can be a fixed frame rate, such as 30 frames per second (fps), 60 fps, 120 fps, etc. In one example, the frame rate can be a dynamic frame rate, where the frame rate changes over time. In one example, the frame rate can include multiplexing and area refresh of the display's partitions (i.e., multiple areas). In one example, area refresh means sequentially refreshing full image frames across multiple areas.

[0039] In one example, automotive displays with regional refresh rates can better match the capabilities of mobile SOCs and automotive SOCs. For instance, automotive displays with regional refresh rates could be a specific type of organic light-emitting diode (OLED) screen technology composed of low-temperature polycrystalline oxide (LTPO).

[0040] In one example, a car display with regional refresh may include random access memory (RAM) or may not include RAM (i.e., no RAM). In one example, a car display with regional refresh may support vertical regional refresh (i.e., regional refresh in the vertical direction).

[0041] In one example, the display can be divided into multiple regions. In one example, this division can result in multiple vertical regions or multiple horizontal regions. In one example, each of these multiple regions can have a region width that conforms to either a hardware pipeline width limit or a hardware mixer width limit. For example, if a mobile SoC has a hardware mixer width limit of 2048 pixels, the region width can be equal to 2048 pixels or 4096 pixels, and an 8k pixel wide display can have 2 or 4 regions, depending on the region width.

[0042] In one example, the DPU can allocate hardware pipelines and hardware mixers that correspond to the region width (e.g., 2048 pixels or 4096 pixels). In another example, content layers can be partitioned along with these multiple regions, and hardware pipelines can be allocated accordingly for each region. For example, if the region width is 2048 pixels, the DPU can allocate one hardware mixer for an 8k-pixel-wide display instead of four. In one example, one hardware pipeline can be allocated to process one content layer within a region. For example, if the region width is 4096 pixels, the DPU can allocate two hardware mixers for an 8k-pixel-wide display instead of four. In yet another example, one hardware pipeline can be allocated to process one content layer within a region.

[0043] In one example, the DPU can process multiple content layers and sequentially synthesize multiple composite image frames through each of these multiple regions. For example, each region can have a width of 2048 pixels or 4096 pixels and a height of 800 pixels. In one example, the display can receive multiple pixels and refresh each region sequentially.

[0044] In one example, an automotive display with regional refresh can be an LPTO OLED, a micro-LED OLED, or any other display technology with regional refresh capability.

[0045] Figure 5 An example display processing unit (DPU) architecture 500 for an automotive display with a very large aspect ratio is illustrated. In one example, a mobile SOC 510 performs display processing for an automotive display 520. For example, the automotive display may have a very wide aspect ratio, such as 7680 pixels by 800 pixels. In one example, the display processing for the automotive display 520 is divided into multiple regions. In one example, these multiple regions include a first region 521, a second region 522, a third region 523, and a fourth region 524. In one example, this division into multiple regions is also referred to as content layer partitioning.

[0046] In one example, the display process performs a region refresh of the vehicle display 520. For example, a region refresh refers to sequentially refreshing the vehicle display 520 for each of the plurality of regions. For instance, the vehicle display 520 sequentially refreshes a first region 521, then a second region 522, then a third region 523, and then a fourth region 524, and then repeats this sequence over time to produce a repeating sequence of display portions 530. In one example, the repeating sequence of display portions 530 includes a first display portion 531, a second display portion 532, a third display portion 533, and a fourth display portion 534. In one example, the first display portion 531 (session A) corresponds to the first region 521, the second display portion 532 (session B) corresponds to the second region 522, the third display portion 533 (session C) corresponds to the third region 523, and the fourth display portion 534 (session D) corresponds to the fourth region 524. In one example, time division multiplexing (TDM) is used for the region refresh.

[0047] Figure 6 An example Time Division Multiplexing (TDM) timeline 600 is illustrated. In one example, TDM timeline 600 divides frame time into multiple event times. In one example, TDM timeline 600 depicts an event sequence 610 at a first frame time 620. In one example, the event sequence 610 at the first frame time 620 includes a first event 611 (session A), a second event 612 (session B), a third event 613 (session C), and a fourth event 614 (session D). In one example, the event sequence 610 continues at a second frame time 630 with a fifth event 615 (session A), and so on. In one example, the event duration for each event (i.e., the first event 611, the second event 612, the third event 613, and the fourth event 614) is 4.15 ms. In one example, the first frame time 620 is 16.6 ms (i.e., four times the duration of each event).

[0048] In one example, TDM area refresh for an automotive display enables a System-on-Chip (SoC) (e.g., mobile SoC, automotive SoC, etc.) to support a very wide aspect ratio primary display and multiple secondary displays. In one example, TDM area refresh can scale content layer processing capabilities from four 8k pixel content layers to eight 8k pixel content layers. In another example, TDM area refresh matches DPU processing capabilities to a very wide aspect ratio automotive display.

[0049] Figure 7Example flowchart 700 illustrates a time-division multiplexing (TDM) area refresh for a display screen. In block 710, the display screen is divided into multiple areas. In one example, the display screen is divided into multiple areas. In one example, the horizontal size of the display screen is eight times larger than its vertical size (i.e., a very wide aspect ratio). In one example, the display screen may include a main screen and multiple secondary screens. In one example, the display screen may display a complete image frame.

[0050] In box 720, multiple content layers are ingested from memory. In one example, multiple content layers are ingested from memory. In one example, the multiple content layers may be a red / green / blue (RGB) surface. In one example, the multiple content layers may be a luminance / chrominance (e.g., YUV) surface. In one example, the multiple content layers include a user interface (UI), a game, a video, an image, etc.

[0051] In box 730, multiple Display Processing Unit (DPU) hardware pipelines are sequentially assigned to the multiple regions for region refresh. In one example, multiple Display Processing Unit (DPU) hardware pipelines are sequentially assigned to the multiple regions for region refresh. In one example, each DPU hardware pipeline is assigned to process one of the multiple content layers in each of the multiple regions. In one example, each of the multiple DPU hardware pipelines is assigned to process one content layer in a subset of the multiple regions.

[0052] In box 740, multiple Display Processing Unit (DPU) hardware mixers are sequentially assigned to the multiple regions for region refresh. In one example, multiple Display Processing Unit (DPU) hardware mixers are sequentially assigned to the multiple regions for region refresh. In another example, each of the multiple DPU hardware mixers is assigned to synthesize a subset of the regions.

[0053] In box 750, image signal processing is performed on the regions of the multiple content layers to produce multiple processed layers for region refresh. In one example, image signal processing is performed on the regions of the multiple content layers to produce multiple processed layers for region refresh. In one example, image signal processing includes color format conversion, scaling, image quality improvement, etc., of the source surface. In one example, image signal processing is performed by multiple DPU hardware pipelines.

[0054] In box 760, the multiple processed layers are used to composite multiple composite image frames for region refresh. In one example, multiple processed layers are used to composite multiple composite image frames for region refresh. In one example, compositing includes blending, mixing, etc. In one example, compositing is performed by multiple DPU hardware mixers.

[0055] In box 770, time-division multiplexing (TDM) is used to synthesize a final image frame from the plurality of synthesized image frames for use in area refresh. In one example, time-division multiplexing (TDM) is used to synthesize a final image frame from the plurality of synthesized image frames for use in area refresh. In another example, the synthesis using TDM operates on one of the plurality of areas at a time in a sequential manner.

[0056] In box 780, area refresh is used to transmit the final image frame to one of the multiple areas on the display screen. In one example, area refresh is used to transmit the final image frame to one of the multiple areas on the display screen. In another example, TDM is used to transmit one area sequentially.

[0057] In one respect, Figure 7 One or more steps in the process of providing a display processing unit (DPU) capability for a very large aspect ratio automotive display can be executed by one or more processors, which may include hardware, software, firmware, etc. For example, one or more processors can be used to execute software or firmware that is executing... Figure 7 The steps required in the flowchart. Software should be interpreted broadly to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, processes, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or others.

[0058] Software may reside on a computer-readable medium. The computer-readable medium may be a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical disks (e.g., compact optical disks (CDs) or digital versatile optical disks (DVDs)), smart cards, flash memory devices (e.g., card, stick, or key drives), random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, removable disks, and any other suitable medium for storing software and / or instructions accessible and readable by a computer. By way of example, the computer-readable medium may also include a carrier wave, a transmit line, and any other suitable medium for transmitting software and / or instructions accessible and readable by a computer. The computer-readable medium may reside in a processing system, outside the processing system, or distributed across multiple entities including the processing system. The computer-readable medium may be implemented in a computer program product. For example, a computer program product may include a computer-readable medium in packaging material. The computer-readable medium may include software or firmware. Those skilled in the art will recognize how best to achieve the functionality described throughout this disclosure depends on the specific application and the overall design constraints imposed on the system as a whole.

[0059] Any circuitry included in the processor is provided merely as an example, and other components for performing the described functions may be included in various aspects of this disclosure, including, but not limited to, instructions stored in a computer-readable medium, or any other suitable means or components described herein and utilizing, for example, the processes and / or algorithms described herein with respect to the example flowcharts.

[0060] Within this disclosure, the term "exemplary" is used to mean "serving as an example, instance, or illustration." Any specific implementation or aspect described herein as "exemplary" is not necessarily to be construed as superior to or better than other aspects of this disclosure. Similarly, the term "aspect" does not require that all aspects of this disclosure include the features, advantages, or modes of operation discussed. The term "coupled" is used herein to refer to direct or indirect coupling between two objects. For example, if object A physically contacts object B, and object B contacts object C, objects A and C can still be considered coupled to each other, even if they are not in direct physical contact. The term "circuit" is used broadly, and it is intended to include both hardware implementations of electronic devices and conductors (where these electronic devices and conductors, when connected and configured, perform the functions described in this disclosure, without limitation on the type of electronic circuit) and software implementations of information and instructions (where these information and instructions, when executed by a processor, perform the functions described in this disclosure).

[0061] One or more of the components, steps, features, and / or functions illustrated in the accompanying drawings may be rearranged and / or combined into a single component, step, feature, or function, or embodied in several components, steps, or functions. Additional elements, components, steps, and / or functions may be added without departing from the novel features disclosed herein. The apparatus, devices, and / or components illustrated in the accompanying drawings may be configured to perform one or more of the methods, features, or steps described herein. The novel algorithms described herein can also be efficiently implemented in software and / or embedded in hardware.

[0062] It should be understood that the specific order or hierarchy of steps in the disclosed methods is an example of an exemplary process. It should be understood that the specific order or hierarchy of steps in these methods may be rearranged based on design preferences. The appended method claims present the elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented, unless specifically stated herein.

[0063] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the principles defined herein can be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are to be consistent with the full scope of the claims, wherein references to elements in the singular form are not intended to mean “one and only one”, but rather “one or more”, unless specifically stated otherwise. Unless specifically stated otherwise, the term “some” means one or more. The phrase “at least one of” referring to the list of items means any combination of those items, including individual members. As an example, “at least one of a, b, or c” is intended to cover: a; b; c; a and b; a and c; b and c; and a, b, and c. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. No element of a claim shall be construed under the provisions of 35 USC §112, paragraph 6, unless the element is explicitly stated using the phrase “for a component of” or, in the case of a method claim, the element is stated using the phrase “for a step of”.

[0064] Those skilled in the art will understand that various features of different implementations can be combined or modified and still remain within the spirit and scope of this disclosure.

Claims

1. An apparatus comprising: Multiple display processing unit (DPU) hardware pipelines are configured to perform one or more of the following: color format conversion, scaling, or image quality improvement for one or more content layers; and Multiple DPU hardware mixers coupled to the multiple display processing unit (DPU) hardware pipelines are configured to generate one or more final image frames by performing one of the following: blending, mixing, or compositing the one or more content layers.

2. The apparatus of claim 1, further comprising a display screen coupled to the plurality of DPU hardware mixers, the display screen being configured to display a main screen and a plurality of auxiliary screens.

3. The apparatus of claim 1, further comprising a cross switch coupled to the plurality of DPU hardware mixers, the cross switch being configured to interconnect the plurality of DPU hardware pipelines with the plurality of DPU hardware mixers.

4. The apparatus according to claim 3, wherein, The multiple DPU hardware pipelines are also configured to perform image signal processing on regions of one or more content layers to produce multiple processed layers for region refresh.

5. The apparatus according to claim 4, wherein, The image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

6. The apparatus according to claim 5, wherein, The multiple DPU hardware mixers are also configured to synthesize regional subsets of multiple regions.

7. The apparatus according to claim 6, wherein, The multiple DPU hardware mixers are assigned to the multiple regions.

8. The apparatus according to claim 1, wherein, The multiple DPU hardware mixers are also configured to use multiple processed layers to synthesize multiple composite image frames for region refresh.

9. The apparatus according to claim 8, wherein, The plurality of DPU hardware mixers are also configured to use time-division multiplexing (TDM) to synthesize a final image frame from the plurality of synthesized image frames for use in area refresh.

10. The apparatus according to claim 9, wherein, The multiple DPU hardware mixers are also configured to use area refresh to transmit the final image frame to one of multiple areas in the display screen.

11. A method, the method comprising: Multiple display processing unit (DPU) hardware pipelines are allocated to multiple regions in a first-order manner for region refresh; Multiple display processing unit (DPU) hardware mixers are allocated to the multiple regions in a second order for region refresh; as well as Image signal processing is performed on regions of multiple content layers to produce multiple processed layers for region refresh.

12. The method according to claim 11, wherein, Each of the plurality of display processing unit (DPU) hardware pipelines is assigned to process one of the plurality of content layers in each of the plurality of regions.

13. The method according to claim 11, wherein, Each of the plurality of display processing unit (DPU) hardware pipelines is assigned to a content layer within a subset of the regions that process the plurality of regions.

14. The method according to claim 11, wherein, Each of the plurality of display processing unit (DPU) hardware mixers is assigned to synthesize a subset of the plurality of regions.

15. The method according to claim 11, wherein, The image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

16. The method of claim 11, further comprising using the plurality of processed layers to synthesize a plurality of synthetic image frames for region refresh.

17. The method of claim 16, further comprising using time division multiplexing (TDM) to synthesize a final image frame from the plurality of synthesized image frames for use in area refresh.

18. The method according to claim 17, wherein, The synthesis of the final image frame is performed sequentially, using one region from the plurality of regions at a time.

19. The method of claim 17, further comprising using area refresh to transmit the final image frame to one of a plurality of areas in the display screen.

20. An apparatus comprising: Multiple display processing unit (DPU) hardware pipelines are allocated to multiple regions in a first-order manner for region refresh; Multiple display processing unit (DPU) hardware mixers are allocated to the multiple regions in a second order for region refresh; Perform image signal processing on regions with multiple content layers to produce multiple processed layers for region refresh; The final image frame is transmitted to one of multiple regions on the display screen using a region refresh method; Each of the plurality of display processing unit (DPU) hardware pipelines is assigned to process one of the plurality of content layers in each of the plurality of regions. Each of the plurality of display processing unit (DPU) hardware mixers is assigned to synthesize a subset of the plurality of regions; and The image signal processing includes one or more of the following: color format conversion, scaling, or image quality improvement of one or more content layers.

Images