Rotation of graphics layers in video applications

Abstract

Aspects of the present disclosure relate to rotation of graphics layers in video applications. According to one aspect, the apparatus includes a memory configured to store a video layer in a compressed format and a graphics layer in an uncompressed format; and a display processing unit (DPU) coupled to the memory, the DPU configured to convert the graphics layer into an original compressed graphics layer, and to perform a graphics rotation on the original compressed graphics layer to generate a rotated compressed graphics layer. Also, a method for rotation of a graphics layer includes converting the graphics layer to an original compressed graphics layer using a processing engine; and generating a rotated compressed graphics layer by performing graphics rotation on the original compressed graphics layer using the processing engine.

Description

Cross Reference to Related Applications

[0001] This application claims priority to and the benefit of non-provisional patent application serial number 18 / 346,746 filed in the U.S. Patent Office on July 3, 2023, the entire contents of which are incorporated herein as if set forth in full in the following and for all applicable purposes. TECHNICAL FIELD

[0002] The present disclosure relates generally to the field of graphics layer rotation, and in particular to display processing unit (DPU) based rotation of graphics layers in video applications. BACKGROUND

[0003] The use of multiple processing engines can leverage differences between the processing engines to optimize performance of user applications. For example, one processing engine can be more power efficient than another processing engine. However, certain processing engines can have operational constraints, such as data formats, that limit the types of operations that can be performed on those processing engines. Thus, there is a motivation to adapt processing engines to certain processing operations for optimal performance. SUMMARY

[0004] To provide an overall understanding of one or more aspects of the disclosure, brief summaries of such aspects are given below. This summary is not intended to be an extensive overview of all contemplated features of the disclosure, and is not intended to identify key or critical elements of the disclosure or to delineate the scope of any or all aspects thereof. Its sole purpose is to present some concepts of one or more aspects of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0005] In one aspect, the present disclosure provides for graphics layer rotation. Accordingly, an apparatus for rotating a graphics layer, the apparatus comprising: a memory configured to store a video layer in a compressed format and a graphics layer in an uncompressed format; and a display processing unit (DPU) coupled to the memory, the DPU configured to convert the graphics layer to a raw compressed graphics layer, and to perform a graphics rotation on the raw compressed graphics layer to generate a rotated compressed graphics layer.

[0006] In one example, the graphics rotation is an orthogonal rotation. In one example, the DPU is further configured to generate a rotated video layer for performing a video rotation on the video layer. In one example, the DPU is further configured to generate a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer.

[0007] In one example, the apparatus further includes a video display coupled to the DPU, the video display configured to display the composite rotated image. In one example, the compressed format is a Universal Bandwidth Compression (UBWC) format. In one example, the uncompressed format is a linear RGB (Red Green Blue) format.

[0008] Another aspect of the disclosure provides an apparatus for rotation of graphics layers, the apparatus comprising: a non-transitory memory configured for storing a video layer in a compressed format and a graphics layer in an uncompressed format; a means for converting the graphics layer to an original compressed graphics layer and for performing a graphics rotation on the original compressed graphics layer to generate a rotated compressed graphics layer; and wherein the non-transitory memory is coupled to the means.

[0009] In one example, the compressed format is a Universal Bandwidth Compression (UBWC) format. In one example, the uncompressed format is a linear RGB (Red Green Blue) format. In one example, the graphics rotation is an orthogonal rotation.

[0010] In one example, the apparatus further includes a means for generating a rotated video layer by performing a video rotation on the video layer; and a means for generating a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer.

[0011] Another aspect of the disclosure provides a method for rotation of graphics layers, the method comprising converting a graphics layer to an original compressed graphics layer using a processing engine; and generating a rotated compressed graphics layer by performing a graphics rotation on the original compressed graphics layer using the processing engine.

[0012] In one example, the graphics layer is in an uncompressed format. In one example, the original compressed graphics layer is in a compressed format. In one example, the compressed format is a Universal Bandwidth Compression (UBWC) format. In one example, the uncompressed format is a linear RGB (Red Green Blue) format. In one example, the graphics rotation is an orthogonal rotation.

[0013] In one example, the method further includes generating a rotated video layer by performing a video rotation on a video layer using the processing engine. In one example, the method further includes generating a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer using the processing engine. In one example, the method further includes delivering the composite rotated image to a video display. In one example, the method further includes generating the video layer in a compressed format. In one example, the method further includes generating the graphics layer in an uncompressed format.

[0014] Another aspect of this disclosure provides a non-transitory computer-readable medium storing computer-executable code operable on a device including at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement rotation of a graphics layer, the computer-executable code including instructions for causing a computer to use a processing engine to convert the graphics layer into a raw compressed graphics layer; and instructions for causing a computer to perform graphics rotation on the raw compressed graphics layer using the processing engine to generate a rotated compressed graphics layer.

[0015] In one example, the non-transitory computer-readable medium of claim 1 further includes instructions for causing a computer to perform video rotation on the video layer using a processing engine to generate a rotated video layer. In one example, the non-transitory computer-readable medium of claim 1 further includes instructions for causing a computer to use a processing engine to generate a composite rotated image by combining a rotated compressed graphics layer and a rotated video layer.

[0016] These and other aspects of this disclosure will be more fully understood upon reading the following detailed description. Other aspects, features, and embodiments of this disclosure will become apparent to those skilled in the art after reading the following description of specific exemplary embodiments of the invention in conjunction with the accompanying drawings. Although features of the invention may be discussed below with respect to certain embodiments and drawings, all embodiments of the invention may include one or more of the advantageous features discussed herein. In other words, while one or more embodiments may be discussed as having certain advantageous features, one or more such features may also be used according to the various embodiments of the invention discussed herein. Similarly, although exemplary embodiments may be discussed below as embodiments of an apparatus, system, or method, it should be understood that such exemplary embodiments may be implemented in various apparatuses, systems, and methods. Attached Figure Description

[0017] Figure 1 An example of an information processing system is shown.

[0018] Figure 2 A first example of an information processing system for video playback data streams is illustrated.

[0019] Figure 3 A second example of an information processing system for video playback data streams is illustrated.

[0020] Figure 4 An example flowchart is shown for implementing the rotation of two layers in a video playback application. Detailed Implementation

[0021] The detailed description below, taken in conjunction with the accompanying drawings, is intended as a description of various configurations and not as representing the only configuration in which the concepts described herein can be practiced. To provide a thorough understanding of the various concepts, the detailed description includes specific details. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring these concepts.

[0022] While for the purpose of simplification, a method is shown and described as a series of actions, it should be understood and recognized that the method is not restricted by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions, depending on one or more aspects. For example, those skilled in the art will understand and recognize that a method may alternatively be represented as a series of related states or events, such as in a state diagram. Furthermore, not all exemplified actions are required to implement a method according to one or more aspects.

[0023] Modern information processing systems rely on multiple processing engines to perform a variety of computational and logical tasks. In one example, multiple processing engines are used to increase operational throughput (i.e., operations per second). For example, a central processing unit (CPU) can be used for general-purpose computing and monitoring other processors. In another example, multiple processing engines may include specialized processing engines specifically designed to perform certain specialized tasks.

[0024] A common use of information processing systems is in mobile devices (e.g., smartphones). In one example, mobile devices provide a variety of user applications, such as mobile phone use, text messaging, email, internet access, social media, video entertainment, games, audio programming, news updates, music entertainment, financial data, and so on. Many user applications require a video display to present information to the user. That is, the information processing system in a mobile device performs video processing to deliver video information to the video display.

[0025] In one example, an information processing system may include the following dedicated processing engines: 1. Graphics Processing Unit (GPU) - Used for graphics or numerical processing. 2. Display Processing Unit (DPU) - Used to display image or video data on a video display. 3. Neural Processing Unit (NPU) - Used for neural processing tasks 4. Image Processing Unit (IPU) - Used for image processing tasks 5. Digital Signal Processor (DSP) - Used for digital signal processing tasks 6. Network Processor (NP) - Used for networking applications

[0026] One application of information processing systems is video processing; that is, the manipulation and display of video information. In one example, video information is a temporal sequence of image frames. For example, an image frame may have a total of N image pixels, which are organized into a concatenation of J rows and K columns. For example, video information may be displayed as a series of image frames at a specific frame rate (in frames per second, fps). In one example, image pixels are the basic elements of an image display or video display.

[0027] In one example, information content can be quantified using an information metric, in bits for static information sources (e.g., a single image frame) or in bits per second for dynamic information sources (e.g., video information). In one example, a bit is an atomic measure of information content that has only two states, generally represented as zero or one. In one example, any type of information can be quantified in bits or bits per second.

[0028] In one example, information entropy quantifies the degrees of freedom (in bits) in an information source. For instance, if a first information source has a first information entropy of H1 bits and a second information source has a second information entropy of H2 bits, then if H1 > H2, the first information source has more degrees of freedom than the second information source. That is, higher information entropy corresponds to greater uncertainty or randomness in the information, and therefore requires more bits to represent it.

[0029] In one example, information can be represented in two common formats: uncompressed and compressed. For example, uncompressed information is information in its original form with the rate of the original source information. For example, compressed information is information in a simplified form with the rate of compressed information. For example, compressed information utilizes information entropy by representing information with fewer bits than uncompressed information. In one example, the rate of compressed information is less than the rate of the original source information. For example, the rate of compressed information may be less than 10% of the rate of the original source information.

[0030] In one example, there are two image data types that can be manipulated and processed before being displayed on the display unit. In one example, the video layer is the actual image to be displayed on the video display. In one example, the video layer is represented in a compressed format. In one example, the compressed format is Universal Bandwidth Compression (UBWC). In one example, the graphics layer is overlay data (e.g., annotations, labels, notes, etc.) to be displayed on the video display. In one example, the graphics layer is represented in an uncompressed format. In one example, the uncompressed format is linear RGB (red, green, blue) format. In one example, the linear RGB format represents information as a superposition of multiple color basis vectors (e.g., red, green, blue) to synthesize arbitrary image pixels.

[0031] In one example, a video playback application on a mobile device may send both a video layer and a graphics layer to an information processing system. For example, the graphics layer may include views, annotations, and overlay data to be displayed on the video data from the video layer. In one example, the graphics layer is generated by the CPU of the information processing system. In one example, the graphics layer is represented in an uncompressed format (e.g., linear RGB format). In one example, the video layer is represented in a compressed format (e.g., UBWC format).

[0032] In one example, both the video layer and the graphics layer may need to be rotated by a certain angle relative to the video display orientation relative to the user holding the mobile device. In one example, the rotation angle is 90 degrees (i.e., orthogonal rotation). In another example, the rotation angle is less than 90 degrees (i.e., non-orthogonal rotation). In yet another example, the rotation angle is greater than 90 degrees (i.e., also non-orthogonal rotation).

[0033] In one example, the first processing engine is capable of rotating two or more layers. In another example, the first processing engine only rotates two or more layers if they are in a compressed format (e.g., UBWC format). In yet another example, the first processing engine is a Display Processing Unit (DPU). For example, the first processing engine is capable of rotating video layers.

[0034] In one example, the second processing engine is capable of performing rotation on at least one layer in an uncompressed format (e.g., linear RGB format). In one example, the second processing engine requires more DC power than the first processing engine to perform the rotation. In one example, the second processing engine is a graphics processing unit (GPU). In one example, the second processing engine is capable of rotating graphics layers.

[0035] In one example, for a video playback application on a mobile device, the rotation of the video layer is performed by the DPU, while the rotation of the graphics layer is performed by the GPU. In one example, the DPU has more efficient DC power consumption compared to the GPU.

[0036] In some applications (e.g., video playback), it is desirable for the DPU to provide a two-pass processing scheme, which allows both the video layer and the graphics layer to rotate with improved DC power efficiency.

[0037] Figure 1An example of an information processing system 100 is illustrated. In one example, the information processing system 100 includes multiple processing engines, such as a central processing unit (CPU) 120, a digital signal processor (DSP) 130, a graphics processing unit (GPU) 140, a display processing unit (DPU) 180, etc. In one example, various other functions may be included in the information processing system 100, such as a support system 110, a modem 150, memory 160, cache memory 170, and a video display 190. For example, the multiple processing engines and various other functions may be interconnected via an interconnect data bus 105 to transfer data and control information. For example, memory 160 and / or cache memory 170 may be shared among the CPU 120, GPU 140, and other processing engines. In one example, the CPU 120 may include a first internal memory that is not shared with other processing engines. In one example, the GPU 140 may include a second internal memory that is not shared with other processing engines. In one example, any of the multiple processing engines may have internal memory that is not shared with other processing engines.

[0038] Figure 2 A first example of an information processing system 200 for video playback data streams is illustrated. The first example of the information processing system 200 includes a memory 210, a display processing unit (DPU) 220, a graphics processing unit (GPU) 230, a video display 240, and a central processing unit (CPU) 250 (not shown). In one example, the memory 210 is a double data rate (DDR) memory.

[0039] In one example, memory 210 stores a first video layer 211 with an initial orientation. In one example, the first video layer 211 includes a first video image. For example, the first video image is a temporally ordered sequence of image frames. For example, the first video image may have a total of N image pixels, which are organized into a concatenation of J rows and K columns. For example, J = 1440 rows and K = 2560 columns, for a total of N = 3,686,400 image pixels. For example, the first video layer 211 may be displayed as a series of image frames at a specific frame rate (in frames per second, fps). In one example, the first video layer 211 is represented in a compressed format. For example, the compressed format is Universal Bandwidth Compression (UBWC) format.

[0040] In one example, memory 210 stores a first graphics layer 212 having a first orientation. In one example, the first graphics layer 212 is a CPU-rendered bullet screen. For example, the first graphics layer 212 may have a total of P image pixels, which are organized into a cascade of L rows and M columns. For example, L = 1440 rows and L = 3120 columns, for a total of P = 4,492,800 image pixels. In one example, the first graphics layer 212 is represented in an uncompressed format. For example, the uncompressed format is linear RGB format.

[0041] In one example, memory 210 stores a second graphics layer 213 with a second orientation. In one example, the second orientation is rotated from the first orientation. In one example, the second orientation is rotated 90 degrees from the first orientation. In one example, the second graphics layer 213 is represented in an uncompressed format. For example, the uncompressed format is a linear RGB format.

[0042] In one example, the graphics processing unit (GPU) 230 accepts a first graphics layer 212 with a first orientation as GPU input. In another example, the GPU 230 rotates the first graphics layer 212 to produce a second graphics layer 213 with a second orientation as GPU output. In yet another example, the GPU 230 operates in an uncompressed format (e.g., linear RGB format).

[0043] In one example, the Display Processing Unit (DPU) 220 includes a first video image processing engine (VIG0) 221, a second video image processing engine (VIG1) 222, a Direct Memory Access (DMA) 223, a first layer mixer (LM0) 224, a second layer mixer (LM1) 225, a first display compressor (DSC) 226, a second display compressor (DSC) 227, an interface (Intf) 228, and a Display Stream Interface (DSI) 229. In one example, the DPU 220 accepts a first video layer 211 with an initial orientation as a first DPU input, which is fed to the first video image processing engine (VIG0) 221 and the second video image processing engine (VIG1) 222. In one example, the video image processing engine (VIG) may be a video image scaler that retrieves video images from memory and performs video processing such as color space conversion, image resizing, and image rotation as necessary. In one example, each VIG has the same design. In one example, multiple video image processing engines can operate in parallel to process high-resolution or high-refresh-rate displays, thereby increasing the video processing throughput of a given display (e.g., by a factor of two, four, etc.). In one example, Direct Memory Access (DMA) can be a graphics image processing pipeline that retrieves graphics images from memory and delivers them to the display. In one example, multiple DMAs can operate in parallel to process high-resolution or high-refresh-rate displays, thereby increasing graphics image processing throughput. In one example, the DMA pipeline may not handle color space conversion, image scaling, and image rotation. In one example, a Layer Mixer (LM) processes overlays of multiple graphics and video layers from multiple VIGs and multiple DMAs before delivering overlay images to the display. In one example, a Display Compressor (DSC) compresses display data to reduce the bandwidth or data rate on the connection between the display processor and the display. In one example, a Display Stream Interface (DSI) can be used to connect the display processor to the actual display.

[0044] In one example, DPU 220 accepts a second graphics layer 213 with a second orientation as a second DPU input fed to Direct Memory Access (DMA) 223. In one example, a first video layer 211 with an initial orientation is rotated by a first video image processing engine (VIG0) 221 and a second video image processing engine (VIG1) 222 to produce a second video layer 214 (not shown) with a rotation orientation. In one example, the rotation orientation is rotated from the initial orientation. In one example, the rotation orientation is rotated 90 degrees from the initial orientation. In one example, the first video image processing engine (VIG0) 221 and the second video image processing engine (VIG1) 222 operate in a compressed format (e.g., UBWC format).

[0045] In one example, the second video layer 214 and the second graphics layer 213 undergo additional processing by a first layer mixer (LM0) 224, a second layer mixer (LM1) 225, a first display compressor (DSC) 226, a second display compressor (DSC) 227, an interface (Intf) 228, and a display stream interface (DSI) 229 to produce a composite rotated image 241 in the video display 240. In one example, the composite rotated image 241 comprises an overlay of the second video layer 214 and the second graphics layer 213 having the same orientation.

[0046] Figure 3 A second example of an information processing system 300 for video playback data streams is illustrated. The second example of the information processing system 300 includes a memory 310, a display processing unit (DPU) 320, a graphics processing unit 330 (not shown), a video display 340, and a central processing unit (CPU) 350 (not shown). In one example, the memory 310 is a double data rate (DDR) memory.

[0047] In one example, memory 310 stores a first video layer 311 with an initial orientation. In one example, the first video layer 311 includes a first video image. For example, the first video image is a temporally ordered sequence of image frames. For example, the first video image may have a total of N image pixels, which are organized into a concatenation of J rows and K columns. For example, J = 1440 rows and K = 2560 columns, for a total of N = 3,686,400 image pixels. For example, the first video layer 311 may be displayed as a series of image frames at a specific frame rate (in frames per second, fps). In one example, the first video layer 311 is represented in a compressed format. For example, the compressed format is Universal Bandwidth Compression (UBWC).

[0048] In one example, memory 310 stores a first graphics layer 312 having a first orientation. In one example, the first graphics layer 312 is a CPU-rendered bullet screen. For example, the first graphics layer 312 may have a total of P image pixels, which are organized into a cascade of L rows and M columns. For example, L = 1440 rows and L = 3120 columns, for a total of P = 4,492,800 image pixels. In one example, the first graphics layer 312 is represented in an uncompressed format. For example, the uncompressed format is linear RGB format.

[0049] In one example, memory 310 stores a second graphics layer 313 having a first orientation. In one example, the second graphics layer 313 is represented in a compressed format. In one example, the compressed format is UBWC format. In one example, source (SRC) splitting is a processing function that splits a video or graphics source image into multiple segments or slices. For example, each slice can be assigned to a VIG or DMA pipeline in a display processor. For example, dividing a video or graphics source image into multiple slices allows for parallel processing by multiple VIG or DMA source pipelines to provide increased processing throughput for high-resolution and / or high-refresh-rate displays with specific pixel throughput speed targets.

[0050] In one example, the display processing unit (DPU) 320 includes a first video image processing engine (VIG0) 321, a second video image processing engine (VIG1) 322, a third video image processing engine (VIG2) 351, a fourth video image processing engine (VIG3) 352, a first layer mixer (LM0) 324, a second layer mixer (LM1) 325, a first display compressor (DSC) 326, a second display compressor (DSC) 327, an interface (Intf) 328, and a display stream interface (DSI) 329.

[0051] In one example, the display processing unit (DPU) 320 also includes a direct memory access (DMA) 323, a third-layer mixer (LM4) 354, a fourth-layer mixer (LM5) 355, a merging function 356, and a display image write-back module (WB2) 357.

[0052] In one example, DPU 320 accepts a first graphics layer 312 as input to Direct Memory Access (DMA) 323. In one example, a third-layer mixer (LM4) 354 and a fourth-layer mixer (LM5) 355 perform format conversion on the first graphics layer 312. In one example, the format conversion is from uncompressed to compressed format. In one example, the format conversion is from linear RGB format to UBWC format. In one example, format conversion is performed to facilitate the operations performed by DPU 320. In one example, a merging function 356 and a display image write-back module (WB2) 357 provide further processing to produce a second graphics layer 313. In one example, the second graphics layer 313 is a compressed version of the first graphics layer 312. In one example, the second graphics layer is stored in memory 310.

[0053] In one example, DMA 323 can be used to acquire the raw graphic image in RGB linear format. For instance, DMA 323 can acquire the left and right halves of the image in parallel to provide increased image processing throughput, thereby doubling the image processing throughput. In one example, if there are more than one layer, the parallel-operating third-layer mixer LM4 354 and fourth-layer mixer LM5 355 can provide further processing, such as layer combination. In one example, the merging function 356 can combine the left and right halves of the image to form a composite image.

[0054] In one example, the display image write-back module (WB2) 357 can be used to write an image back to memory. In one example, WB2 356 can also perform UBWC compression on the image to produce a UBWC compressed format. In one example, VIG0 321 and VIG1 322 can be used to process video images in parallel, with each processing half of the video image. In one example, VIG0 321 and VIG1 322 can perform color space conversion, scaling, and image rotation. In one example, VIG2 351 and VIG3 352 can be used to process graphics images in parallel, with each processing half of the graphics image by performing image rotation.

[0055] In one example, LM0 324 and LM1 325 operate in parallel to perform overlay of graphic and video images to produce an output image. For example, LM0 324 and LM1 325 each process half of the output image.

[0056] In one example, the output image is compressed by a display compressor (DSC) before being sent to interface (Intf) 328. In one example, interface (Intf) 328 provides display timing information and supplies the output display image (pixel-by-pixel, line-by-line) to the display output connected to the video display 340 via a physical connection. In one example, the physical connection uses display stream interface (DSI) 329.

[0057] In one example, DPU 320 accepts a first video layer 311 with an initial orientation as a first DPU input, which is fed to a first video image processing engine (VIG0) 321 and a second video image processing engine (VIG1) 322. In one example, the first video layer 311 with the initial orientation is rotated by the first video image processing engine (VIG0) 321 and the second video image processing engine (VIG1) 322 to produce a second video layer 314 (not shown) with a rotation orientation. In one example, the rotation orientation is rotated from the initial orientation. In another example, the rotation orientation is rotated 90 degrees from the initial orientation. In one example, the first video image processing engine (VIG0) 321 and the second video image processing engine (VIG1) 322 operate in a compressed format (e.g., UBWC format).

[0058] In one example, DPU 320 accepts a second graphics layer 313 with a first orientation as a second DPU input, which is fed to a third video image processing engine (VIG2) 351 and a fourth video image processing engine (VIG3) 352. In one example, the second graphics layer 313 with the first orientation is rotated by the third video image processing engine (VIG2) 351 and the fourth video image processing engine (VIG3) 352 to produce a third graphics layer 315 (not shown) with a second orientation. In one example, the second orientation is rotated from the first orientation. In one example, the second orientation is rotated 90 degrees from the first orientation. In one example, the third video image processing engine (VIG2) 351 and the fourth video image processing engine (VIG3) 352 operate in a compressed format (e.g., UBWC format). In one example, the first orientation is the same as the initial orientation. In one example, the second orientation is the same as the rotated orientation.

[0059] In one example, the second video layer 314 and the third graphics layer 315 undergo additional processing by a first layer mixer (LM0) 324, a second layer mixer (LM1) 325, a first display compressor (DSC) 326, a second display compressor (DSC) 327, an interface (Intf) 328, and a display stream interface (DSI) 329 to produce a composite rotated image 341 in the video display 340. In one example, the composite rotated image 341 comprises a superposition of the second video layer 314 and the third graphics layer 315 with the same rotation angle. That is, the second video layer 314 and the third graphics layer 315 have a certain orientation after being rotated by the same rotation angle. In one example, the same rotation angle is 90 degrees (i.e., orthogonal rotation). In one example, the same rotation angle is less than or greater than 90 degrees (i.e., non-orthogonal rotation). In one example, the composite rotated image 341 includes a video image and views, annotations, and overlay data displayed together with the video image from the video layer.

[0060] Figure 4 Example flowchart 400 illustrates the rotation of two layers for a video playback application. In box 410, the video layer is generated in a compressed format. That is, the video layer is generated in a compressed format.

[0061] In one example, the video layer is first generated in an uncompressed format before the source decoding process that generates the video layer in a compressed format. In one example, the compressed format is Universal Bandwidth Compression (UBWC). In one example, the video layer is a temporally ordered sequence of image frames. For example, an image frame may have a total of N image pixels, organized into a concatenation of J rows and K columns. For example, video information can be displayed as a series of image frames at a specific frame rate (in frames per second, fps). In one example, the video layer may be stored in memory. In one example, the memory is DDR memory. In one example, the memory is included in the mobile device. In one example, the memory is non-transitory memory.

[0062] In box 420, the graphics layer is generated in an uncompressed format. That is, the graphics layer is generated in an uncompressed format. In one example, the uncompressed format is linear RGB (red, green, blue) format. In one example, the graphics layer is a CPU-rendered bullet screen. For example, the graphics layer may have a total of P image pixels, which are organized into a cascade of L rows and M columns. In one example, the graphics layer may be stored in memory. In one example, the memory is DDR memory. In one example, the memory is non-transitory memory.

[0063] In box 430, the uncompressed graphics layer is converted to a compressed original graphics layer. That is, the uncompressed graphics layer is converted to a compressed original graphics layer. In one example, the compression format is UBWC format. In one example, the conversion from uncompressed to compressed format is performed by a first processing engine. In one example, the first processing engine is a Display Processing Unit (DPU). In one example, the conversion from uncompressed to compressed format is performed by cascading two parallel processing paths within the first processing engine. In one example, the original compressed graphics layer may be stored in memory. In one example, the memory is DDR memory. In one example, the first processing engine is included in a mobile device.

[0064] In box 440, a rotated compressed graphics layer is generated by performing a graphics rotation on the original compressed graphics layer. That is, the rotated compressed graphics layer is generated by performing a graphics rotation on the original compressed graphics layer. In one example, the graphics rotation is performed by a first processing engine. In another example, the graphics rotation is performed by a first segment of the video image processing engine within the first processing engine. In one example, the graphics rotation results in an orientation of the rotated compressed graphics layer that is orthogonal to the original compressed graphics layer. In another example, the graphics rotation results in an orientation of the rotated compressed graphics layer that is not orthogonal to the original compressed graphics layer. In yet another example, the graphics rotation depends on the video display orientation relative to the user holding the mobile device.

[0065] In box 450, a rotated video layer is generated by performing video rotation on the video layer. That is, the rotated video layer is generated by performing video rotation on the video layer. In one example, the video rotation is performed by a first processing engine. In another example, the video rotation is performed by a second segment of the video image processing engine within the first processing engine. In one example, the video rotation results in a rotated video layer with an orientation orthogonal to the video layer. In another example, the graphic rotation results in a rotated video layer with an orientation not orthogonal to the video layer. In yet another example, the graphic rotation depends on the video display orientation relative to the user holding the mobile device.

[0066] In box 460, a composite rotated image is generated by combining a rotated compressed graphics layer and a rotated video layer. That is, the composite rotated image is generated by combining a rotated compressed graphics layer and a rotated video layer. In one example, the composite rotated image is combined by a first processing engine. In one example, the composite rotated image has a compositing orientation that depends on the video display orientation relative to the user holding the mobile device.

[0067] In box 470, the composite rotated image is delivered to the video display. That is, the composite rotated image is delivered to the video display. In one example, the video display converts the composite rotated image from a compressed format to an uncompressed format before displaying it on the video display. In one example, the video display is included in a mobile device.

[0068] In one respect, Figure 4 One or more steps in the process can be executed by one or more processors, which may include hardware, software, firmware, etc. For example, one or more processors can be used to execute software or firmware that is executing... Figure 4 The steps required in the flowchart. Software should be interpreted broadly to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, processes, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or others.

[0069] Software may reside on a computer-readable medium. The computer-readable medium may be a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes), optical disks (e.g., compact optical disks (CDs) or digital versatile optical disks (DVDs)), smart cards, flash memory devices (e.g., card, stick, or key drives), random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, removable disks, and any other suitable medium for storing software and / or instructions accessible and readable by a computer. By way of example, the computer-readable medium may also include a carrier wave, a transmit line, and any other suitable medium for transmitting software and / or instructions accessible and readable by a computer. The computer-readable medium may reside in a processing system, outside the processing system, or distributed across multiple entities including the processing system. The computer-readable medium may be implemented in a computer program product. For example, a computer program product may include a computer-readable medium in packaging material. The computer-readable medium may include software or firmware. Those skilled in the art will recognize how best to achieve the functionality described throughout this disclosure depends on the specific application and the overall design constraints imposed on the system as a whole.

[0070] Any circuitry included in the processor is provided merely as an example, and other components for performing the described functions may be included in various aspects of this disclosure, including, but not limited to, instructions stored in a computer-readable medium, or any other suitable means or components described herein and utilizing, for example, the processes and / or algorithms described herein with respect to the example flowcharts.

[0071] Within this disclosure, the term "exemplary" is used to mean "serving as an example, instance, or illustration." Any specific implementation or aspect described herein as "exemplary" is not necessarily to be construed as superior to or better than other aspects of this disclosure. Similarly, the term "aspect" does not require that all aspects of this disclosure include the features, advantages, or modes of operation discussed. The term "coupled" is used herein to refer to direct or indirect coupling between two objects. For example, if object A physically contacts object B, and object B contacts object C, then object A and object C can still be considered coupled to each other, even if they are not in direct physical contact. The term "circuit" is used broadly, and it is intended to include both hardware implementations of electronic devices and conductors (where these electronic devices and conductors, when connected and configured, perform the functions described in this disclosure, without limitation on the type of electronic circuit) and software implementations of information and instructions (where these information and instructions, when executed by a processor, perform the functions described in this disclosure).

[0072] One or more of the components, steps, features, and / or functions illustrated in the accompanying drawings may be rearranged and / or combined into a single component, step, feature, or function, or embodied in several components, steps, or functions. Additional elements, components, steps, and / or functions may be added without departing from the novel features disclosed herein. The apparatus, devices, and / or components illustrated in the accompanying drawings may be configured to perform one or more of the methods, features, or steps described herein. The novel algorithms described herein can also be efficiently implemented in software and / or embedded in hardware.

[0073] It should be understood that the specific order or hierarchy of steps in the disclosed methods is an example of an exemplary process. It should be understood that the specific order or hierarchy of steps in these methods may be rearranged based on design preferences. The appended method claims present the elements of various steps in an exemplary order, but are not intended to limit them to the specific order or hierarchy presented, unless specifically stated herein.

[0074] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are to be consistent with the full scope of the claims, wherein references to elements in the singular form are not intended to mean “one and only one”, but rather “one or more”, unless specifically stated otherwise. Unless specifically stated otherwise, the term “some” refers to one or more. The phrase “at least one of” referring to the list of items means any combination of those items, including a single member. As an example, “at least one of a, b, or c” is intended to cover: a; b; c; a and b; a and c; b and c; and a, b, and c. All structural and functional equivalents of the elements throughout the various aspects described herein that are known to or will later be known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims.

[0075] Those skilled in the art will understand that various features of different implementations can be combined or modified and still remain within the spirit and scope of this disclosure.

Claims

1. An apparatus for rotating a graphic layer, the apparatus comprising: A memory configured to store the video layer in a compressed format and the graphics layer in an uncompressed format; and Display processing unit (DPU), coupled to the memory, is configured to convert the graphics layer into a raw compressed graphics layer and to perform graphics rotation on the raw compressed graphics layer to generate a rotated compressed graphics layer.

2. The apparatus according to claim 1, wherein the graphic rotation is an orthogonal rotation.

3. The apparatus of claim 1, wherein the DPU is further configured to generate a rotated video layer for performing video rotation on the video layer.

4. The apparatus of claim 3, wherein the DPU is further configured to generate a synthetic rotating image by combining the rotating compressed graphics layer and the rotating video layer.

5. The apparatus of claim 4, further comprising a video display coupled to the DPU, the video display being configured to display the synthesized rotating image.

6. The apparatus of claim 4, wherein the compression format is Universal Bandwidth Compression (UBWC) format.

7. The apparatus of claim 6, wherein the uncompressed format is a linear RGB (red, green, blue) format.

8. An apparatus for rotating a graphic layer, the apparatus comprising: Non-transitory memory, configured to store the video layer in a compressed format and the graphics layer in an uncompressed format; Components for converting the graphic layer into an original compressed graphic layer and for performing graphic rotation on the original compressed graphic layer to generate a rotated compressed graphic layer; and The non-transitory memory is coupled to the component.

9. The apparatus of claim 8, wherein the compression format is Universal Bandwidth Compression (UBWC) format.

10. The apparatus of claim 8, wherein the uncompressed format is a linear RGB (red, green, blue) format.

11. The apparatus of claim 9, further comprising: Components for generating a rotated video layer by performing video rotation on the video layer; and A component for generating a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer.

12. The apparatus of claim 8, wherein the graphic rotation is an orthogonal rotation.

13. A method for rotating a graphics layer, the method comprising: The processing engine is used to convert the graphics layer into a raw, compressed graphics layer. as well as A rotated compressed graphics layer is generated by performing graphic rotation on the original compressed graphics layer using the processing engine.

14. The method of claim 13, wherein the graphics layer is in an uncompressed format.

15. The method of claim 14, wherein the original compressed graphics layer is in a compressed format.

16. The method of claim 15, further comprising generating a rotated video layer by performing video rotation on the video layer using the processing engine.

17. The method of claim 16, further comprising using the processing engine to generate a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer.

18. The method of claim 17, further comprising delivering the synthesized rotated image to a video display.

19. The method of claim 18, further comprising generating the video layer in the compression format.

20. The method of claim 19, wherein the compression format is Universal Bandwidth Compression (UBWC) format.

21. The method of claim 19, further comprising generating the graphics layer in the uncompressed format.

22. The method of claim 21, wherein the uncompressed format is a linear RGB (red, green, blue) format.

23. The method of claim 13, wherein the graphic rotation is an orthogonal rotation.

24. A non-transitory computer-readable medium storing computer-executable code, the computer-executable code being operable on a device, the device including at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement rotation of a graphics layer, the computer-executable code comprising: Instructions for enabling a computer to use its processing engine to convert the graphics layer into a raw, compressed graphics layer; and Instructions for causing the computer to generate a rotated compressed graphics layer by performing graphics rotation on the original compressed graphics layer using the processing engine.

25. The non-transitory computer-readable medium of claim 24, further comprising instructions for causing the computer to generate a rotated video layer by performing video rotation on the video layer using the processing engine.

26. The non-transitory computer-readable medium of claim 24, further comprising instructions for causing the computer to use the processing engine to generate a composite rotated image by combining the rotated compressed graphics layer and the rotated video layer.

Images