Video frame codec architecture

Abstract

Video frame codec architectures are disclosed. Techniques and apparatuses for video frame codec architectures are described. A frame decompressor decompresses compressed frames to produce decompressed frames. A frame decompressor controller arbitrates shared access to the frame decompressor. Multiple cores of a SoC request decompressed frames from the frame decompressor via the frame decompressor controller. The frame decompressor controller can implement a request queue and can order servicing of requests based on priority of the requests or requesting core. The frame decompressor controller can also establish a time-sharing protocol for access by the multiple cores. In some implementations, a video decoder is integrated with frame decompressor logic and stores portions of the decompressed frames in a video buffer and a display controller retrieves portions for display using a synchronization mechanism. In a similar manner, a frame compressor controller can arbitrate shared access to a frame compressor by multiple cores.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese Invention Patent Application No. 201880028788.3, ​​filed on July 13, 2018. Technical Field

[0003] This application relates to video frame codec architecture. Background Technology

[0004] Electronic devices play an indispensable role in manufacturing, communications, healthcare, commerce, social interaction, and entertainment. For example, electronic devices power server clusters, which in turn provide cloud-based distributed computing capabilities for business and communications. Computational devices are also embedded in many different types of modern equipment, from medical devices to appliances, from vehicles to industrial tools. Furthermore, one electronic device—the smartphone—has become an essential item to carry with you.

[0005] Many electronic devices, such as those with cameras or displays, can manipulate video data. For example, video can be acquired using a security camera and then enhanced to improve certain visual aspects, such as sharpness or contrast. Existing video data can also be manipulated to improve the appearance of individual video frames used for presentation on the screens of smartphones or television monitors. For example, video data from a movie can be processed to enhance the realism of artificial graphics or increase display resolution. Video image data is also manipulated in industrial and medical settings. For instance, image data from a 3D body scan can be stitched together to create a video presentation for doctors to examine and analyze.

[0006] In any of these cases, manipulating video data is a processing-intensive task. This is partly due to the size or amount of information typically present in video data. Therefore, the area of ​​an integrated circuit (IC) chip dedicated to processing video data can be larger than that used for other types of data. The ever-increasing display resolution of video that electronic devices are expected to process further complicates video data processing. For example, high-definition (HD) video data is approximately four times that of standard-definition (SD) video, and ultra-high-definition (UHD) or 4K video data is approximately four times that of HD video.

[0007] Therefore, the amount of video data that electronic devices are expected to process has increased dramatically over the past decade or so. With the increasing prevalence of virtual reality (VR) and artificial reality (AR), the demand for video data processing is expected to increase further in the coming years. Consequently, electronic device manufacturers continue to strive to improve the ability of their devices to handle the ever-growing volume of video data.

[0008] This background description is provided to present the overall context of this disclosure. Unless otherwise stated herein, the material described in this section is neither explicitly nor implicitly considered to be prior art of this disclosure or the appended claims. Summary of the Invention

[0009] Techniques and apparatus for video frame codec architectures are described. These techniques and apparatus enable integrated circuit (IC) chips to process high-bandwidth video data with fewer circuit resources, while also facilitating streamlined workflows for upgrades to newer frame compression / decompression technologies, including lossless technologies. To this end, multiple separate frame decompression units are avoided across multiple different cores of the IC chip. Instead, the frame decompressor can provide frame decompression services to multiple different cores acting as frame decompressor client circuitry. The frame decompressor controller facilitates the sharing of decompression services using a queuing or prioritization mechanism that orders decompression requests received from one or more of the multiple cores. The frame decompressor controller can also arbitrate access to the frame decompressor according to a time-sharing protocol. In an example implementation, the frame decompressor is located in the same location as the video decoder client circuitry, and the frame decompressor is shared at least with the display controller. Similarly, the frame compressor can provide frame compression services to multiple different cores acting as frame compressor client circuitry. The frame compressor controller facilitates the sharing of compression services with multiple cores. Furthermore, both frame compression and frame decompression services can be provided within a single IC chip for sharing among multiple cores.

[0010] The aspects described below include an electronic device comprising a frame decompressor and a frame decompressor controller. The frame decompressor is configured to decompress a plurality of compressed frames to produce a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and configured to arbitrate access to the frame decompressor by a plurality of cores. The plurality of cores includes a first core and a second core. The first core is coupled to the frame decompressor controller and is configured to obtain, via the frame decompressor controller, one of the decompressed frames produced by the frame decompressor. The second core is coupled to the frame decompressor controller and is configured to obtain, via the frame decompressor controller, another decompressed frame from the plurality of decompressed frames produced by the frame decompressor. For example, the first core may be configured to obtain a first decompressed frame from the plurality of decompressed frames produced by the frame decompressor via the frame decompressor controller. The second core may be configured to obtain a second decompressed frame from the plurality of decompressed frames produced by the frame decompressor via the frame decompressor controller.

[0011] The aspects described below also include a method for sharing frame decompression circuitry among multiple cores. The method includes receiving a first request from a first core for a first decompressed frame. The method further includes decompressing the first compressed frame to produce a first decompressed frame. In response to the first request, the first decompressed frame is provided to the first core. The method also includes receiving a second request from a second core for a second decompressed frame. The method further includes decompressing the second compressed frame to produce a second decompressed frame. In response to the second request, the second decompressed frame is provided to the second core.

[0012] The aspects described below include another electronic device comprising a video decoder and a display controller. The video decoder is configured to decode a video stream to produce a plurality of decoded frames. The video decoder includes a frame compressor, a frame decompressor, and a frame decompressor controller. The frame compressor is configured to compress the plurality of decoded frames to produce a plurality of compressed frames. The frame decompressor is configured to decompress the plurality of compressed frames to produce a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and is configured to arbitrate access to the frame decompressor. The display controller is coupled to the frame decompressor controller. The display controller is configured to obtain one of the plurality of decompressed frames produced by the frame decompressor via the frame decompressor controller.

[0013] The aspects described below also include a system comprising a frame decompressor and a plurality of cores. The frame decompressor is configured to decompress a plurality of compressed frames to produce a plurality of decompressed frames. The plurality of cores includes a first core and a second core. The first core is coupled to the frame decompressor and is configured to obtain one of the decompressed frames from the plurality of decompressed frames. The second core is coupled to the frame decompressor and is configured to obtain another decompressed frame from the plurality of decompressed frames. For example, the first core may be configured to obtain a first decompressed frame from the plurality of decompressed frames. The second core may be configured to obtain a second decompressed frame from the plurality of decompressed frames. The system also includes control means for controlling the frame decompressor to arbitrate access to the frame decompressor for the plurality of cores (including the first and second cores). Additionally or alternatively, the system may include a frame compressor configured to compress a plurality of uncompressed frames to produce a plurality of compressed frames. The first core and the second core may each obtain a corresponding compressed frame from the plurality of compressed frames. Therefore, the system may also include control means for arbitrating access to the frame compressor for multiple cores (including the first core and the second core). Attached Figure Description

[0014] The following figures illustrate apparatus and techniques for implementing a video frame codec architecture.

[0015] In all the accompanying figures, the same numbers are used to refer to similar features and components:

[0016] Figure 1 An example environment including a printed circuit board is shown, in which a video frame codec architecture can be implemented.

[0017] Figure 2 Other aspects of an example environment in which a video frame codec architecture can be implemented are shown.

[0018] Figure 3 A system-on-chip (SoC) with an example implementation of a video frame codec architecture is shown, which includes a frame compressor-decompressor, a frame compressor-decompressor controller, and multiple cores.

[0019] Figure 3-1 An example implementation of a SoC with a video frame codec architecture including a frame compressor, a frame compressor controller, and multiple cores is shown.

[0020] Figure 3-2 An example implementation of a SoC with a video frame codec architecture including a frame decompressor, a frame decompressor controller, and multiple cores is shown.

[0021] Figure 4 An example frame compressor-decompressor controller is shown, which combines a frame decompressor and a core.

[0022] Figure 5 An example approach to implementing a video frame codec architecture is shown, in which multiple core components include a video decoder and a display controller.

[0023] Figure 6 An example technique for routing decompressed display frames from the video decoder to the display controller is shown.

[0024] Figure 7 An example scheme is shown where the frame decompressor controller manages requests for decompressed frames issued by multiple cores.

[0025] Figure 8 An example scheme is shown whereby a frame compressor-decompressor controller establishes a time-sharing protocol for sharing frame compression or decompression resources.

[0026] Figure 9 An example method for operating the video frame codec architecture described in this paper is shown.

[0027] Figure 10 Various components of an example electronic device that can implement a video frame codec architecture according to one or more embodiments are shown. Detailed Implementation

[0028] Overview

[0029] Movie or other video data consumes significant storage bandwidth when idle and also transmits bandwidth when propagating between electronic devices or their internal components. As video display resolution increases, so does bandwidth demand. For example, using Ultra High Definition (UHD) or 4K video requires processing approximately 15-20 times more video data than SD video, which was common a decade ago. This increase in data volume makes video management difficult, even within a single integrated circuit (IC) chip. For instance, a typical System-on-a-Chip (SoC) has a system bus that can become overloaded if raw decoded video data is used to transfer between different SoC components.

[0030] As is well known, video bandwidth can be reduced by encoding it using a lossy codec (such as H.264). The encoded video can then be streamed from a cloud server to a tablet computer, or stored on a Blu-ray disc or flash memory. The end-user's electronic device, for example, is then responsible for decoding the video to display it on a screen. Decoding requires converting the stream of binary 1s and 0s, or bits, into individual decoded video frames, which can be displayed sequentially to represent the video. As part of this video encoding and decoding process, the video data is compressed and decompressed. Using this video-level process (e.g., using the H.264 codec), the compression is significant, but it is also lossy. Therefore, due to the video-level compression / decompression process, some video information is lost to a degree that can significantly alter image quality.

[0031] This loss of video information is acceptable in order to enable the transfer of large amounts of video data between devices or to store it using a reasonable amount of storage. After the video stream is decoded at the electronic device to produce decoded video frames, this large amount of video data is reconstructed into a set of frames or many individual frames on a frame-by-frame basis. Each of these individual frames is still a considerable amount of data. In order to process this amount of video data at the electronic device, if the decoded video frames are not currently being processed or displayed, the decoded video frames are compressed to produce compressed video frames. Because a limited number of frames are decoded from the stream at any given time, lossless compression algorithms can be applied to the decoded frames in some implementations. The lossless compression and decompression frame-level process prevents any further image degradation. Compressed video frames occupy less memory space and consume less bandwidth on interconnects (such as the system bus of a SoC). Therefore, compressed video frames can be transferred between different chips on a printed circuit board, between the SoC and main memory, and even between different components of a single SoC.

[0032] To enable per-frame or frame-level compression in existing systems, each corresponding component or core of the SoC that processes video data includes a dedicated decompression unit. Using this intuitive approach, each core can independently generate decompressed video frames from compressed video frames and then process them according to the video-related purpose of that given core. However, this intuitive approach has several incurred costs. First, a considerable area on the IC die is dedicated to replicating the video frame decompression or compression circuitry. Second, the display path for presenting the video on the screen includes a separate decompression unit. Third, upgrading the video frame compression and decompression technology of the SoC is highly complex. This complexity stems from multiple distinct cores, each including either an individual decompression unit or an individual compression unit (in some cases, both). In other words, upgrading to a newer, more efficient decompression algorithm requires modifying each core, including either the compression or decompression unit, simultaneously and then re-integrating it with the rest of the SoC. Therefore, this complexity of the upgrade process slows the adoption of improved compression / decompression algorithms across multiple core chips.

[0033] Conversely, some implementations described herein use a shared frame compressor-decompressor. The frame compressor-decompressor may include a frame compressor or a frame decompressor, or both. Multiple cores of an IC chip (e.g., a SoC) can obtain decompressed versions of compressed frames from, for example, a shared frame decompressor. A given core can request a decompressed frame, and the frame decompressor can provide a response including the requested decompressed frame. Thus, the frame decompressor can act as a frame decompression service provider, and each core can act as a client of the frame decompression service. Similarly, a frame compressor can act as a frame compression service provider, and each core can act as a client of the frame compression service. These frame-level compression and decompression services can use lossless algorithms.

[0034] In some implementations, the SoC includes several different components, such as a video decoder and a display controller, which operate as distinct core components. The video decoder includes a stream decoder that decodes the video stream to produce decoded frames. The video decoder also includes a frame compressor that produces compressed frames from the decoded frames. These different components of the SoC are configured to process the decompressed frames. For example, the video decoder can use a decompressed reference frame to further decode the video stream. Additionally, the display controller can use the decompressed display frames to present the video on a display screen.

[0035] To obtain any of these example types of decompressed frames, the frame decompressor decompresses the corresponding compressed frame. The frame decompressor may route the requested decompressed frame to the requesting core using one or more buses, at least one buffer, or some other routing mechanism. The frame decompressor controller arbitrates shared access to the frame decompressor between at least two cores (e.g., a video decoder and a display controller). The frame decompressor controller may manage a request queue for compressed frame requests. Management may require prioritizing the service of frame requests according to a priority scheme. The frame decompressor controller may also establish a time-sharing protocol for accessing the frame decompressor. The time-sharing protocol may include allocating time slots to different cores, accepting interrupts from cores to gain control of the frame decompressor, and so on.

[0036] In these ways, frame compressor or frame decompressor resources can be shared between two or more client cores that process video data. By sharing, for example, the frame decompressor, the area of ​​the SoC dedicated to frame decompression is reduced. Furthermore, there is no need to insert a separate frame decompressor along the video display path. Additionally, the compression / decompression algorithm is decoupled from the overall SoC architecture. Therefore, the compression / decompression algorithm can be updated more easily and thus more frequently, because the SoC includes fewer or even one frame compressor or frame decompressor, and thus the update workflow is easier.

[0037] The following discussion, with reference to the accompanying drawings, outlines example implementations at different levels of detail. The discussion begins with a description of the example operating environment, followed by the example scheme and hardware, then the example method, and concludes with an example electronic device and related example aspects.

[0038] Example Environment

[0039] Figure 1 An example environment 100 including a printed circuit board 104 (PCB) is shown, in which a video frame codec architecture can be implemented. As shown, environment 100 includes electronic device 102. Electronic device 102 includes at least one PCB 104. PCB 104 includes one or more integrated circuits, such as integrated circuit 106 (IC). See below for reference. Figure 2 and Figure 3 The PCB 104 may include other integrated circuits, such as at least one memory separate from IC 106. IC 106 includes: at least one frame decompressor 108-2; multiple cores 110-1, 110-2…110-n, where n represents an integer greater than 1; and multiple video frames, such as at least one compressed frame 112 and at least one decompressed frame 114. For simplicity, Figure 1 Only the frame decompressor 108-2 is depicted, along with an example environment 100. However, references are made accordingly below. Figure 3 and Figure 3-1 Describe frame compressor-decompressor 108 and frame compressor 108-1.

[0040] In the example implementation, frame decompressor 108-2 is communicatively coupled to at least a portion of a plurality of cores 110-1 to 110-n. Frame decompressor 108-2 includes circuitry for decompressing compressed video frames. Thus, frame decompressor 108-2 can decompress compressed frame 112 to produce decompressed frame 114. In operation, core 110 uses frame decompressor 108-2 to obtain a decompressed version of the compressed frame. For example, the nth core 110-n sends a request 116 to frame decompressor 108-2 indicating that it is requesting a decompressed version of the identified compressed frame 112. Frame decompressor 108-2 provides a response 118 including the decompressed frame 114. Although video frames are referenced as an example context, frames described herein can include any frame having visual data, including computer-generated graphics frames, video frames, composite frames, etc.

[0041] In a similar manner, frame decompressor 108-2 provides decompression services to individual cores of multiple cores 110-1 to 110-n. Similarly, (for example, Figure 3-1 The frame compressor 108-1 can provide compression services to individual cores of multiple cores 110-1 to 110-n. Therefore, the frame decompressor 108-2 or the frame compressor 108-1 (or both) implements at least a portion of the frame compression-decompression (FCD) server circuitry 122 for IC 106. Similarly, multiple cores 110-1 to 110-n implement multi-frame compression-decompression (FCD) client circuitry 120. Using this client-server architecture, the frame compressor unit or frame decompressor unit can be shared among two or more cores 110 to save space on IC 106 and simplify the workflow of compression / decompression techniques used to compress / decompress video frames for upgrading IC 106. See below for reference. Figure 3 , Figure 3-1 and Figure 3-2 A more detailed example architecture describing IC 106 and PCB 104 is provided below. However, please refer to the following... Figure 2 Describe additional aspects of the example implementation.

[0042] Figure 2Other aspects of an example environment 200 in which a video frame codec architecture as described herein can be implemented are shown. Electronic device 102 is shown with various non-limiting example devices: smartphone 102-1, laptop computer 102-2, television 102-3, desktop computer 102-4, tablet computer 102-5, and wearable device 102-6. As shown on the right, electronic device 102 includes one or more processors 202, one or more computer-readable media 204, and at least one interconnect 216. The computer-readable media 204 may store, hold, or otherwise include code, data, instructions, other information, etc. Electronic device 102 may also include an operating system 212. Although described separately, the operating system 212 may be stored on one or more computer-readable media 204.

[0043] An application (not shown) or operating system 212, implemented as computer-readable instructions on computer-readable medium 204, may be executed by processor 202. Operating system 212 or basic input / output system (BIOS) may include frame codec parameter module 214. Frame codec parameter module 214 may set one or more parameters to enable, authorize, tune, or otherwise facilitate the execution of the shared frame compression and decompression functions described herein.

[0044] As shown, computer-readable medium 204 may include at least one video buffer 206, at least one shared local cache 208, and at least one main memory 210. In some embodiments, the video buffer 206 and the shared local cache 208 are separate memory blocks on the IC. In other embodiments, the video buffer 206 and the shared local cache 208 are part of the same memory block, for example, if a portion of the shared local cache 208 is used as the video buffer 206 in a dynamically changing or fixed allocation scheme. Interconnect 216 may include at least one system bus 218, at least one video bus 220, and at least one external bus 222. In some embodiments, the video bus 220 and the system bus 218 are different buses. In other embodiments, there is no separate video bus, so video data propagates around the IC using the system bus 218. References are made below. Figures 4 to 6 Example implementations of these computer-readable media 204 and these interconnections 216 are described.

[0045] Example components and technologies

[0046] Figure 3A portion of PCB 104 is shown. PCB 104 includes a System-on-Chip (SoC) 302 and main memory 210. SoC 302 depicts an example implementation of a video frame codec architecture including a frame compressor-decompressor 108 (FCD), a frame compressor-decompressor controller 304, and multiple cores 110-1 to 110-n. SoC 302 also includes a shared local cache 208 and a system bus 218. Main memory 210 is coupled to SoC 302 via an external bus 222, which is included as part of PCB 104 (e.g., disposed on PCB 104).

[0047] PCB 104 can be implemented using rigid or flexible materials and is used to mount or secure multiple IC chips, interconnects, interfaces, etc. Main memory 210 can be implemented using, for example, dynamic random access memory (DRAM) that is periodically refreshed to maintain its contents, or flash memory that can maintain its contents without power. Typically, accessing data stored in main memory 210 consumes more energy than accessing data stored in SoC 302 (e.g., shared local cache 208). Shared local cache 208 can be implemented using, for example, static random access memory (SRAM), DRAM, flash memory, or some combination thereof.

[0048] System bus 218 interconnects multiple cores 110-1 to 110-n, a shared local cache 208, a frame compressor-decompressor controller 304, and other components and interfaces (e.g., frame compressor-decompressor 108 may be directly coupled to system bus 218). Each core 110 of the multiple cores 110-1 to 110-n can use system bus 218 to store data at or retrieve data from shared local cache 208. Similarly, each core 110 of the multiple cores 110-1 to 110-n can use external bus 222, for example, also by using system bus 218 or shared local cache 208, to store data at or retrieve data from main memory 210. For example, a first core 110-1 can store data in shared local cache 208, and then a second core 110-2 can retrieve the stored data from shared local cache 208.

[0049] In the example operation, the frame compressor-decompressor 108 (FCD) processes (e.g., compresses or decompresses) multiple unprocessed frames 306 to produce multiple processed frames 308. A frame compressor-decompressor controller 304 (FCD controller) is coupled to the frame compressor-decompressor 108 and arbitrates access to the frame compressor-decompressor 108 for multiple cores 110-1 to 110-n. Although shown separately, the frame compressor-decompressor 108 and the frame compressor-decompressor controller 304 can be logically integrated together. At least two cores 110 of the multiple cores 110-1 to 110-n are coupled to the frame compressor-decompressor controller 304. Therefore, each core 110 can obtain the processed frames 308 produced by the frame compressor-decompressor 108 from its unprocessed frame version via the frame compressor-decompressor controller 304. The processed frames 308 can be obtained using, for example, a request 116 and a corresponding response 118.

[0050] Frame compressor-decompressor 108 may include frame compressor 308-1 (e.g., as...) Figure 3-1 (as depicted), frame decompressor 108-2 (e.g., as described) Figure 3-2 (as shown) or both. Similarly, the frame compressor-decompressor controller 304 may include a frame compressor controller 304-1 (e.g., as shown). Figure 3-1 (as shown), frame decompressor controller 304-2 (e.g., as shown) Figure 3-2 (as shown) or both. Depending on whether the processing operation of the frame compressor-decompressor 108 is compression or decompression, one of the unprocessed frame 306 or the processed frame 308 corresponds to a compressed frame, and the other corresponds to a decompressed frame. If a given implementation includes frame compressor 108-1 and frame decompressor 108-2 (and / or both frame compressor controller 304-1 and frame decompressor controller 304-2), these components can be located in the same location, adjacent to each other, or in different locations on the IC. For example, each can be located closer to the core, which is likely the most common client of the corresponding compression or decompression service. Furthermore, if the chip includes frame compressor 108-1 and frame decompressor 108-2 located in the same location, each can include completely separate circuitry or they can share circuitry. In some implementations, frame compressor-decompressor 108 can be implemented as a lossless frame data manipulator that adjusts the storage size of the data used for the frame as part of a lossless compression or lossless decompression operation upon request 116. References are made accordingly below. Figure 3-1 and Figure 3-2 Example implementations of compression and decompression operations are described.

[0051] Figure 3-1A SoC with an example implementation of a video frame codec architecture is shown, comprising a frame compressor 108-1 (FC), a frame compressor controller 304-1 (FC controller), and multiple cores. The frame compressor 108-1 compresses uncompressed frames (e.g., decompressed frame 114 or an uncompressed frame) to produce a compressed frame 112. The frame compressor controller 304-1 is coupled to the frame compressor 108-1 and arbitrates access to the frame compressor 108-1 for the multiple cores 110-1 to 110-n. Although shown separately, the frame compressor 108-1 and the frame compressor controller 304-1 can be logically integrated.

[0052] At least two cores 110 from 110-1 to 110-n are coupled to frame compressor controller 304-1. Each core 110 may include components or blocks that generate visual data. Thus, each core 110 can obtain a compressed frame 112 generated by frame compressor 108-1 from its decompressed version of frame 114 via frame compressor controller 304-1. Therefore, compressed frame 112 can be obtained using, for example, a request 116 and a corresponding response 118. In operation, frame compressor controller 304-1 may pipeline access to the compression engine of frame compressor 108-1 to some requesting cores 110 on system bus 218 to avoid increasing traffic to external bus 222, which provides access to main memory 210. In some embodiments, frame compressor controller 304-1 may temporarily grant exclusive access to requesting cores 110 to access compression resources.

[0053] Figure 3-2 A SoC with an example implementation of a video frame codec architecture is shown, comprising a frame decompressor 108-2 (FD), a frame decompressor controller 304-2 (FD controller), and multiple cores. The frame decompressor controller 304-2 is coupled to the frame decompressor 108-2 and arbitrates access to the frame decompressor 108-2 for the multiple cores 110-1 to 110-n. Although shown separately, the frame decompressor 108-2 and the frame decompressor controller 304-2 can be logically integrated. At least two cores 110 of the multiple cores 110-1 to 110-n are coupled to the frame decompressor controller 304-2. Thus, each core 110 can obtain, via the frame decompressor controller 304-2, a decompressed frame 114 generated by the frame decompressor 108-2 from its compressed frame 112 version. Therefore, the decompressed frame 114 can be obtained using, for example, a request 116 and a corresponding response 118.

[0054] The following description focuses on embodiments capable of performing at least a decompression operation. Therefore, each may explicitly include a frame decompressor 108-2 and a frame compressor-decompressor controller 304 or a frame decompressor controller 304-2. However, each embodiment may additionally or alternatively include a frame compressor 108-2 for performing a compression operation. Furthermore, how the compression operation and the movement of the compressed frame 112 can be at least similar to the reference... Figures 4 to 6 The method for describing the decompression operation is self-evident. Furthermore, refer to... Figure 7 Description of request processing and reference Figure 8 The time-sharing protocol described can also be used for compression operations in a similar manner.

[0055] At least some of the cores in Core 110 can manipulate video data for video-related purposes. Video-related purposes include: video decoding, video display, video transcoding (e.g., decoding from one encoding format using one algorithm and encoding to another encoding format using another algorithm), video compensation (e.g., a graphics processing unit (GPU) manipulating video frames to improve or enhance an image), and so on. [Reference] Figure 5 The following describes an example implementation, in which the first core is implemented as a video decoder and the second core is implemented as a display controller. However, first refer to Figure 4 An example implementation of the frame decompressor controller 304 is described.

[0056] Figure 4 An example frame compressor-decompressor controller 304, in combination with frame decompressors 108-2 and core 110, is illustrated at 400°. Core 110 includes an FCD client interface 410, and frame decompressor controller 304 includes an FCD service interface 408. The FCD client interface 410 and the FCD service interface 408 together enable communication between core 110 and frame compressor-decompressor controller 304. Thus, using these two interfaces, core 110 can send a request 116 to frame compressor-decompressor controller 304, and frame compressor-decompressor controller 304 can send a response 118 to core 110.

[0057] The FCD client interface 410 and the FCD service interface 408 can communicate via the system bus 218 (not shown), video bus 218, etc. Request 116 can be transmitted using bit lines, one or more bits of a register, messages formulated as at least one packet, etc. Response 118, including the decompressed frame 114, can be transmitted using a bus coupling the core 110 to the frame decompressor 108-2, memory accessible by both the core 110 and the frame decompressor 108-2, or some combination thereof. See below for further details. Figure 5 and Figure 6Examples describing communication between different interfaces.

[0058] As shown in the figure, the frame compressor-decompressor controller 304 includes a request manager 402, a request queue 404, a priority manager 406, and a time-sharing protocol handler 412. The FCD service interface 408 is coupled to at least the request manager 402, the priority manager 406, and the time-sharing protocol handler 412. See below for reference. Figure 7 Describe an example scheme for managing requests, including prioritizing the services requested. See below for reference. Figure 8 This describes an example scheme for using the time-sharing protocol processor 412 to establish a time-sharing protocol to access the frame decompressor 108-2.

[0059] Figure 5 An example implementation of a video frame codec architecture 500 is shown, in which multiple cores of the SoC include a video decoder 502 and a display controller 504. On the left, the video decoder 502 includes a stream decoder 506, which includes a frame compressor 108-1. However, the frame compressor 108-1 may be implemented separately from either the stream decoder 506 or the video decoder 502. On the right, the display controller 504 includes a frame buffer 518. Frame decompressors 108-2, a frame compressor-decompressor controller 304 (FCD controller), and a video buffer 206 are associated with the video decoder 502. For example, these three components may be located in the same location as the video decoder 502 (e.g., placed next to or physically integrated with the video decoder 502), may be logically integrated with the video decoder 502 at least partially, or some combination thereof. Although described with reference to the video decoder 502 and the display controller 504, these principles generally apply to core 110. Furthermore, although the video buffer 206 is described separately from the shared local cache 208, the video buffer 206 may also be incorporated into or allocated from the shared local cache 208 as part of the shared local cache 208.

[0060] In an example implementation, an electronic device receives a video stream 514, which may include a bitstream encoded to represent video. The video stream 514 consists of a plurality of raw frames 516 encoded using some codec such as H.264. A stream decoder 506 receives the video stream 514 via a system bus 218 and decodes it to produce a plurality of decoded frames 512, one of which is explicitly shown. A frame compressor 108-1 compresses the plurality of decoded frames 512 to produce a plurality of compressed frames 112. As shown, the frame compressor 108-1 is integrated with the stream decoder 506. Therefore, the frame compressor 108-1 can compress the decoded frames 512 as part of the decoding process; however, frame compression can be implemented in an alternative manner. These decoded and compressed frames 112 can be stored in a video buffer 206, a shared local cache 208, or main memory 210 for subsequent use by the video decoder 502 or another core.

[0061] The decoded and compressed frame 112 can be used for different video-related purposes. For example, if the compressed frame is to be used for further video decoding purposes, the compressed frame 112 may include a compressed reference frame 112R. Alternatively, if another compressed frame is used for video display purposes, the other compressed frame 112 may include a compressed display frame 112D. For the video decoding function of the video decoder 502, the frame decompressor 108-2 decompresses the compressed reference frame 112R to produce a decompressed reference frame 114R. The video decoder 502 can use the decompressed reference frame 114 to continue decoding the video stream 514.

[0062] For the video display function of the display controller 504, the frame decompressor 108-2 decompresses the compressed display frame 112D to generate the decompressed display frame 114D. The display controller 504 can be configured via the frame decompressor controller client interface 410 and the frame decompressor controller service interface 408. Figure 4 The two) from the frame compressor-decompressor controller 304 (e.g., which includes at least Figure 3-2 The frame compressor-decompressor controller 304-2 requests the decompression service. The frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via routing mechanism 510 (indicated by the solid wavy arrow). The display controller 504 can store the decompressed display frame 114D in the frame buffer 518. The display controller 504 can then use the stored decompressed display frame 114D to render the image of the video frame on the display screen. Although... Figure 5The frame compressor-decompressor controller 304 is described based on providing decompression services to multiple cores using frame decompressor 108-2, but these principles also apply to the frame compressor-decompressor controller 304 which provides compression services to multiple cores using frame compressor 108-1.

[0063] Figure 5 Several example routing mechanisms 510 are indicated by small dashed lines. Routing mechanisms 510 represent different paths by which the frame compressor-decompressor controller 304 provides the decompressed frame 114 to a requesting core (e.g., video decoder 502 or display controller 504). In a first routing mechanism 510-1, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the frame buffer 518 of the display controller 504 via the system bus 218. In a second routing mechanism 510-2, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and a shared local cache 208. For example, the frame compressor-decompressor controller 304 may store the decompressed display frame 114D in the shared local cache 208, and the display controller 504 may retrieve the stored decompressed display frame 114D from the shared local cache 208.

[0064] In the third routing mechanism 510-3, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and main memory 210. For example, the frame compressor-decompressor controller 304 can store the decompressed display frame 114D off-chip in main memory 210 via the system bus 218 and external bus 222. The display controller 504 can retrieve the stored decompressed display frame 114D from main memory 210 using both buses and then store the frame in frame buffer 518.

[0065] The fourth routing mechanism 510-4 involves serving the video decoder 502 as a client for frame decompression services. In the fourth routing mechanism 510-4, the frame compressor-decompressor controller 304 provides the video decoder 502 with a decompressed reference frame 114R. To do this, the frame compressor-decompressor controller 304 transfers the decompressed reference frame 114R to the video buffer 206. The video decoder 502 can then access the decompressed reference frame 114R from the video buffer 206.

[0066] In the fifth routing mechanism 510-5, if the integrated circuit includes a video bus 220 and a system bus 218, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video bus 220, which is at least partially separated from the system bus 218. As shown, the video bus 220 is dedicated to transmitting video data and extends at least between the frame compressor-decompressor controller 304 and the display controller 504. However, the video bus 220 can be implemented differently, for example, by transmitting other video data to and from other cores.

[0067] In the sixth routing mechanism 510-6, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video buffer 206. The frame compressor-decompressor controller 304 stores at least a portion of the decompressed display frame 114D in the video buffer 206. In one approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the system bus 218. In another approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the video bus 220 (e.g., if it exists separately from the system bus 218). Regardless of the approach used, the sixth routing mechanism 510-6 may further require the display controller 504 to access the memory mapping mechanism of the video buffer 206. (Refer to the following...) Figure 6 An example implementation of the memory mapping mechanism is described.

[0068] Figure 6 An example technique 600 is shown for routing a decompressed display frame 114D from a video decoder 502 to a display controller 504 using a sixth routing mechanism 510-6. In this example, the video decoder 502 is integrated with a frame decompressor 108-2 and a frame compressor-decompressor controller 304. In addition to a video buffer 206, the video decoder 502 also includes a synchronization register 602. In this example, the video buffer 206 is used as an output buffer. Therefore, the frame compressor-decompressor controller 304 causes the frame decompressor 108-2 to store at least a portion of the decompressed display frame 114D in the video buffer 206.

[0069] The example technique 600 of the sixth routing mechanism 510-6 involves at least one state communication 606. A first state communication 606-1 occurs between the frame compressor-decompressor controller 304 and the synchronization register 602. A second state communication 606-2 occurs between the display controller 504 and the synchronization register 602. Each state communication 606 may include a polling operation or an update operation and can be performed via the system bus 218 or the video bus 220.

[0070] In response to storing at least a portion of the decompressed display frame 114D in the video buffer 206, the frame compressor-decompressor controller 304 writes value 604 into the synchronization register 602 as part of the update operation of the first state communication 606-1. This value 604 indicates the storage state of the contents of the video buffer 206. Here, value 604 indicates that specific video data (e.g., one or more display lines 608 of the decompressed display frame 114D) has been newly placed in the video buffer 206. Therefore, the video buffer 206 can be used as a line buffer to hold one or more display lines 608.

[0071] Simultaneously, the display controller 504 extracts value 604 from the synchronization register 602 as part of the polling operation of the second state communication 606-2. Based on value 604, the display controller 504 determines that one or more new display lines 608 of the requested decompressed display frame 114D exist and are available in the video buffer 206. The display controller 504 then retrieves at least one display line 608 from the video buffer 206 via the system bus 218 or the video bus 220 according to value 604. In response to the retrieval, the display controller 504 writes another value 604 into the synchronization register 602 as part of the update operation of the second state communication 606-2 to indicate the retrieval status of the corresponding display line 608. This other value 604 thus indicates that at least one corresponding display line 608 has been read from the video buffer 206.

[0072] Periodically, or as part of certain operations, the frame compressor-decompressor controller 304 polls the synchronization register 602 as part of the first state communication 606-1. The frame compressor-decompressor controller 304 checks another value 604 to determine whether the corresponding display line 608 has been accessed and retrieved by the display controller 504. If so, the frame compressor-decompressor controller 304 knows that the storage location of the corresponding display line 608 can be rewritten with new data.

[0073] Value 604 or another value 604 may include, for example, one or more bits. Value 604 and another value 604 may correspond to different or the same storage location in the synchronization register 602. Although described with reference to the video decoder 502 and the display controller 504, these principles generally apply to core 110. By polling and updating value 604 in the synchronization register 602, a memory-mapped transfer mechanism can be used to control the routing of the decompressed display frame 114D.

[0074] Figure 7 An example scheme 700 is shown, in which a frame compressor-decompressor controller 304 manages requests 116 issued by multiple cores for decompressed frames. In this example, a video decoder 502 is integrated with a frame decompressor 108-2 and a frame compressor-decompressor controller 304. The frame compressor-decompressor controller 304 consists of a request manager 402, a request queue 404, and a priority manager 406 (also...). Figure 4 (As shown in the figure). As shown, request queue 404 includes multiple requests 116. Each request 116 corresponds to a priority 706. Each request 116 also includes or corresponds to a requester identifier 702 and a frame identifier 704.

[0075] Some requests for decompressed frames may be more important or time-dependent than others. For example, for an end user, a request related to the real-time rendering of streaming video may be more important than a request for video transcoding operations that can be performed in the background at best-efforts rate. Therefore, requests for video-related purposes in the former can take precedence over requests for video-related purposes in the latter. For this reason, priority 706 can be associated with requests, cores, frames, etc.

[0076] In the example implementation, requester identifier 702 identifies request core 110, for example ( Figure 5 and 6 The video decoder 502 or display controller 504. Frame identifier 704 identifies the requested decompressed frame 114, for example, by frame number, time index, or offset thereof. Requester identifier 702 may be implemented using specific voltage values ​​on one or more bit lines, based on the origin or bus used to provide request 116 (e.g., if a dedicated bus is provided for each core accordingly), with an assigned identifier value transferred along with request 116. Frame identifier 704 may be explicitly included in each request 116, may be implicitly included in request 116 (e.g., if the next consecutive frame is required), may be included in a previous request as part of a frame range, etc.

[0077] In operation, request manager 402 receives request 116 from core 110, for example, via frame decompressor controller service interface 408. Request manager 402 inserts request 116 into request queue 404. In one approach, request manager 402 operates request queue 404 in a first-in, first-out (FIFO) manner. Alternatively, request manager 402 may insert each request 116 into request queue 404 based on a corresponding priority 706. The corresponding priority 706 for each individual request 116 may be included or may accompany the corresponding individual request 116. Alternatively, priority 706 may be explicitly batch-assigned to a group of requests 116, or each core 110 may be associated with a specific priority 706, which is implicitly assigned to each request 116 issued by the associated core 110.

[0078] As shown, each request 116 is stored in request queue 404 associated with a corresponding priority 706, such that requests 116 with different priorities 706 are stored in the same request queue 404. Alternatively, the frame compressor-decompressor controller 304 may implement the illustrated request queue 404 using multiple request queues 404, each request queue 404 associated with a corresponding priority 706. Thus, each request 116 is stored in the corresponding request queue 404 associated with the corresponding priority 706 of that request 116.

[0079] Regardless of the method used to implement the request queue 404 shown, the priority manager 406 prioritizes the services of the multiple requests 116 according to multiple priorities 706 corresponding to the multiple requests 116. Therefore, the priority manager 406 uses a priority scheme based on the relative priorities between two or more requests 116 to retrieve the next request 116. The priority manager 406 forwards the next request 116 to the frame decompressor 108-2. The frame decompressor 108-2 retrieves the compressed frame 112 corresponding to the requested decompressed frame 114. Continuing... Figure 5 and 6 In an example scenario, frame decompressor 108-2 retrieves the compressed display frame 112D and performs a decompression operation to produce a decompressed display frame 114D. Frame compressor-decompressor controller interface 408 enables frame decompressor 108-2 to route the decompressed display frame 114D to video decoder 502 as part of response 118.

[0080] Figure 8 The frame compressor-decompressor controller 304 is shown establishing a mechanism for sharing frame compression resources (e.g., frame compressor 108-1, e.g., Figure 3-1 Or frame decompression resources (e.g., frame decompressor 108-2 (e.g., Figure 3-2At least one example scheme of time-sharing protocol 800 is described. The time-sharing protocol handler 412 of the frame compressor-decompressor controller 304 (not explicitly shown) can establish at least one example time-sharing protocol 800-1, 800-2, or 800-3. As indicated by arrow 812, time increases in the rightward direction. Each example time-sharing protocol 800 is described based on sharing frame decompression resources between two cores; however, the disclosed time-sharing protocol 800 can be applied to sharing frame decompression resources between more than two cores. Although Figure 8 It is based on the time-division description of frame decompression, but these principles also apply to time-division protocols that implement frame compression.

[0081] Using the first time-sharing protocol 800-1, time is divided into multiple time slots. These time slots have fixed durations, and each corresponding time slot is allocated to a corresponding core. Time slot 802-1 corresponds to the first core, such as video decoder 502. Time slot 802-2 corresponds to the second core, such as display controller 504. In operation, the time-sharing protocol processor 412 rotates access to frame decompression resources between the two cores over a specified length of two different time slots. In the first time-sharing protocol 800-1, interrupts are not enabled.

[0082] Conversely, interrupt 806 is enabled for the second time-sharing protocol 800-2. Here, frame decompression resources are typically reserved for one of the two clients (the first core in this case). Therefore, time slots 804-1 correspond to the first core and can extend indefinitely. These time slots 804-1 extend until the time-sharing protocol processor 412 receives interrupt 806. Core 110 can signal interrupt 806 via a message on the system bus, using a dedicated video bus, or by driving the interrupt line high. In response to detecting interrupt 806, the time-sharing protocol processor 412 initiates a context switch to transfer access to the frame decompression resources to a different core and begins another time slot.

[0083] As shown in the second time-sharing protocol 800-2, the second core issues an interrupt 806a. The time-sharing protocol processor 412 detects the interrupt 806a and transfers access to the frame decompressor 108-2 to the second core that initiated the interrupt. Therefore, the frame decompressor 108-2 performs at least one decompression operation to generate a decompressed frame 114 for the requesting second core during time slot 804-2 following the interrupt 806a. The length of each time slot 804-2 can be fixed, depending on the number of requests queued for the second core, the priority of each queued request, etc. After the first core is re-authorized to access the frame decompression resources for another time slot 804-1, the second core can subsequently issue another interrupt 806b. Alternatively, each time slot 804-2 allocated to the second core can continue until another core issues an interrupt (not shown). Therefore, in this alternative, a context switch can occur in response to interrupt 806 and is independent of elapsed time.

[0084] Using the third time-sharing protocol 800-3, time is divided into multiple time slots. The third time-sharing protocol 800-3 is a hybrid of the first and second time-sharing protocols. Some time slots, such as time slot 808-1, have fixed, uninterruptible durations. Other time slots, such as time slot 810-1, have variable durations that can be interrupted by another core. As shown, the first core is allocated to time slots 808-1 and 810-1. An interrupt 806c from the second core arrives during time slot 808-1. However, the time-sharing protocol handler 412 ignores (or queues) this interrupt 806c. On the other hand, an interrupt 806d arriving during time slot 810-1 triggers a context switch. Using the context switch, after interrupt 806d, frame decompression resources are allocated to the second core's persistent time slot 804-2. If interrupt 806c is queued (instead of being dequeued) during time slot 808-1, then in response to interrupt 806c, a context switch can be initialized at the end of time slot 808-1 or at the beginning of time slot 810-1.

[0085] Having already described in general the technologies and environments in which video frame codec architectures can be implemented, the discussion now turns to example methods.

[0086] Example Method

[0087] The following discussion describes methods for operating video frame codec architectures. These methods can be implemented using the examples described above, such as... Figure 1 and Figure 3 The electronic device 102 and PCB 104 shown, and Figure 10 The described electronic device 1000. Aspects of these methods are... Figure 9The diagram illustrates operations 902-912 performed by one or more components of at least one IC chip. The order in which these methods are shown and / or described is not intended to be construed as limiting, and any number or combination of the described method operations can be combined in any order to implement the methods or alternative methods. Although Figure 9 The flowchart is described based on the operation frame decompressor 108-2, but these principles can be similarly applied to the operation frame compressor 108-1.

[0088] Figure 9 An example method 900 for operating a video frame codec architecture is shown, which can be implemented by circuitry providing frame decompression services to multiple different cores. At 902, a first request for a first decompressed frame is received from a first core. For example, a frame compressor-decompressor controller 304 (e.g., its frame decompressor controller 304-2) can receive a first request 116 for a first decompressed frame 114 from the first core 110-1. Acceptance can be performed via system bus 218, video bus 220, synchronization register 602, etc.

[0089] At 904, the first compressed frame is decompressed to produce a first decompressed frame. For example, frame decompressor 108-2 can decompress the first compressed frame 112 to produce a first decompressed frame 114. The service of the first request 116 can be performed according to the associated priority 706.

[0090] At 906, in response to the first request, the first decompressed frame is provided to the first core. For example, the frame compressor-decompressor controller 304 may provide the first decompressed frame 114 to the first core 110-1 in response to the first request 116. The first decompressed frame 114 may be transferred from the frame decompressor 108-2 to the first core 110-1 using the routing mechanism 500.

[0091] At 908, a second request for a second decompressed frame is received from the second core. For example, the frame compressor-decompressor controller 304 may receive a second request 116 for a second decompressed frame 114 from the second core 110-2. The first core 110-1 may include a display controller 504, and the second core 110-2 may include a video decoder 502.

[0092] At 910, the second compressed frame is decompressed to produce a second decompressed frame. For example, frame decompressor 108-2 can decompress the second compressed frame 112 to produce a second decompressed frame 114. Decompression can be performed using a lossless compression / decompression algorithm. At 912, in response to a second request, the second decompressed frame is provided to the second core. For example, frame compressor-decompressor controller 304 can provide the second decompressed frame 114 to the second core 110-2 in response to a second request 116. After obtaining the second decompressed frame 114, the second core 110-2 can process the frame according to the corresponding video-related functions.

[0093] The preceding discussion described methods related to video frame codec architecture. Aspects of these methods can be implemented, for example, in hardware (e.g., fixed logic circuitry), firmware, or some combination thereof. These techniques can be used... Figures 1 to 8 and Figure 10 One or more entities or components shown below (electronic device 1000) Figure 10 These entities or components (described in the figures) can be further divided, combined, etc. Therefore, these figures illustrate some of the many possible systems or devices capable of employing the described technology. The entities and components in these figures typically represent firmware, hardware, IC chips, circuits, or combinations thereof.

[0094] Example electronic devices

[0095] Figure 10 Various components of the example electronic device 1000 are shown. The example electronic device 1000 can be configured according to previous references. Figures 1 to 9 Any one or more embodiments described herein are used to implement a video frame codec architecture. Electronic device 1000 can be implemented as any one or a combination of fixed or mobile devices, and may be a consumer, computer, portable, user, server, communication, telephone, navigation, gaming, audio, camera, messaging, media playback, and / or other type of electronic device 102 (e.g., ...). Figure 1 Any form of smartphone shown.

[0096] Electronic device 1000 includes a communication transceiver 1002 that allows wired and / or wireless communication of device data 1004, such as receiving data, transmitting data, or other information as described above. Example communication transceivers 1002 include NFC transceivers, compliant with various IEEE 802.15 (Bluetooth) standards, etc. TM WPAN radio conforming to various IEEE 802.11 (WiFi) standards TMWLAN radios of any of the standards in the IEEE 802.16 standard, WWAN (3GPP compliant) radios for cellular phones, and various IEEE 802.16 (WiMAX) compliant standards. TM Standard wireless metropolitan area network (WMAN) radios and wired local area network (LAN) Ethernet transceivers.

[0097] The electronic device 1000 may also include one or more data input ports 1006 through which any type of data, media content, and / or other input can be received, such as user-selectable input, messages, applications, music, television content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source. The data input port 1006 may include a USB port, a coaxial cable port, and other serial or parallel connectors (including internal connectors) for flash memory, DVDs, CDs, etc. These data input ports 1006 can be used to couple the electronic device to components, peripherals, or accessories, such as a keyboard, microphone, or camera.

[0098] The example electronic device 1000 includes at least one processor 1008 (e.g., any one or more of an application processor, microprocessor, digital signal processor (DSP), controller, etc.), which may include a combined processor and memory system (e.g., implemented as part of a SoC) that processes (e.g., executes) computer-executable instructions stored on a computer-readable medium to control the operation of the device. The processor 1008 may be implemented as an application processor, embedded controller, microcontroller, SoC, etc. Typically, the processor or processing system may be implemented at least partially in hardware, which may include components of integrated circuits or systems-on-a-chip, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon and / or other hardware.

[0099] Alternatively or additionally, the electronic device 1000 may be implemented using any one or a combination of electronic circuits, which may include software, hardware, firmware, or fixed logic circuits, combined with processing and control circuits, generally represented by 1010 (as electronic circuit 1010). This electronic circuit 1010 may be implemented as an executable or hardware-based module (not shown) using logic circuits and / or hardware (e.g., such as an FPGA).

[0100] Although not shown, electronic device 1000 may include system buses, interconnects, crossbars, or data transfer systems that couple various components within the device. The system bus or interconnect may include any one or a combination of different bus architectures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures.

[0101] Electronic device 1000 also includes one or more storage devices 1012 that enable data storage, examples of which include random access memory (RAM), non-volatile memory (e.g., read-only memory (ROM), flash memory, EPROM, and EEPROM), and disk storage devices. The storage devices 1012 provide data storage mechanisms to store device data 1004, other types of code and / or data, and various device applications 1020 (e.g., software applications or programs). For example, operating system 1014 may be stored as software instructions in the storage device 1012 and executed by processor 1008.

[0102] As shown in the figure, the electronic device 1000 also includes an audio and / or video processing system 1016 that processes audio data and / or transmits audio and video data to an audio system 1018 and / or a display system 1022 (e.g., a video buffer or the screen of a smartphone or camera). The audio system 1018 and / or display system 1022 may include any device that processes, displays, and / or otherwise presents audio, video, display, and / or image data. Display data and audio signals may be transmitted to the audio and / or display components via an RF (radio frequency) link, an S-video link, an HDMI (High Definition Multimedia Interface), a composite video link, a component video link, a DVI (Digital Video Interface), an analog audio connection, or other similar communication links such as a media data port 1024. In some embodiments, the audio system 1018 and / or display system 1022 are external or separate components of the electronic device 1000. Alternatively, the display system 1022 may be an integrated component of the example electronic device 1000, such as part of an integrated touch interface.

[0103] Figure 10 Electronic device 1000 is Figure 1 An example implementation of the electronic device 102. Therefore, the processor 1008 is (for example, correspondingly for) Figure 1 , Figure 2 and Figure 3 Examples of integrated circuits 106, processors 202, or SoCs 302. Therefore, as... Figure 10As shown, the processor 1008 may include a frame compressor-decompressor 108, multiple cores 110-1 to 110-n, and a frame compressor-decompressor controller 304. Figure 10 In this context, the video buffer 206, the shared local cache 208, and the main memory 210 (e.g., also...) Figure 2 , Figure 3 , Figure 5 or Figure 6 Each of these memory components is described as forming at least a part of a storage system, for example, by being part of one or more storage devices 1012. As described above, one or more of these memory components may be part of the same integrated circuit as the integrated circuit of the processor 1008 (e.g., the video buffer 206 and the shared local cache 208 are...). Figure 3 This is part of the SoC 302, or alternatively located on a separate die, or even situated on a different printed circuit board. Therefore, the principles of the video frame codec architecture described herein can be derived from... Figure 10 Implemented in 1000 electronic devices, or in combination with Figure 10 The electronic equipment 1000 is implemented.

[0104] Although implementations of the video frame codec architecture have been described in language specific to certain features and / or methods, the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary implementations of the video frame codec architecture.

Claims

1. An electronic device, comprising: a frame decompressor configured to individually decompress a plurality of compressed frames at a frame level to produce a plurality of decompressed frames, the plurality of compressed frames respectively originating from a plurality of decoded frames; a frame decompressor controller coupled to the frame decompressor and configured to arbitrate access to the frame decompressor for a plurality of cores; a first core of the plurality of cores coupled to the frame decompressor controller, the first core configured to obtain, via the frame decompressor controller, a decompressed frame of the plurality of decompressed frames produced by the frame decompressor at a frame level; and a second core of the plurality of cores coupled to the frame decompressor controller, the second core configured to obtain, via the frame decompressor controller, another decompressed frame of the plurality of decompressed frames produced by the frame decompressor at a frame level, the first core and the second core configured to perform video transcoding.

2. The electronic device of claim 1, wherein: the first core is configured to decode a video stream to produce a plurality of decoded frames, at least one decoded frame of the plurality of decoded frames being produced using the decompressed frame; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using the other decompressed frame for the video transcoding.

3. The electronic device of claim 2, wherein: the first core includes a video decoder, the video decoder including a stream decoder configured to decode the video stream to produce the plurality of decoded frames; the frame decompressor is logically integrated with the video decoder; and the second core includes an encoder configured to encode the plurality of decoded frames to at least partially perform the video transcoding.

4. The electronic device of claim 1, wherein, the frame decompressor controller includes: a request queue configured to store a plurality of requests, each request corresponding to at least one decompressed frame; and a request manager configured to receive a plurality of requests from the plurality of cores and insert the plurality of requests into the request queue.

5. The electronic device of claim 4, wherein, the frame decompressor controller includes a priority manager configured to order servicing of the plurality of requests according to a plurality of priorities corresponding to the plurality of requests.

6. The electronic device of claim 1, wherein, the frame decompressor controller includes a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame decompressor among the plurality of cores.

7. The electronic device of claim 6, wherein, the time-sharing protocol handler is configured to at least one of: establish the time-sharing protocol to include a plurality of time slots allocated to at least one core of the plurality of cores; or establish the time-sharing protocol to allow receiving an interrupt signal from one core of the plurality of cores, the interrupt signal indicating a request for at least one decompressed frame.

8. The electronic device of claim 1, further comprising: a frame compressor configured to individually compress one or more uncompressed frames at a frame level to produce one or more compressed frames; and a frame compressor controller coupled to the frame compressor and configured to arbitrate access to the frame compressor for the plurality of cores.

9. The electronic device of claim 8, the frame compressor configured to individually compress the plurality of decoded frames at a frame level to produce the plurality of compressed frames. 10.The electronic device of claim 1, wherein, the frame decompressor configured to implement a lossless frame-level compression algorithm to correspondingly produce the plurality of decompressed frames from the plurality of compressed frames.

11. A method for sharing frame decompression circuitry among a plurality of cores, the method comprising: compressing a first decoded frame at a frame level to produce a first compressed frame; compressing a second decoded frame at the frame level to produce a second compressed frame; accepting a first request for a first decompressed frame from a first core; individually decompressing the first compressed frame at a frame level to produce the first decompressed frame; in response to the first request, providing the first decompressed frame to the first core; accepting a second request for a second decompressed frame from a second core; individually decompressing the second compressed frame at a frame level to produce the second decompressed frame; in response to the second request, providing the second decompressed frame to the second core; and performing video transcoding by the first core and the second core.

12. The method of claim 11, wherein the performing comprises: decoding, by the first core, a video stream to produce a plurality of decoded frames including the first decoded frame and the second decoded frame; and encoding, by the second core, the plurality of decoded frames to produce another video stream.

13. The method of claim 12, wherein: the decoding comprises decoding, by the first core, the video stream to produce at least one of the plurality of decoded frames using the first decompressed frame; and the encoding comprises encoding, by the second core, one or more of the plurality of decoded frames to produce the another video stream using the second decompressed frame.

14. The method of claim 11, further comprising: accepting a third request for a third decompressed frame from a third core; individually decompressing a third compressed frame at a frame level to produce the third decompressed frame; and in response to the third request, providing the third decompressed frame to the third core, wherein the first request indicates the first core; the second request indicates the second core; and the third request indicates the third core.

15. The method of claim 11, further comprising: establishing, by at least the first core and the second core, a time-sharing protocol for access to the frame decompression circuitry. the establishing comprises:

16. The method of claim 15, wherein, during a time slot corresponding to the first core, receiving an interrupt from the second core; and in response to receiving the interrupt, switching from performing decompression for the first core to performing decompression for the second core. ​ 17. The method of claim 11, further comprising: performing providing the first decompressed frame and providing the second decompressed frame in an order that depends at least in part on relative priorities between the first request and the second request.

18. The method of claim 11, further comprising: accepting, from a plurality of cores, a plurality of requests for respective ones of a plurality of compressed frames; compressing a plurality of decompressed frames to correspondingly produce the plurality of compressed frames; and providing the plurality of compressed frames to the plurality of cores in response to the plurality of requests.

19. An electronic device, comprising: a frame decompressor configured to decompress the plurality of compressed frames to produce a plurality of decompressed frames; a frame decompressor controller coupled to the frame decompressor and configured to arbitrate access to the frame decompressor for a plurality of cores, the frame decompressor controller comprising a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame decompressor among the plurality of cores, the time-sharing protocol handler configured to at least one of: establish the time-sharing protocol to include a plurality of time slots allocated to at least one of the plurality of cores; or establish the time-sharing protocol to allow receiving an interrupt signal from one of the plurality of cores, the interrupt signal indicating a request for at least one decompressed frame; a first core of the plurality of cores coupled to the frame decompressor controller, the first core configured to obtain, via the frame decompressor controller, a decompressed frame of the plurality of decompressed frames produced by the frame decompressor; and a second core of the plurality of cores coupled to the frame decompressor controller, the second core configured to obtain, via the frame decompressor controller, another decompressed frame of the plurality of decompressed frames produced by the frame decompressor, the first core and the second core configured to perform video transcoding.

20. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

21. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

22. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

23. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

24. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

25. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

26. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

27. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

28. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

29. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

30. The electronic device of claim 19, wherein: the first core is configured to decode a video stream using the decompressed frame to produce at least one decoded frame of a plurality of decoded frames; and the second core is configured to encode one or more decoded frames of the plurality of decoded frames using another decompressed frame for the video transcoding.

Images