Video frame codec architecture

Abstract

A video frame codec architecture is disclosed. Techniques and apparatus for a video frame codec architecture are described. A frame decompressor decompresses compressed frames to produce decompressed frames. A frame decompressor controller arbitrates shared access to the frame decompressor. Multiple cores of a SoC request to receive decompressed frames from the frame decompressor via the frame decompressor controller. The frame decompressor controller can implement a request queue and can order the servicing of requests based on the priority of the request or the requesting core. The frame decompressor controller can also establish a time-sharing protocol for access by multiple cores. In some embodiments, a video decoder is integrated with the frame decompressor logic and stores portions of the decompressed frame in a video buffer, and a display controller retrieves the portion for display using a synchronization mechanism. In a similar manner, the frame compressor controller can arbitrate shared access to the frame compressor for multiple cores.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 201880028788.3, ​​filed on July 13, 2018. Technical Field

[0003] The present application relates to a video frame codec architecture. Background Art

[0004] Electronic devices play an integral role in manufacturing, communications, healthcare, commerce, social interaction, and entertainment. For example, they power server farms, which in turn provide cloud-based distributed computing for commerce and communications. Computing devices are also embedded in many different types of modern equipment, from medical devices to appliances, from vehicles to industrial tools. Furthermore, one electronic device—the smartphone—has become a must-have.

[0005] Many electronic devices, such as those with cameras or displays, can manipulate video data. For example, video can be acquired using a security camera and then enhanced to improve certain visual aspects, such as clarity or contrast. Existing video data can also be manipulated to improve the appearance of individual video frames for presentation on the display screen of a smartphone or television monitor. For example, video data from a movie can be processed to enhance the realism of artificial graphics or improve display resolution. Video image data is also manipulated in industrial and medical settings. For example, image data from a 3D body scan can be stitched together into a video presentation for examination and analysis by a doctor.

[0006] In any of these situations, manipulating video data is a processing-intensive task. This is due, in part, to the size and amount of information typically present in video data. Consequently, the area of ​​an integrated circuit (IC) chip dedicated to processing video data can be larger than that used for other types of data. The increasing display resolution of video that electronic devices are expected to process exacerbates the difficulty of processing video data. For example, high-definition (HD) video contains approximately four times the video data of standard-definition (SD) video, and ultra-high-definition (UHD) or 4K video contains approximately four times the video data of HD video.

[0007] As a result, the amount of video data that electronic devices are expected to process has increased significantly over the past decade or so. As the use of virtual reality (VR) and artificial reality (AR) becomes more prevalent, the demand for video data processing is expected to increase further in the coming years. Consequently, electronic device manufacturers continue to work hard to improve the ability of electronic devices to handle this ever-increasing amount of video data.

[0008] This background description is provided to generally present the context of the present disclosure. Unless otherwise indicated herein, the materials described in this section are not admitted, either explicitly or implicitly, to be prior art to the present disclosure or the appended claims. Summary of the Invention

[0009] Techniques and devices for a video frame codec architecture are described. These techniques and devices enable integrated circuit (IC) chips to process high-bandwidth video data using a reduced amount of circuit resources, while also simplifying the workflow for upgrading to newer frame compression / decompression technologies, including lossless technologies. To this end, the inclusion of multiple separate frame decompression units in multiple different cores of the IC chip is avoided. Instead, a frame decompressor can provide frame decompression services to multiple different cores acting as frame decompressor client circuits. A frame decompressor controller uses a queuing or priority mechanism to sort decompression requests received from one or more of the multiple cores to facilitate sharing of decompression services. The frame decompressor controller can also arbitrate access to the frame decompressor according to a time-sharing protocol. In an example embodiment, the frame decompressor is co-located with the video decoder client circuits and is shared with at least a display controller. Similarly, the frame compressor can provide frame compression services to multiple different cores acting as frame compressor client circuits. The frame compressor controller facilitates sharing of compression services with multiple cores. Furthermore, both frame compression services and frame decompression services can be provided in a single IC chip for sharing among multiple cores.

[0010] Aspects described below include an electronic device comprising a frame decompressor and a frame decompressor controller. The frame decompressor is configured to decompress multiple compressed frames to generate multiple decompressed frames. The frame decompressor controller is coupled to the frame decompressor and configured to arbitrate access to the frame decompressor for multiple cores. The multiple cores include a first core and a second core. The first core is coupled to the frame decompressor controller and configured to obtain, via the frame decompressor controller, one of the multiple decompressed frames generated by the frame decompressor. The second core is coupled to the frame decompressor controller and configured to obtain, via the frame decompressor controller, another of the multiple decompressed frames generated by the frame decompressor. For example, the first core may be configured to obtain, via the frame decompressor controller, a first decompressed frame from the multiple decompressed frames generated by the frame decompressor. The second core may be configured to obtain, via the frame decompressor controller, a second decompressed frame from the multiple decompressed frames generated by the frame decompressor.

[0011] Aspects described below also include a method for sharing frame decompression circuitry among multiple cores. The method includes accepting a first request for a first decompressed frame from a first core. The method also includes decompressing the first compressed frame to produce a first decompressed frame. In response to the first request, the first decompressed frame is provided to the first core. The method also includes accepting a second request for a second decompressed frame from a second core. The method also includes decompressing the second compressed frame to produce a second decompressed frame. In response to the second request, the second decompressed frame is provided to the second core.

[0012] Aspects described below include another electronic device comprising a video decoder and a display controller. The video decoder is configured to decode a video stream to generate a plurality of decoded frames. The video decoder includes a frame compressor, a frame decompressor, and a frame decompressor controller. The frame compressor is configured to compress the plurality of decoded frames to generate a plurality of compressed frames. The frame decompressor is configured to decompress the plurality of compressed frames to generate a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and configured to arbitrate access to the frame decompressor. The display controller is coupled to the frame decompressor controller. The display controller is configured to obtain, via the frame decompressor controller, a decompressed frame from the plurality of decompressed frames generated by the frame decompressor.

[0013] Aspects described below also include a system comprising a frame decompressor and multiple cores. The frame decompressor is configured to decompress multiple compressed frames to generate multiple decompressed frames. The multiple cores include a first core and a second core. The first core is coupled to the frame decompressor and configured to obtain a decompressed frame from the multiple decompressed frames. The second core is coupled to the frame decompressor and configured to obtain another decompressed frame from the multiple decompressed frames. For example, the first core can be configured to obtain a first decompressed frame from the multiple decompressed frames. The second core can be configured to obtain a second decompressed frame from the multiple decompressed frames. The system also includes a control device for controlling the frame decompressor to arbitrate access to the frame decompressor for the multiple cores (including the first core and the second core). Additionally or alternatively, the system can include a frame compressor configured to compress multiple uncompressed frames to generate multiple compressed frames. The first core and the second core can each obtain a corresponding compressed frame from the multiple compressed frames. Therefore, the system may further include a control device for controlling the frame compressor to arbitrate access to the frame compressor for the plurality of cores (including the first core and the second core). BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The apparatus and techniques for implementing a video frame codec architecture are described with reference to the following figures. Throughout the figures, like numbers are used to refer to similar features and components:

[0015] Figure 1 An example environment including a printed circuit board is shown in which a video frame codec architecture may be implemented.

[0016] Figure 2 Other aspects of an example environment in which a video frame codec architecture may be implemented are shown.

[0017] Figure 3 A system on a chip (SoC) is shown having an example implementation of a video frame codec architecture including a frame compressor-decompressor, a frame compressor-decompressor controller, and multiple cores.

[0018] Figure 3-1 An SoC is shown with an example implementation of a video frame codec architecture including a frame compressor, a frame compressor controller, and multiple cores.

[0019] Figure 3-2 An SoC is shown with an example implementation of a video frame codec architecture including a frame decompressor, a frame decompressor controller, and multiple cores.

[0020] Figure 4 An example frame compressor-decompressor controller in conjunction with a frame decompressor and core is shown.

[0021] Figure 5 An example approach to implementing a video frame codec architecture is shown, where multiple cores include a video decoder and a display controller.

[0022] Figure 6 Example techniques for routing decompressed display frames from a video decoder to a display controller are shown.

[0023] Figure 7 An example scheme is shown in which a frame decompressor controller manages decompressed frame requests issued by multiple cores.

[0024] Figure 8 An example scheme is shown in which a frame compressor-decompressor controller establishes a time-sharing protocol for sharing frame compression resources or frame decompression resources.

[0025] Figure 9 An example method for operating the video frame codec architecture described herein is shown.

[0026] Figure 10 Various components of an example electronic device are shown that may implement a video frame codec architecture in accordance with one or more embodiments. DETAILED DESCRIPTION

[0027] Overview

[0028] Movie or other video data consumes significant storage bandwidth when idle and also consumes significant transmission bandwidth when traveling between electronic devices or their internal components. As video display resolutions increase, bandwidth requirements also increase. For example, with ultra-high-definition (UHD) or 4K video, the amount of video data to be processed is approximately 15-20 times greater than the SD video commonly used a decade ago. This increase in data volume makes it difficult to manage video, even within a single integrated circuit (IC) chip. For example, a typical system-on-chip (SoC) has a system bus that can become overloaded if used to transport raw decoded video data between different SoC components.

[0029] It is well known that the bandwidth of a video can be reduced by encoding it using some kind of lossy codec, such as H.264. The encoded video can then be streamed from a cloud server to a tablet computer, or stored on a Blu-ray disc or flash memory. The end-user electronic device is then responsible for decoding the video for presentation on a display screen, for example. Decoding entails converting the stream of binary ones and zeros, or bits, into individual decoded video frames that can be displayed sequentially to represent the video. As part of this video encoding and decoding process, the video data is compressed and decompressed. Using this video-level process (for example, using the H.264 codec), the amount of compression is significant, but the compression is also lossy. Therefore, due to the video-level compression / decompression process, some video information is lost to the extent that the image quality may be noticeably altered.

[0030] This loss of video information is accepted so that large amounts of video data can be transmitted between devices or stored using a reasonable amount of memory. After the video stream is decoded at the electronic device to produce decoded video frames, this large amount of video data is reconstructed into a group of frames or many individual frames on a frame-by-frame basis. Each of these individual frames is still a considerable amount of data. To process this amount of video data at the electronic device, if the decoded video frames are not currently being processed or displayed, the decoded video frames are compressed to produce compressed video frames. Because a limited number of frames are decoded from the stream at any given time, in some embodiments, a lossless compression algorithm can be applied to the decoded frames. The lossless compression and decompression frame-level process can prevent any further image degradation. Compressed video frames take up less memory space and consume less bandwidth on interconnects (such as the system bus of an SoC). Therefore, compressed video frames can be transferred between different chips on a printed circuit board, between an SoC and main memory, or even between different components of a single SoC.

[0031] To enable per-video-frame or frame-level compression in existing systems, each corresponding component or core in the SoC that processes video data includes a corresponding decompression unit. Using this straightforward, conventional approach, each core can independently generate decompressed video frames from compressed ones, then process the decompressed frames according to the given core's video-related purpose. However, this straightforward approach comes with several associated costs. First, a significant area on the IC die is dedicated to duplicating either the video frame decompression or compression circuitry. Second, the display path used to render video on the display includes a separate decompression unit. Third, the workflow for upgrading the SoC's video frame compression and decompression technology is complex. This complexity stems from the multiple distinct cores, each of which includes a separate decompression unit or a separate compression unit (or, in some cases, both). In other words, upgrading to a newer, more efficient decompression algorithm requires simultaneously modifying each core that includes a compression or decompression unit and then reintegrating it with the rest of the SoC. Consequently, this complexity in the upgrade workflow slows the adoption of improved compression / decompression algorithms across multiple core chips.

[0032] In contrast, certain embodiments described herein use a shared frame compressor-decompressor. The frame compressor-decompressor may include a frame compressor or a frame decompressor, or both. Multiple cores of an IC chip (e.g., an SoC) may obtain a decompressed version of a compressed frame from, for example, a shared frame decompressor. A given core may request a decompressed frame, and the frame decompressor may provide a response including the requested decompressed frame. Therefore, the frame decompressor may serve as a provider of frame decompression services, and each core may serve as a client of the frame decompression services. Similarly, the frame compressor may serve as a provider of frame compression services, and each core may serve as a client of the frame compression services. These frame-level compression and decompression services may use lossless algorithms.

[0033] In some embodiments, the SoC includes multiple different components, such as a video decoder and a display controller, which operate as different cores. The video decoder includes a stream decoder that decodes a video stream to generate decoded frames. The video decoder also includes a frame compressor that generates compressed frames from the decoded frames. The multiple different components of the SoC are configured to process decompressed frames. For example, the video decoder can use decompressed reference frames to further decode the video stream. In addition, the display controller can use the decompressed display frames to present the video on a display screen.

[0034] To obtain any of these example types of decompressed frames, a frame decompressor decompresses the corresponding compressed frame. The frame decompressor may use one or more buses, at least one buffer, or some other routing mechanism to route the requested decompressed frame to the requesting core. A frame decompressor controller arbitrates shared access to the frame decompressor between at least two cores (e.g., a video decoder and a display controller). The frame decompressor controller may manage a request queue for compressed frame requests. Management may require ordering the servicing of frame requests according to a priority scheme. The frame decompressor controller may also establish a time-sharing protocol for accessing the frame decompressor. The time-sharing protocol may include time slots allocated to different cores, interrupts issued by the receiving core to obtain control of the frame decompressor, and the like.

[0035] In this way, frame compressor or frame decompressor resources can be shared between two or more client cores processing video data. By sharing, for example, a frame decompressor, the area of ​​the SoC dedicated to frame decompression is reduced. In addition, there is no need to insert a separate frame decompressor along the video display path. In addition, the compression / decompression algorithm is decoupled from the overall SoC architecture. As a result, the compression / decompression algorithm can be updated more easily and therefore more frequently because fewer or even one frame compressor or frame decompressor is included on the SoC, and the update workflow is therefore easier to perform.

[0036] Example implementations at varying levels of detail are discussed below with reference to the associated figures. The following discussion first describes an example operating environment, then example scenarios and hardware, then example methods, and concludes with example electronic devices and related example aspects.

[0037] Sample Environment

[0038] Figure 1 An example environment 100 is shown including a printed circuit board 104 (PCB) in which a video frame codec architecture may be implemented. As shown, the environment 100 includes an electronic device 102. The electronic device 102 includes at least one PCB 104. The PCB 104 includes one or more integrated circuits, such as an integrated circuit 106 (IC). Figure 2 and Figure 3 As mentioned above, PCB 104 may include other integrated circuits, such as at least one memory separate from IC 106. IC 106 includes: at least one frame decompressor 108-2; a plurality of cores 110-1, 110-2, ..., 110-n, where n represents an integer greater than 1; and a plurality of video frames, such as at least one compressed frame 112 and at least one decompressed frame 114. For simplicity, Figure 1 Only the frame decompressor 108-2 is depicted while describing the example environment 100. However, reference is made below accordingly. Figure 3 and Figure 3-1 Frame compressor-decompressor 108 and frame compressor 108-1 are described.

[0039] In an example embodiment, a frame decompressor 108-2 is communicatively coupled to at least a portion of the plurality of cores 110-1 through 110-n. The frame decompressor 108-2 includes circuitry for decompressing compressed video frames. Thus, the frame decompressor 108-2 can decompress a compressed frame 112 to produce a decompressed frame 114. In operation, the cores 110 use the frame decompressor 108-2 to obtain a decompressed version of the compressed frame. For example, the nth core 110-n sends a request 116 to the frame decompressor 108-2 indicating that a decompressed version of the identified compressed frame 112 is being requested. The frame decompressor 108-2 provides a response 118 including the decompressed frame 114. Although video frames are referenced as an example context, the frames described herein may include any frame having visual data, including computer-generated graphics frames, video frames, composite frames, and the like.

[0040] In a similar manner, the frame decompressor 108-2 provides decompression services to individual cores of the plurality of cores 110-1 to 110-n. Similarly, (e.g., Figure 3-1 The frame compressor 108-1 of the plurality of cores 110-1 to 110-n may provide compression services to individual cores of the plurality of cores 110-1 to 110-n. Thus, the frame decompressor 108-2 or the frame compressor 108-1 (or both) implement at least a portion of a frame compression-decompression (FCD) server circuit 122 for the IC 106. Similarly, the plurality of cores 110-1 to 110-n implement multiple frame compression-decompression (FCD) client circuits 120. Using this client-server architecture, a frame compressor unit or a frame decompressor unit may be shared between two or more cores 110 to save space on the IC 106 and to simplify the workflow for upgrading the compression / decompression technology used to compress / decompress video frames of the IC 106. Figure 3 、 Figure 3-1 and Figure 3-2 A more detailed example architecture of the IC 106 and PCB 104 is described. However, reference is made next to Figure 2 Additional aspects of example implementations are described.

[0041] Figure 2Other aspects of an example environment 200 in which a video frame codec architecture as described herein may be implemented are shown. An electronic device 102 is shown using various non-limiting example devices: a smartphone 102-1, a laptop computer 102-2, a television 102-3, a desktop computer 102-4, a tablet computer 102-5, and a wearable device 102-6. As shown on the right, the electronic device 102 includes one or more processors 202, one or more computer-readable media 204, and at least one interconnect 216. The computer-readable media 204 may store, retain, or otherwise include code, data, instructions, other information, and the like. The electronic device 102 may also include an operating system 212. Although depicted separately, the operating system 212 may be stored on the one or more computer-readable media 204.

[0042] An application (not shown) or operating system 212, implemented as computer-readable instructions on computer-readable medium 204, may be executed by processor 202. Operating system 212 or basic input / output system (BIOS) may include a frame codec parameter module 214. Frame codec parameter module 214 may set one or more parameters to enable, authorize, tune, or otherwise facilitate performance of the shared frame compression and decompression functionality described herein.

[0043] As shown, the computer readable medium 204 may include at least one video buffer 206, at least one shared local cache 208, and at least one main memory 210. In some embodiments, the video buffer 206 and the shared local cache 208 are separate memory blocks on the IC. In other embodiments, the video buffer 206 and the shared local cache 208 are part of the same memory block, such as if a portion of the shared local cache 208 is used as the video buffer 206 in a dynamically changing or fixed allocation scheme. The interconnect 216 may include at least one system bus 218, at least one video bus 220, and at least one external bus 222. In some embodiments, the video bus 220 and the system bus 218 are different buses. In other embodiments, there is no separate video bus, so video data travels around the IC using the system bus 218. Figures 4 to 6 Example implementations of the computer-readable media 204 and the interconnects 216 are described.

[0044] Example Components and Techniques

[0045] Figure 3A portion of PCB 104 is shown. PCB 104 includes a system on a chip 302 (SoC) and main memory 210. SoC 302 depicts an example implementation of a video frame codec architecture that includes a frame compressor-decompressor 108 (FCD), a frame compressor-decompressor controller 304, and a plurality of cores 110-1 through 110-n. SoC 302 also includes a shared local cache 208 and a system bus 218. Main memory 210 is coupled to SoC 302 via an external bus 222, which is included as part of PCB 104 (e.g., disposed on PCB 104).

[0046] PCB 104 can be implemented with rigid or flexible materials for mounting or securing multiple IC chips, interconnects, interfaces, etc. Main memory 210 can be implemented using, for example, dynamic random access memory (DRAM) that is periodically refreshed to retain memory contents, or flash memory that can retain memory contents even without power. Typically, accessing data stored in main memory 210 consumes more energy than accessing data stored in SoC 302 (e.g., shared local cache 208). Shared local cache 208 can be implemented using, for example, static random access memory (SRAM), DRAM, flash memory, or some combination thereof.

[0047] The system bus 218 interconnects the plurality of cores 110-1 to 110-n, the shared local cache 208, the frame compressor-decompressor controller 304, and other components and interfaces (e.g., the frame compressor-decompressor 108 may be directly coupled to the system bus 218). Each core 110 in the plurality of cores 110-1 to 110-n may use the system bus 218 to store data at or retrieve data from the shared local cache 208. Similarly, each core 110 in the plurality of cores 110-1 to 110-n may use the external bus 222, e.g., also by using the system bus 218 or the shared local cache 208, to store data at or retrieve data from the main memory 210. For example, the first core 110-1 may store data in the shared local cache 208, and then the second core 110-2 may retrieve the stored data from the shared local cache 208.

[0048] In exemplary operation, the frame compressor-decompressor 108 (FCD) processes (e.g., compresses or decompresses) a plurality of unprocessed frames 306 to generate a plurality of processed frames 308. A frame compressor-decompressor controller 304 (FCD controller) is coupled to the frame compressor-decompressor 108 and arbitrates access to the frame compressor-decompressor 108 for the plurality of cores 110-1 through 110-n. Although shown separately, the frame compressor-decompressor 108 and the frame compressor-decompressor controller 304 may be logically integrated together. At least two cores 110 of the plurality of cores 110-1 through 110-n are coupled to the frame compressor-decompressor controller 304. Thus, each core 110 may obtain, via the frame compressor-decompressor controller 304, processed frames 308 generated by the frame compressor-decompressor 108 from its unprocessed frame versions. The processed frames 308 may be obtained using, for example, a request 116 and a corresponding response 118.

[0049] The frame compressor-decompressor 108 may include a frame compressor 308-1 (e.g., Figure 3-1 ), frame decompressor 108-2 (e.g., as depicted), Figure 3-2 Similarly, the frame compressor-decompressor controller 304 may include a frame compressor controller 304-1 (e.g., as shown in FIG. Figure 3-1 ), frame decompressor controller 304-2 (e.g., as shown in Figure 3-2 ) or both. Depending on whether the processing operation of the frame compressor-decompressor 108 is a compression or decompression operation, one of the unprocessed frames 306 or the processed frames 308 corresponds to a compressed frame and the other corresponds to a decompressed frame. If a given embodiment includes a frame compressor 108-1 and a frame decompressor 108-2 (and / or both a frame compressor controller 304-1 and a frame decompressor controller 304-2), these components can be located in the same location, adjacent to each other, or in different locations on the IC. For example, each can be located closer to a core that may be the most common client of the corresponding compression or decompression service. In addition, if the chip includes a co-located frame compressor 108-1 and a frame decompressor 108-2, each can include entirely separate circuitry or they can share circuitry. In some embodiments, the frame compressor-decompressor 108 can be implemented as a lossless frame data manipulator that adjusts the storage size of the data used for the frame as part of a lossless compression operation or a lossless decompression operation based on the request 116. Reference is made below accordingly. Figure 3-1 and Figure 3-2 Example implementations of compression and decompression operations are described.

[0050] Figure 3-1A SoC is shown having an example embodiment of a video frame codec architecture including a frame compressor 108-1 (FC), a frame compressor controller 304-1 (FC controller), and a plurality of cores. The frame compressor 108-1 compresses uncompressed frames (e.g., decompressed frames 114 or frames that have not yet been compressed) to produce compressed frames 112. The frame compressor controller 304-1 is coupled to the frame compressor 108-1 and arbitrates access to the frame compressor 108-1 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame compressor 108-1 and the frame compressor controller 304-1 can be logically integrated together.

[0051] At least two cores 110 of the plurality of cores 110-1 through 110-n are coupled to a frame compressor controller 304-1. Here, each core 110 may include a component or block that generates visual data. Thus, each core 110 may obtain, via the frame compressor controller 304-1, a compressed frame 112 generated by the frame compressor 108-1 from its decompressed version of the frame 114. Thus, the compressed frame 112 may be obtained using, for example, a request 116 and a corresponding response 118. In operation, the frame compressor controller 304-1 may grant access to the compression engine of the frame compressor 108-1 to some requesting cores 110 on the system bus 218 in a pipelined manner to avoid increasing traffic on the external bus 222 that provides access to the main memory 210. In some embodiments, the frame compressor controller 304-1 may temporarily grant exclusive access to the requesting core 110 to access compression resources.

[0052] Figure 3-2 A SoC is shown having an example embodiment of a video frame codec architecture including a frame decompressor 108-2 (FD), a frame decompressor controller 304-2 (FD controller), and a plurality of cores. The frame decompressor controller 304-2 is coupled to the frame decompressor 108-2 and arbitrates access to the frame decompressor 108-2 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame decompressor 108-2 and the frame decompressor controller 304-2 may be logically integrated together. At least two cores 110 of the plurality of cores 110-1 to 110-n are coupled to the frame decompressor controller 304-2. Thus, each core 110 can obtain a decompressed frame 114 generated by the frame decompressor 108-2 from its compressed version of the frame 112 via the frame decompressor controller 304-2. Thus, the decompressed frame 114 can be obtained using, for example, a request 116 and a corresponding response 118.

[0053] Some of the following description focuses on embodiments that can at least perform decompression operations. Thus, each embodiment may explicitly include a frame decompressor 108-2 and a frame compressor-decompressor controller 304 or a frame decompressor controller 304-2. However, each embodiment may additionally or alternatively include a frame compressor 108-2 for performing compression operations. Furthermore, how the compression operations and the movement of the compressed frames 112 can be performed in a manner at least similar to that of the reference frame 112 is discussed. Figures 4 to 6 The implementation of the method described for the decompression operation is obvious. Figure 7 Describes request processing and references Figure 8 The described time-sharing protocol can also be used in a similar manner for compression operations.

[0054] At least some of the cores 110 may operate on video data for video-related purposes. Video-related purposes include: video decoding, video display, video transcoding (e.g., decoding from one encoding format using one algorithm and encoding into another encoding format using another algorithm), video compensation (e.g., a graphics processing unit (GPU) operates on video frames to improve or enhance the image), and the like. Figure 5 , an example embodiment is described below in which the first core is implemented as a video decoder and the second core is implemented as a display controller. However, first referring to Figure 4 An example implementation of the frame decompressor controller 304 is described.

[0055] Figure 4 An example frame compressor-decompressor controller 304 in conjunction with the frame decompressor 108-2 and the core 110 is shown generally at 400. The core 110 includes an FCD client interface 410, and the frame decompressor controller 304 includes an FCD service interface 408. The FCD client interface 410 and the FCD service interface 408 collectively enable communication between the core 110 and the frame compressor-decompressor controller 304. Thus, using these two interfaces, the core 110 can send requests 116 to the frame compressor-decompressor controller 304, and the frame compressor-decompressor controller 304 can send responses 118 to the core 110.

[0056] The FCD client interface 410 and the FCD service interface 408 may communicate via the system bus 218 (not shown), the video bus 218, or the like. The request 116 may be transmitted using a bit line, one or more bits of a register, a message formulated as at least one packet, or the like. The response 118, including the decompressed frame 114, may be transmitted using a bus coupling the core 110 to the frame decompressor 108-2, using a memory accessible to both the core 110 and the frame decompressor 108-2, some combination thereof, or the like. Figure 5 and Figure 6Describes examples of communication with different interfaces.

[0057] As shown, the frame compressor-decompressor controller 304 includes a request manager 402, a request queue 404, a priority manager 406, and a time-sharing protocol handler 412. The FCD service interface 408 is coupled to at least the request manager 402, the priority manager 406, and the time-sharing protocol handler 412. Figure 7 Describes an example scenario for managing requests, including prioritizing the services of requests. Figure 8 An example scenario is described for using the time-sharing protocol handler 412 to establish a time-sharing protocol to access the frame decompressor 108-2.

[0058] Figure 5 An example approach 500 for implementing a video frame codec architecture is shown, in which multiple cores of a SoC include a video decoder 502 and a display controller 504. On the left, the video decoder 502 includes a stream decoder 506, which includes a frame compressor 108-1. However, the frame compressor 108-1 can be implemented separately from the stream decoder 506 or the video decoder 502. On the right, the display controller 504 includes a frame buffer 518. The frame decompressor 108-2, the frame compressor-decompressor controller 304 (FCD controller), and the video buffer 206 are associated with the video decoder 502. For example, these three components can be co-located with the video decoder 502 (e.g., located next to the video decoder 502 or physically intermixed with it), can be at least partially logically integrated with the video decoder 502, or some combination thereof. Although described with reference to the video decoder 502 and the display controller 504, these principles apply generally to the core 110. Furthermore, although the video buffer 206 is depicted separately from the shared local cache 208 , the video buffer 206 may also be incorporated as part of or allocated from the shared local cache 208 .

[0059] In an example embodiment, an electronic device obtains a video stream 514, which may include a bitstream encoded to represent a video. Video stream 514 is composed of multiple raw frames 516, which are encoded using a codec such as H.264. Stream decoder 506 receives video stream 514 via system bus 218 and decodes video stream 514 to produce multiple decoded frames 512, one of which is explicitly shown. Frame compressor 108-1 compresses multiple decoded frames 512 to produce multiple compressed frames 112. As shown, frame compressor 108-1 is integrated with stream decoder 506. Therefore, frame compressor 108-1 can compress decoded frames 512 as part of the decoding process; however, frame compression can be implemented in alternative ways. These decoded and compressed frames 112 can be stored in video buffer 206, shared local cache 208, or main memory 210 for subsequent use by video decoder 502 or another core.

[0060] The decoded and compressed frames 112 can be used for different video-related purposes. For example, if the compressed frame is to be used for further video decoding purposes, the compressed frame 112 can include a compressed reference frame 112R. Alternatively, if another compressed frame is to be used for video display purposes, the other compressed frame 112 can include a compressed display frame 112D. For the video decoding function of the video decoder 502, the frame decompressor 108-2 decompresses the compressed reference frame 112R to produce a decompressed reference frame 114R. The video decoder 502 can use the decompressed reference frame 114 to continue decoding the video stream 514.

[0061] For the video display function of the display controller 504, the frame decompressor 108-2 decompresses the compressed display frame 112D to generate the decompressed display frame 114D. The display controller 504 can access the display frame 112D via the frame decompressor controller client interface 410 and the frame decompressor controller service interface 408 ( Figure 4 ) from the frame compressor-decompressor controller 304 (eg, which includes at least Figure 3-2 The frame decompressor controller 304-2 of the display controller 504 requests the decompression service. The frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the routing mechanism 510 (indicated by the solid wavy arrow). The display controller 504 can store the decompressed display frame 114D in the frame buffer 518. The display controller 504 can therefore use the stored decompressed display frame 114D to present an image of the video frame on the display screen. Although Figure 5The frame compressor-decompressor controller 304 is described in terms of utilizing the frame decompressor 108-2 to provide decompression services to multiple cores, but the principles also apply to the frame compressor-decompressor controller 304 utilizing the frame compressor 108-1 to provide compression services to multiple cores.

[0062] Figure 5 2. A plurality of example routing mechanisms 510 are indicated using small dashed lines. The routing mechanisms 510 represent different ways that the frame compressor-decompressor controller 304 provides decompressed frames 114 to a requesting core (e.g., the video decoder 502 or the display controller 504). In a first routing mechanism 510-1, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to a frame buffer 518 of the display controller 504 via the system bus 218. In a second routing mechanism 510-2, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and the shared local cache 208. For example, the frame compressor-decompressor controller 304 may store the decompressed display frame 114D in the shared local cache 208, and the display controller 504 may retrieve the stored decompressed display frame 114D from the shared local cache 208.

[0063] In a third routing mechanism 510-3, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and the main memory 210. For example, the frame compressor-decompressor controller 304 may store the decompressed display frame 114D off-chip in the main memory 210 via the system bus 218 and the external bus 222. The display controller 504 may retrieve the stored decompressed display frame 114D from the main memory 210 using both buses and then store the frame in the frame buffer 518.

[0064] The fourth routing mechanism 510-4 involves servicing the video decoder 502 as a client of the frame decompression service. In the fourth routing mechanism 510-4, the frame compressor-decompressor controller 304 provides the decompressed reference frame 114R to the video decoder 502. To do so, the frame compressor-decompressor controller 304 transfers the decompressed reference frame 114R to the video buffer 206. The video decoder 502 can access the decompressed reference frame 114R from the video buffer 206.

[0065] In the fifth routing mechanism 510-5, if the integrated circuit includes a video bus 220 as well as a system bus 218, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video bus 220, which is at least partially separate from the system bus 218. As shown, the video bus 220 is dedicated to communicating video data and extends at least between the frame compressor-decompressor controller 304 and the display controller 504. However, the video bus 220 may be implemented differently, such as by transmitting other video data to and from other cores.

[0066] In the sixth routing mechanism 510-6, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video buffer 206. The frame compressor-decompressor controller 304 stores at least a portion of the decompressed display frame 114D in the video buffer 206. In one approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the system bus 218. In another approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the video bus 220 (e.g., if present separately from the system bus 218). Regardless of the approach employed, the sixth routing mechanism 510-6 may further require the display controller 504 to access a memory mapping mechanism for the video buffer 206. Next, refer to Figure 6 An example implementation of a memory mapping mechanism is described.

[0067] Figure 6 An example technique 600 is shown for routing decompressed display frames 114D from the video decoder 502 to the display controller 504 using the sixth routing mechanism 510-6. In this example, the video decoder 502 is integrated with the frame decompressor 108-2 and the frame compressor-decompressor controller 304. In addition to the video buffer 206, the video decoder 502 also includes synchronization registers 602. In this example, the video buffer 206 is used as an output buffer. Therefore, the frame compressor-decompressor controller 304 causes the frame decompressor 108-2 to store at least a portion of the decompressed display frame 114D in the video buffer 206.

[0068] The example technique 600 for the sixth routing mechanism 510-6 involves at least one status communication 606. A first status communication 606-1 occurs between the frame compressor-decompressor controller 304 and the synchronization register 602. A second status communication 606-2 occurs between the display controller 504 and the synchronization register 602. Each status communication 606 may include a polling operation or an update operation and may be performed over the system bus 218 or the video bus 220.

[0069] In response to storing at least a portion of the decompressed display frame 114D in the video buffer 206, the frame compressor-decompressor controller 304 writes a value 604 to the synchronization register 602 as part of an update operation of a first status communication 606-1. The value 604 indicates the storage state of the contents of the video buffer 206. Here, the value 604 indicates that specific video data (e.g., one or more display lines 608 of the decompressed display frame 114D) has been newly placed in the video buffer 206. Thus, the video buffer 206 can function as a line buffer for holding the one or more display lines 608.

[0070] At the same time, the display controller 504 extracts a value 604 from the synchronization register 602 as part of a polling operation of a second status communication 606-2. Based on the value 604, the display controller 504 determines that one or more new display lines 608 of the requested decompressed display frame 114D are present and available in the video buffer 206. The display controller 504 therefore retrieves at least one display line 608 from the video buffer 206 via the system bus 218 or the video bus 220 based on the value 604. In response to the retrieval, the display controller 504 writes another value 604 to the synchronization register 602 as part of an update operation of the second status communication 606-2 to indicate the retrieval status of the corresponding display line 608. The another value 604 therefore indicates that at least one corresponding display line 608 has been read from the video buffer 206.

[0071] Periodically, or as part of certain operations, the frame compressor-decompressor controller 304 polls the synchronization register 602 as part of a first status communication 606-1. The frame compressor-decompressor controller 304 checks another value 604 to determine whether the corresponding display line 608 has been accessed and retrieved by the display controller 504. If so, the frame compressor-decompressor controller 304 knows that the storage location of the corresponding display line 608 can be overwritten with new data.

[0072] The value 604 or the further value 604 may comprise, for example, one or more bits. The value 604 and the further value 604 may correspond to different memory locations or the same memory location of the synchronization register 602. Although described with reference to the video decoder 502 and the display controller 504, these principles apply generally to the core 110. By polling and updating the value 604 in the synchronization register 602, a memory-mapped transfer mechanism may be used to control the routing of the decompressed display frame 114D.

[0073] Figure 7 An example scheme 700 is shown in which the frame compressor-decompressor controller 304 manages requests 116 issued by multiple cores for decompressed frames. In this example, the video decoder 502 is integrated with the frame decompressor 108-2 and the frame compressor-decompressor controller 304. The frame compressor-decompressor controller 304 is composed of a request manager 402, a request queue 404, and a priority manager 406 (also in FIG. Figure 4 As shown in FIG. 4 , request queue 404 includes a plurality of requests 116 . Each request 116 corresponds to a priority 706 . Each request 116 also includes or corresponds to a requester identifier 702 and a frame identifier 704 .

[0074] Some requests for decompressed frames may be more important or temporally related than other requests. For example, a request related to real-time presentation of streaming video may be more important to an end user than a request for a video transcoding operation that can be performed in the background at a best-efforts rate. Therefore, a request for the former video-related purpose may take precedence over a request for the latter video-related purpose. To this end, a priority 706 may be associated with a request, core, frame, etc.

[0075] In an example embodiment, the requester identification 702 identifies the requesting core 110, for example ( Figure 5 and 6 ) video decoder 502 or display controller 504. The frame identifier 704 identifies the requested decompressed frame 114, for example, by a frame number, a time index, or an offset thereof. The requester identifier 702 can be implemented using a specific voltage value on one or more bit lines, with an assigned identification value transferred with the request 116, based on the origin or bus used to provide the request 116 (e.g., if a dedicated bus is provided for each core). The frame identifier 704 can be explicitly included in each request 116, can be implicitly included in the request 116 (e.g., if the next consecutive frame is required), can be included in a previous request as part of a frame range, etc.

[0076] In operation, the request manager 402 receives requests 116 from the cores 110, for example, via the frame decompressor controller service interface 408. The request manager 402 inserts the requests 116 into the request queue 404. In one approach, the request manager 402 operates the request queue 404 in a first-in, first-out (FIFO) manner. Alternatively, the request manager 402 may insert each request 116 into the request queue 404 based on a corresponding priority 706. The corresponding priority 706 for each individual request 116 may be included or may accompany the corresponding individual request 116. Alternatively, the priority 706 may be explicitly assigned to a group of requests 116 in bulk, or each core 110 may be associated with a specific priority 706 that is implicitly assigned to each request 116 issued by the associated core 110.

[0077] As shown, each request 116 is stored in the request queue 404 in association with a corresponding priority level 706, such that requests 116 having different priorities 706 are stored in the same request queue 404. Alternatively, the frame compressor-decompressor controller 304 may implement the illustrated request queue 404 using multiple request queues 404, each request queue 404 being associated with a corresponding priority level 706. Thus, each request 116 is stored in a corresponding request queue 404 associated with the corresponding priority level 706 corresponding to the request 116.

[0078] Regardless of the approach used to implement the illustrated request queue 404, the priority manager 406 orders the servicing of the plurality of requests 116 according to the plurality of priorities 706 corresponding to the plurality of requests 116. Thus, the priority manager 406 retrieves the next request 116 using a priority scheme based on the relative priorities between two or more requests 116. The priority manager 406 forwards the next request 116 to the frame decompressor 108-2. The frame decompressor 108-2 retrieves the compressed frame 112 corresponding to the requested decompressed frame 114. Figure 5 and 6 In the example scenario of FIG. 1 , frame decompressor 108 - 2 retrieves compressed display frame 112D and performs a decompression operation to produce decompressed display frame 114D. Frame compressor-decompressor controller interface 408 causes frame decompressor 108 - 2 to route decompressed display frame 114D to video decoder 502 as part of response 118 .

[0079] Figure 8 The frame compressor-decompressor controller 304 is shown establishing a shared frame compression resource (eg, frame compressor 108-1 (eg, Figure 3-1 )) or frame decompression resources (e.g., frame decompressor 108-2 (e.g., Figure 3-2)) of at least one time-sharing protocol 800. The time-sharing protocol handler 412 of the frame compressor-decompressor controller 304 (not explicitly shown) can establish at least one example time-sharing protocol 800-1, 800-2, or 800-3. As shown by arrow 812, time increases in the right direction. Each example time-sharing protocol 800 is described in terms of sharing frame decompression resources between two cores; however, the disclosed time-sharing protocol 800 can be applied to sharing frame decompression resources between more than two cores. Although Figure 8 It is described in terms of time-sharing for frame decompression, but the principles also apply to time-sharing protocols that implement frame compression.

[0080] Using the first time-sharing protocol 800-1, time is divided into multiple time slots. These time slots have fixed durations, and each corresponding time slot is assigned to a corresponding core. Time slot 802-1 corresponds to the first core, such as the video decoder 502. Time slot 802-2 corresponds to the second core, such as the display controller 504. In operation, the time-sharing protocol handler 412 rotates access to frame decompression resources between the two cores at the specified lengths of two different time slots. In the first time-sharing protocol 800-1, interrupts are not enabled.

[0081] In contrast, interrupt 806 is enabled for the second time-sharing protocol 800-2. Here, the frame decompression resources are typically reserved for one of the two clients (in this case, the first core). Therefore, time slots 804-1 correspond to the first core and can extend indefinitely. These time slots 804-1 extend until the time-sharing protocol handler 412 receives an interrupt 806. The core 110 can signal the interrupt 806 via a message on the system bus, using a dedicated video bus, by driving an interrupt line high, etc. In response to detecting the interrupt 806, the time-sharing protocol handler 412 initiates a context switch to transfer access to the frame decompression resources to a different core and begins another time slot.

[0082] As shown in the second time-sharing protocol 800-2, the second core issues an interrupt 806a. The time-sharing protocol handler 412 detects the interrupt 806a and transfers access to the frame decompressor 108-2 to the second core that issued the interrupt. Consequently, the frame decompressor 108-2 performs at least one decompression operation to generate a decompressed frame 114 for the requesting second core during the time slot 804-2 following the interrupt 806a. The length of each time slot 804-2 can be fixed, can depend on the number of requests queued for the second core, can depend on the priority of each queued request, and so on. After the first core is again granted access to the frame decompression resources for another time slot 804-1, the second core can subsequently issue another interrupt 806b. In an alternative approach, each time slot 804-2 allocated to the second core can continue until another core issues an interrupt (not shown). Thus, in this alternative approach, a context switch can occur in response to the interrupt 806 and is independent of the elapsed time.

[0083] Utilizing the third time-sharing protocol 800-3, time is divided into multiple time slots. The third time-sharing protocol 800-3 is a hybrid of the first and second time-sharing protocols. Some time slots, such as time slot 808-1, have fixed, uninterruptible times. Other time slots, such as time slot 810-1, have variable times that can be interrupted by another core. As shown, the first core is assigned to time slots 808-1 and 810-1. An interrupt 806c from the second core arrives during time slot 808-1. However, the time-sharing protocol handler 412 ignores the interrupt 806c (or queues it). On the other hand, an interrupt 806d arriving during time slot 810-1 triggers a context switch. Utilizing the context switch, after interrupt 806d, frame decompression resources are allocated to the second core for time slot 804-2. If the interrupt 806c is queued (rather than dequeued) during the time slot 808-1, then a context switch may be initiated at the end of the time slot 808-1 or at the beginning of the time slot 810-1 in response to the interrupt 806c.

[0084] Having generally described an environment in which techniques and apparatuses may be implemented for a video frame codec architecture, discussion now turns to example methods.

[0085] Example Method

[0086] The following discussion describes methods for operating a video frame codec architecture. These methods can be implemented using the examples described above, such as Figure 1 and Figure 3 The electronic device 102 and PCB 104 are shown, and Figure 10 The electronic device 1000 is depicted. Aspects of these methods are Figure 9, which are shown as operations 902-912 performed by one or more components of at least one IC chip. The order in which the operations of these methods are shown and / or described is not intended to be construed as limiting, and any number or combination of the described method operations can be combined in any order to implement a method or an alternative method. Although Figure 9 The flowchart is described in terms of operating the frame decompressor 108-2, but the principles may be similarly applied to operating the frame compressor 108-1.

[0087] Figure 9 An example method 900 for operating a video frame codec architecture is shown, which can be implemented by circuitry providing frame decompression services to multiple different cores. At 902, a first request for a first decompressed frame is accepted from a first core. For example, the frame compressor-decompressor controller 304 (e.g., its frame decompressor controller 304-2) can accept the first request 116 for the first decompressed frame 114 from the first core 110-1. The acceptance can be performed via the system bus 218, the video bus 220, the synchronization register 602, etc.

[0088] At 904, the first compressed frame is decompressed to produce a first decompressed frame. For example, the frame decompressor 108-2 may decompress the first compressed frame 112 to produce the first decompressed frame 114. The servicing of the first request 116 may be performed according to the associated priority 706.

[0089] At 906, in response to the first request, the first decompressed frame is provided to the first core. For example, frame compressor-decompressor controller 304 may provide first decompressed frame 114 to first core 110-1 in response to first request 116. Routing mechanism 500 may be used to transfer first decompressed frame 114 from frame decompressor 108-2 to first core 110-1.

[0090] At 908, a second request for a second decompressed frame is accepted from the second core. For example, the frame compressor-decompressor controller 304 may accept the second request 116 for the second decompressed frame 114 from the second core 110-2. The first core 110-1 may include the display controller 504, and the second core 110-2 may include the video decoder 502.

[0091] At 910, the second compressed frame is decompressed to generate a second decompressed frame. For example, frame decompressor 108-2 may decompress second compressed frame 112 to generate second decompressed frame 114. Decompression may be performed using a lossless compression / decompression algorithm. At 912, in response to a second request, the second decompressed frame is provided to the second core. For example, frame compressor-decompressor controller 304 may provide second decompressed frame 114 to second core 110-2 in response to second request 116. After obtaining second decompressed frame 114, second core 110-2 may process the frame according to corresponding video-related functions.

[0092] The previous discussion describes methods related to video frame codec architectures. Aspects of these methods can be implemented in, for example, hardware (e.g., fixed logic circuitry), firmware, or some combination thereof. These techniques can be used Figures 1 to 8 and Figure 10 One or more entities or components shown (electronic device 1000 in the following Figure 10 These entities or components may be further divided, combined, etc. Thus, these figures illustrate some of the many possible systems or devices that can employ the described technology. The entities and components of these figures generally represent firmware, hardware, IC chips, circuits, or combinations thereof.

[0093] Example electronic device

[0094] Figure 10 Various components of an example electronic device 1000 are shown, and the example electronic device 1000 may be described with reference to the previous Figures 1 to 9 The electronic device 1000 may be implemented as any one or a combination of fixed or mobile devices, in the form of a consumer, computer, portable, user, server, communication, telephone, navigation, gaming, audio, camera, messaging, media playback, and / or other types of electronic devices 102 (e.g., Figure 1 any form of smartphone shown).

[0095] The electronic device 1000 includes a communication transceiver 1002 that allows for wired and / or wireless communication of device data 1004, such as received data, transmitted data, or other information as described above. Example communication transceivers 1002 include NFC transceivers, various IEEE 802.15 (Bluetooth) TM ) standard WPAN radio, compliant with various IEEE 802.11 (WiFi TM) standards, WLAN radios for cellular phones (compliant with 3GPP), various IEEE 802.16 (WiMAX) TM ) standard Wireless Metropolitan Area Network (WMAN) radios and wired Local Area Network (LAN) Ethernet transceivers.

[0096] The electronic device 1000 may also include one or more data input ports 1006 through which any type of data, media content, and / or other input may be received, such as user-selectable input, messages, applications, music, television content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source. The data input ports 1006 may include USB ports, coaxial cable ports, and other serial or parallel connectors (including internal connectors) for flash memory, DVDs, CDs, and the like. These data input ports 1006 may be used to couple the electronic device to components, peripherals, or accessories, such as a keyboard, microphone, or camera.

[0097] The electronic device 1000 of this example includes at least one processor 1008 (e.g., any one or more of an application processor, a microprocessor, a digital signal processor (DSP), a controller, etc.), which may include a combined processor and memory system (e.g., implemented as part of a SoC) that processes (e.g., executes) computer-executable instructions stored on a computer-readable medium to control the operation of the device. The processor 1008 may be implemented as an application processor, an embedded controller, a microcontroller, a SoC, etc. In general, the processor or processing system may be implemented at least partially in hardware, which may include components of an integrated circuit or system on a chip, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations in silicon and / or other hardware.

[0098] Alternatively or additionally, the electronic device 1000 may be implemented using any one or combination of electronic circuits, which may include software, hardware, firmware, or fixed logic circuits, in combination with processing and control circuits, which are generally represented by 1010 (as electronic circuit 1010). The electronic circuit 1010 may implement executable or hardware-based modules (not shown) using logic circuits and / or hardware (e.g., such as an FPGA).

[0099] Although not shown, the electronic device 1000 may include a system bus, interconnect, crossbar, or data transfer system that couples the various components within the device. The system bus or interconnect may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any one of a variety of bus architectures.

[0100] The electronic device 1000 also includes one or more storage devices 1012 that enable data storage, examples of which include random access memory (RAM), non-volatile memory (e.g., read-only memory (ROM), flash memory, EPROM, and EEPROM), and disk storage devices. The storage device(s) 1012 provide a data storage mechanism to store device data 1004, other types of code and / or data, and various device applications 1020 (e.g., software applications or programs). For example, an operating system 1014 can be stored as software instructions in the memory device 1012 and executed by the processor 1008.

[0101] As shown, electronic device 1000 also includes audio and / or video processing system 1016, which processes audio data and / or passes audio and video data to audio system 1018 and / or display system 1022 (e.g., a video buffer or a screen of a smart phone or camera). Audio system 1018 and / or display system 1022 can include any device that processes, displays and / or otherwise presents audio, video, display and / or image data. Display data and audio signals can be transmitted to audio components and / or display components via RF (radio frequency) link, S-video link, HDMI (high-definition multimedia interface), composite video link, component video link, DVI (digital video interface), analog audio connection or other similar communication links such as media data port 1024. In some embodiments, audio system 1018 and / or display system 1022 are external or separate components of electronic device 1000. Alternatively, display system 1022 can be an integrated component of example electronic device 1000, such as a part of an integrated touch interface.

[0102] Figure 10 The electronic device 1000 is Figure 1 Thus, the processor 1008 is (eg, correspondingly Figure 1 、 Figure 2 and Figure 3 ) integrated circuit 106, processor 202 or SoC 302. Therefore, as Figure 10As shown, the processor 1008 may include a frame compressor-decompressor 108, a plurality of cores 110-1 to 110-n, and a frame compressor-decompressor controller 304. Figure 10 In the video buffer 206, the shared local cache 208 and the main memory 210 (e.g., also Figure 2 、 Figure 3 、 Figure 5 or Figure 6 Each of) is depicted as forming at least a portion of the memory system, for example by being part of one or more storage devices 1012. As described above, one or more of these memory components may be part of the same integrated circuit as the processor 1008 (e.g., the video buffer 206 and the shared local cache 208 are Figure 3 302), or alternatively located on a separate die, or even provided on a different printed circuit board. Thus, the principles of the video frame codec architecture described herein can be implemented by Figure 10 The electronic device 1000 is implemented, or combined with Figure 10 The electronic device 1000 is implemented.

[0103] Although embodiments of the video frame codec architecture have been described in language specific to certain features and / or methods, the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example embodiments of the video frame codec architecture.

Claims

1. An electronic device comprising: a frame compressor configured to individually compress a plurality of uncompressed frames at a frame level to produce a plurality of compressed frames; a frame compressor controller coupled to the frame compressor and configured to arbitrate access to the frame compressor for a plurality of cores; a first core of the plurality of cores coupled to the frame compressor controller, the first core configured to obtain, via the frame compressor controller, a compressed frame of the plurality of compressed frames generated by the frame compressor at a frame level; and A second core of the plurality of cores is coupled to the frame compressor controller, the second core being configured to obtain, via the frame compressor controller, another compressed frame of the plurality of compressed frames generated by the frame compressor at a frame level.

2. The electronic device according to claim 1, wherein: The first core is configured to use the compressed frame for a first video-related purpose; The second core is configured to use the another compressed frame for a second video-related purpose; and The first video-related purpose and the second video-related purpose include different video-related purposes selected from a group of video-related purposes, wherein the group of video-related purposes includes: video decoding, video display, video transcoding or video compensation.

3. The electronic device according to claim 1, wherein: The first core includes a video decoder configured to decode a video stream to produce a plurality of decoded frames; The frame compressor is logically integrated with the video decoder; and The second core includes a display controller.

4. The electronic device according to claim 1, wherein The frame compressor controller comprises: a request queue configured to store a plurality of requests, each request corresponding to at least one compressed frame; and A request manager is configured to receive a plurality of requests from the plurality of cores and insert the plurality of requests into the request queue.

5. The electronic device according to claim 4, wherein The frame compressor controller includes a priority manager configured to order servicing of the plurality of requests according to a plurality of priorities corresponding to the plurality of requests. The electronic device according to claim 1 , wherein: The frame compressor controller includes a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame compressor among the plurality of cores.

7. The electronic device according to claim 1, further comprising: a frame decompressor configured to individually decompress one or more compressed frames at a frame level to produce one or more decompressed frames, the one or more compressed frames being derived from the plurality of decoded frames; and A frame decompressor controller is coupled to the frame decompressor and configured to arbitrate access to the frame decompressor for the plurality of cores.

8. The electronic device according to claim 7, further comprising: a video buffer associated with the frame decompressor; and a synchronization register corresponding to the video buffer, the synchronization register being configured to store one or more values ​​indicative of a state of contents of the video buffer, wherein: The frame decompressor is configured to store at least a portion of a decompressed frame of the one or more decompressed frames in the video buffer; and The frame decompressor controller is configured to write at least one value indicative of a status of the stored portion to the synchronization register.

9. The electronic device according to claim 6, wherein: The time-sharing protocol handler is configured to at least one of: establishing the time-sharing agreement to include a plurality of time slots assigned to at least one core of the plurality of cores; or The time-sharing protocol is established to enable receipt of an interrupt signal from one of the plurality of cores, the interrupt signal indicating a request for at least one compressed frame.

10. The electronic device according to claim 1, wherein The frame compressor is configured to implement a lossless frame-level compression algorithm to generate the plurality of compressed frames from the plurality of uncompressed frames, respectively.

11. An electronic device comprising: a frame compressor configured to compress a plurality of uncompressed frames to produce a plurality of compressed frames; a frame compressor controller coupled to the frame compressor and configured to arbitrate access to the frame compressor for a plurality of cores, the frame compressor controller including a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame compressor among the plurality of cores, the time-sharing protocol handler configured to at least one of: establishing the time-sharing agreement to include a plurality of time slots assigned to at least one core of the plurality of cores; or establishing the time-sharing protocol to enable receipt of an interrupt signal from one of the plurality of cores, the interrupt signal indicating a request for at least one compressed frame; a first core of the plurality of cores coupled to the frame compressor controller, the first core configured to obtain, via the frame compressor controller, a compressed frame of the plurality of compressed frames generated by the frame compressor; and A second core of the plurality of cores is coupled to the frame compressor controller, the second core being configured to obtain, via the frame compressor controller, another compressed frame of the plurality of compressed frames generated by the frame compressor.

12. The electronic device according to claim 11, wherein: The first core includes a video decoder configured to decode a video stream to produce a plurality of decoded frames; The frame compressor is logically integrated with the video decoder; and The second core includes a display controller.

13. A method for sharing frame compression circuitry among multiple cores, the method comprising: accepting a first request for a first compressed frame from a first core; individually compressing the first uncompressed frames at a frame level to generate the first compressed frames; providing the first compressed frame to the first core in response to the first request; accepting a second request for a second compressed frame from a second core; individually compressing the second uncompressed frames at a frame level to produce second compressed frames; and In response to the second request, the second compressed frame is provided to the second core.

14. The method according to claim 13, further comprising: accepting a third request for a third compressed frame from a third core; individually compressing the third uncompressed frame at a frame level to generate the third compressed frame; and In response to the third request, providing the third compressed frame to the third core, wherein The first request indicates the first core; The second request indicates the second core; and The third request indicates the third core.

15. The method according to claim 13, further comprising: A time-sharing protocol for accessing the frame compression circuit is established by at least the first core and the second core.

16. The method according to claim 15, wherein The establishment includes: receiving an interrupt from the second core during a time slot corresponding to the first core; and In response to receiving the interrupt, switching from performing compression on the first core to performing compression on the second core.

17. The method according to claim 13, further comprising: Providing the first compressed frame and providing the second compressed frame are performed in an order that depends at least in part on a relative priority between the first request and the second request.

18. The method of claim 13, wherein: The second compressed frame includes lines and additional lines; and Providing the second compressed frame includes: polling a synchronization register to obtain at least one value indicative of a read status of the row relative to a video buffer; and The additional line is stored in the video buffer in response to the at least one value.

19. The method according to claim 13, further comprising: accepting a plurality of requests for corresponding ones of the plurality of decompressed frames from the plurality of cores; decompressing the plurality of compressed frames to generate the plurality of decompressed frames accordingly; and In response to the plurality of requests, the plurality of decompressed frames are provided to the plurality of cores.

20. The method of claim 13, wherein: The first uncompressed frame comprises or is derived from a first decoded frame; and The second uncompressed frame includes or is derived from a second decoded frame.

Images