Video frame codec architecture

Abstract

Video frame codec architectures are disclosed. Techniques and apparatuses for video frame codec architectures are described. A frame decompressor decompresses compressed frames to produce decompressed frames. A frame decompressor controller arbitrates shared access to the frame decompressor. Multiple cores of a SoC request decompressed frames from the frame decompressor via the frame decompressor controller. The frame decompressor controller can implement a request queue and can order service of requests based on priority of the requests or requesting core. The frame decompressor controller can also establish a time-sharing protocol for access by the multiple cores. In some implementations, a video decoder is integrated with frame decompressor logic and stores portions of the decompressed frames in a video buffer and a display controller retrieves portions for display using a synchronization mechanism. In a similar manner, a frame compressor controller can arbitrate shared access to a frame compressor by multiple cores.

Description

[0001] Divisional

[0002] This application is a divisional of Chinese Patent Application No. 201880028788.3, filed on July 13, 2018, which has a priority date of July 13, 2018. TECHNICAL FIELD

[0003] This application relates to video frame codec architecture. BACKGROUND

[0004] Electronic devices play an indispensable role in manufacturing, communications, healthcare, commerce, socializing, and entertainment. For example, electronic devices power server farms that provide cloud-based distributed computing functionality for commerce and communications. Devices with computing capability are also embedded in many different types of modern equipment, from medical devices to appliances, from vehicles to industrial tools. In addition, one electronic device, the smartphone, has become something that must be carried on one’s person.

[0005] Many electronic devices, such as those with cameras or display screens, can manipulate video data. For example, video can be obtained using a security camera and then enhanced to improve certain visual aspects, such as sharpness or contrast. Existing video data can also be manipulated to improve the appearance of individual video frames for presentation on a display screen of a smartphone or television monitor. For example, video data for a movie can be processed to improve the realism of artificial graphics or to increase display resolution. Video image data is also manipulated in industrial and medical settings. For example, image data from a three-dimensional body scan can be stitched together into a video presentation for review and analysis by a physician.

[0006] In any of these cases, manipulation of video data is a processing-intensive task. This is due in part to the size or amount of information that is typically present in video data. As a result, the area of an integrated circuit (IC) chip dedicated to processing video data can be larger than that used for other types of data. The difficulty of processing video data is exacerbated by the increasing display resolutions of videos that electronic devices are expected to process. For example, the video data for high definition (HD) video is approximately four times that of standard definition (SD) video, and the video data for ultra-high definition (UHD) or 4K video is approximately four times that of HD video.

[0007] As a result, the amount of video data that electronic devices are expected to process has increased substantially over the past decade or so. With the use of virtual reality (VR) and artificial reality (AR) becoming more widespread, video data processing demands are expected to increase further in the coming years. As a result, electronic device manufacturers continue to work to improve the ability of electronic devices to process increasing amounts of video data.

[0008] This background description is provided for the purpose of generally presenting the context of the disclosure. The subject matter of the present disclosure, however, is not directed to any one particular background nor is it expressly or implicitly admitted as prior art to any patent application including this background. SUMMARY

[0009] Techniques and apparatuses for video frame codec architectures are described. These techniques and apparatuses enable integrated circuit (IC) chips to process high bandwidth video data using a smaller amount of circuit resources, while also facilitating a simplified workflow to upgrade to newer frame compression / decompression technologies, including lossless technologies. To this end, the inclusion of multiple separate frame decompression units in multiple different cores of an IC chip is avoided. Instead, a frame decompressor can provide frame decompression services to multiple different cores that act as frame decompressor client circuits. A frame decompressor controller facilitates sharing of the decompression services using a queuing or priority mechanism that orders decompression requests received from one or more of the multiple cores. The frame decompressor controller can also arbitrate access to the frame decompressor according to a time-sharing protocol. In example implementations, the frame decompressor is co-located with video decoder client circuits, and the frame decompressor is shared at least with a display controller. Similarly, a frame compressor can provide frame compression services to multiple different cores that act as frame compressor client circuits. A frame compressor controller facilitates sharing of the compression services with the multiple cores. Moreover, both frame compression services and frame decompression services can be provided in a single IC chip for sharing among multiple cores.

[0010] Aspects described below include an electronic device that includes a frame decompressor and a frame decompressor controller. The frame decompressor is configured to decompress a plurality of compressed frames to produce a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and is configured to arbitrate access to the frame decompressor for a plurality of cores. The plurality of cores includes a first core and a second core. The first core is coupled to the frame decompressor controller and is configured to obtain, via the frame decompressor controller, one of the plurality of decompressed frames produced by the frame decompressor. The second core is coupled to the frame decompressor controller and is configured to obtain, via the frame decompressor controller, another of the plurality of decompressed frames produced by the frame decompressor. For example, the first core can be configured to obtain, via the frame decompressor controller, a first decompressed frame of the plurality of decompressed frames produced by the frame decompressor. The second core can be configured to obtain, via the frame decompressor controller, a second decompressed frame of the plurality of decompressed frames produced by the frame decompressor.

[0011] The aspects described below also include a method for sharing frame decompression circuitry among multiple cores. The method includes accepting a first request from a first core for a first decompressed frame. The method also includes decompressing a first compressed frame to produce the first decompressed frame. The first decompressed frame is provided to the first core in response to the first request. The method also includes accepting a second request from a second core for a second decompressed frame. The method also includes decompressing a second compressed frame to produce the second decompressed frame. The second decompressed frame is provided to the second core in response to the second request.

[0012] The aspects described below include another electronic device that includes a video decoder and a display controller. The video decoder is configured to decode a video stream to produce a plurality of decoded frames. The video decoder includes a frame compressor, a frame decompressor, and a frame decompressor controller. The frame compressor is configured to compress the plurality of decoded frames to produce a plurality of compressed frames. The frame decompressor is configured to decompress the plurality of compressed frames to produce a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and is configured to arbitrate access to the frame decompressor. The display controller is coupled to the frame decompressor controller. The display controller is configured to obtain, via the frame decompressor controller, one of the plurality of decompressed frames produced by the frame decompressor.

[0013] The aspects described below also include a system that includes a frame decompressor and a plurality of cores. The frame decompressor is configured to decompress a plurality of compressed frames to produce a plurality of decompressed frames. The plurality of cores includes a first core and a second core. The first core is coupled to the frame decompressor and is configured to obtain one of the plurality of decompressed frames. The second core is coupled to the frame decompressor and is configured to obtain another one of the plurality of decompressed frames. For example, the first core can be configured to obtain a first one of the plurality of decompressed frames. The second core can be configured to obtain a second one of the plurality of decompressed frames. The system also includes control means for controlling the frame decompressor to arbitrate access to the frame decompressor for the plurality of cores, including the first core and the second core. Additionally or alternatively, the system can include a frame compressor configured to compress a plurality of uncompressed frames to produce the plurality of compressed frames. The first core and the second core can each obtain a respective one of the plurality of compressed frames. Accordingly, the system can also include control means for controlling the frame compressor to arbitrate access to the frame compressor for the plurality of cores, including the first core and the second core. BRIEF DESCRIPTION OF DRAWINGS

[0014] Apparatuses and techniques for implementing a video frame codec architecture are described with reference to the following figures.

[0015] The same numbers are used throughout the drawings to reference like features and components:

[0016] Figure 1 An example environment including a printed circuit board in which a video frame codec architecture can be implemented is shown.

[0017] Figure 2 Other aspects of an example environment in which a video frame codec architecture can be implemented are shown.

[0018] Figure 3 A system on a chip (SoC) with an example implementation of a video frame codec architecture including a frame compressor-decompressor, a frame compressor-decompressor controller, and a plurality of cores is shown.

[0019] Figure 3-1 A SoC with an example implementation of a video frame codec architecture including a frame compressor, a frame compressor controller, and a plurality of cores is shown.

[0020] Figure 3-2 A SoC with an example implementation of a video frame codec architecture including a frame decompressor, a frame decompressor controller, and a plurality of cores is shown.

[0021] Figure 4 An example frame compressor-decompressor controller in conjunction with a frame decompressor and a core is shown.

[0022] Figure 5 An example approach to implementing a video frame codec architecture in which a plurality of cores includes a video decoder and a display controller is shown.

[0023] Figure 6 An example technique for routing decompressed display frames from a video decoder to a display controller is shown.

[0024] Figure 7 An example scheme in which a frame decompressor controller manages decompressed frame requests issued by a plurality of cores is shown.

[0025] Figure 8 An example scheme in which a frame compressor-decompressor controller establishes a time-sharing protocol for sharing frame compression resources or frame decompression resources is shown.

[0026] Figure 9 An example method for operating a video frame codec architecture as described herein is shown.

[0027] Figure 10 Various components of an example electronic device in which a video frame codec architecture can be implemented according to one or more implementations are shown. DETAILED DESCRIPTION

[0028] SUMMARY

[0029] Data for movies or other videos consumes a large amount of storage bandwidth when idle and a large amount of transmission bandwidth when propagating between electronic devices or internal components thereof. As video display resolutions increase, the bandwidth requirements also increase. For example, with ultra-high definition (UHD) or 4K video, the video data to be processed is about 15-20 times that of the SD video commonly used a decade ago. This increase in data volume makes managing video difficult, even within a single integrated circuit (IC) chip. For example, a typical system-on-a-chip (SoC) has a system bus that can become overloaded if the system bus is used to transport raw decoded video data between different SoC components.

[0030] It is well known that the bandwidth of video can be reduced by encoding the video using some form of lossy codec, such as H.264. The encoded video can then be streamed from a cloud server to a tablet computer or saved on a Blu-ray disk or flash memory. The end-user electronic device is then responsible for decoding the video for presentation on a display screen, for example. Decoding requires converting a stream of binary ones and zeros, or bits, into individual decoded video frames that can be sequentially displayed to represent the video. As part of this video encoding and decoding process, the video data is compressed and decompressed. With this video-level process (e.g., with the H.264 codec), the amount of compression is large, but the compression is also lossy. Thus, due to the video-level compression / decompression process, some video information is lost to the extent that the image quality can change noticeably.

[0031] This loss of video information is accepted in order to enable the transmission of large amounts of video data between devices or to store using a reasonable amount of storage. After the video stream is decoded at an electronic device to produce decoded video frames, this large amount of video data is reconstructed on a frame-by-frame basis into a set of frames or many individual frames. Each of these individual frames is still a considerable amount of data. To handle this amount of video data at the electronic device, if the decoded video frames are not currently being processed or displayed, the decoded video frames are compressed to produce compressed video frames. Because a limited number of frames are decoded from the stream at any given time, a lossless compression algorithm can be applied to the decoded frames in some implementations. The lossless compression and decompressed frame-level process prevents any further image degradation. The compressed video frames take up less memory space and consume less bandwidth on interconnects, such as the system bus of an SoC. Thus, the compressed video frames can be transferred between different chips on a printed circuit board, between an SoC and main memory, or even between different components of a single SoC.

[0032] To enable per-video frame or frame-level compression in existing systems, each respective component or core of a SoC that processes video data includes a respective decompression unit. With this straightforward conventional approach, each core can independently produce decompressed video frames from compressed video frames, and then process the decompressed video frames according to the video-related purposes of the given core. However, this straightforward approach has several attendant costs. First, a significant area on the IC die is dedicated to duplicating video frame decompression circuitry or compression circuitry. Second, the display path for presenting video on a display screen includes a separate decompression unit. Third, upgrading the video frame compression and decompression technology of a SoC is very complex. This complexity arises from the multiple different cores, each of which includes an individual decompression unit or an individual compression unit (in some cases, both). In other words, to upgrade to a newer, more efficient decompression algorithm, each core that includes a compression unit or a decompression unit must be modified, and then re-integrated with the rest of the SoC. Thus, the complexity of this workflow upgrade slows the adoption of improved compression / decompression algorithms in multiple core chips.

[0033] In contrast, certain embodiments described herein use a shared frame compressor-decompressor. The frame compressor-decompressor can include a frame compressor or a frame decompressor, or both. Multiple cores of an IC chip, such as a SoC, can obtain decompressed versions of compressed frames from, for example, a shared frame decompressor. A given core can request a decompressed frame, and the frame decompressor can provide a response that includes the requested decompressed frame. Thus, the frame decompressor can function as a frame decompression service provider, and each core can function as a client of the frame decompression service. Similarly, a frame compressor can function as a frame compression service provider, and each core can function as a client of the frame compression service. These frame-level compression and decompression services can use lossless algorithms.

[0034] In some embodiments, a SoC includes multiple different components, such as a video decoder and a display controller, that operate as different cores. The video decoder includes a stream decoder that decodes a video stream to produce a stream of decoded frames. The video decoder also includes a frame compressor that produces compressed frames from the decoded frames. The multiple different components of the SoC are configured to process decompressed frames. For example, the video decoder can use decompressed reference frames to further decode the video stream. In addition, the display controller can use decompressed display frames to present video on a display screen.

[0035] To obtain any of these example types of decompressed frames, a frame decompressor decompresses the corresponding compressed frames. The frame decompressor can route the requested decompressed frames to the requesting core using one or more buses, at least one buffer, or some other routing mechanism. A frame decompressor controller arbitrates shared access to the frame decompressor between at least two cores, such as a video decoder and a display controller. The frame decompressor controller can manage a request queue of compressed frame requests. Management can entail ordering service of the frame requests according to a priority scheme. The frame decompressor controller can also establish a time-sharing protocol for accessing the frame decompressor. The time-sharing protocol can include assigning time slots to different cores, accepting interrupts from the cores to take control of the frame decompressor, and so on.

[0036] In these ways, frame compressor resources or frame decompressor resources can be shared among two or more client cores that process video data. By sharing, for example, a frame decompressor, the area of an SoC dedicated to frame decompression is reduced. Moreover, a separate frame decompressor need not be inserted along the video display path. Furthermore, the compression / decompression algorithm is decoupled from the overall SoC architecture. Thus, the compression / decompression algorithm can be more easily and, therefore, more frequently updated because fewer or even one frame compressor or frame decompressor is included on the SoC and, therefore, the update workflow is eased.

[0037] Example implementations at different levels of detail are discussed below with reference to the associated drawings. The discussion below first describes an example operating environment, followed by example schemes and hardware, then example methods, and ends with an example electronic device and related example aspects.

[0038] Example Environment

[0039] Figure 1 An example environment 100 is shown that includes a printed circuit board 104 (PCB) in which a video frame codec architecture can be implemented. As shown, the environment 100 includes an electronic device 102. The electronic device 102 includes at least one PCB 104. The PCB 104 includes one or more integrated circuits, such as an integrated circuit 106 (IC). As discussed below with reference to Figure 2 and Figure 3 The PCB 104 can include other integrated circuits, such as at least one memory that is separate from the IC 106, as discussed below with reference to FIG. 2. The IC 106 includes: at least one frame decompressor 108-2; a plurality of cores 110-1, 110-2,... 110-n, n representing an integer greater than 1; and a plurality of video frames, such as at least one compressed frame 112 and at least one decompressed frame 114. For simplicity, Figure 1 Only the frame decompressor 108-2 is depicted while describing the example environment 100. However, the example environment 100 is described below with reference to FIG. 2 accordingly.Figure 3 and Figure 3-1 Describe frame compressor-decompressor 108 and frame compressor 108-1.

[0040] In the example implementation, frame decompressor 108-2 is communicatively coupled to at least a portion of a plurality of cores 110-1 to 110-n. Frame decompressor 108-2 includes circuitry for decompressing compressed video frames. Thus, frame decompressor 108-2 can decompress compressed frame 112 to produce decompressed frame 114. In operation, core 110 uses frame decompressor 108-2 to obtain a decompressed version of the compressed frame. For example, the nth core 110-n sends a request 116 to frame decompressor 108-2 indicating that it is requesting a decompressed version of the identified compressed frame 112. Frame decompressor 108-2 provides a response 118 including the decompressed frame 114. Although video frames are referenced as an example context, frames described herein can include any frame having visual data, including computer-generated graphics frames, video frames, composite frames, etc.

[0041] In a similar manner, frame decompressor 108-2 provides decompression services to individual cores of multiple cores 110-1 to 110-n. Similarly, (for example, Figure 3-1 The frame compressor 108-1 can provide compression services to individual cores of multiple cores 110-1 to 110-n. Therefore, the frame decompressor 108-2 or the frame compressor 108-1 (or both) implements at least a portion of the frame compression-decompression (FCD) server circuitry 122 for IC 106. Similarly, multiple cores 110-1 to 110-n implement multi-frame compression-decompression (FCD) client circuitry 120. Using this client-server architecture, the frame compressor unit or frame decompressor unit can be shared among two or more cores 110 to save space on IC 106 and simplify the workflow of compression / decompression techniques used to compress / decompress video frames for upgrading IC 106. See below for reference. Figure 3 , Figure 3-1 and Figure 3-2 A more detailed example architecture describing IC 106 and PCB 104 is provided below. However, please refer to the following... Figure 2 Describe additional aspects of the example implementation.

[0042] Figure 2Other aspects of an example environment 200 in which video frame codec architectures as described herein can be implemented are shown. Electronic device 102 is shown with various non-limiting example devices: a smartphone 102-1, a notebook computer 102-2, a television 102-3, a desktop computer 102-4, a tablet computer 102-5, and a wearable device 102-6. As shown on the right, electronic device 102 includes one or more processors 202, one or more computer-readable media 204, and at least one interconnect 216. Computer-readable media 204 can store, maintain, or otherwise include code, data, instructions, other information, etc. Electronic device 102 can also include an operating system 212. Although described separately, operating system 212 can be stored on one or more computer-readable media 204.

[0043] An application (not shown) or operating system 212 implemented as computer- readable instructions on computer-readable media 204 can be executed by processor 202. Operating system 212 or a basic input / output system (BIOS) can include a frame codec parameter module 214. Frame codec parameter module 214 can set one or more parameters to enable, authorize, tune, or otherwise facilitate the execution of the shared frame compression and decompression functionality described herein.

[0044] As shown, computer-readable media 204 can include at least one video buffer 206, at least one shared local cache 208, and at least one main memory 210. In some implementations, video buffer 206 and shared local cache 208 are separate blocks of memory on an IC. In other implementations, video buffer 206 and shared local cache 208 are part of the same block of memory, such as if a portion of shared local cache 208 is used as video buffer 206 in a dynamic or fixed allocation scheme. Interconnect 216 can include at least one system bus 218, at least one video bus 220, and at least one external bus 222. In some implementations, video bus 220 and system bus 218 are different buses. In other implementations, there is no separate video bus, and so video data travels around the IC using system bus 218. Reference is made to Figure 4 to Figure 6 Example implementations of these computer-readable media 204 and these interconnects 216 are described.

[0045] Example components and techniques

[0046] Figure 3A portion of the PCB 104 is shown. The PCB 104 includes a system on a chip 302 (SoC) and a main memory 210. The SoC 302 depicts an example implementation of a video frame codec architecture that includes a frame compressor-decompressor 108 (FCD), a frame compressor-decompressor controller 304, and a plurality of cores 110-1 through 110-n. The SoC 302 also includes a shared local cache 208 and a system bus 218. The main memory 210 is coupled to the SoC 302 via an external bus 222 that is included as part of (e.g., disposed on) the PCB 104.

[0047] The PCB 104 can be implemented in a rigid or flexible material for mounting or securing a plurality of IC chips, interconnects, interfaces, etc. The main memory 210 can be implemented using dynamic random access memory (DRAM) that is periodically refreshed to maintain the memory contents, for example, or flash memory that can maintain the memory contents without power. Generally, accessing data stored on the main memory 210 consumes more energy than accessing data stored on the SoC 302 (e.g., the shared local cache 208). The shared local cache 208 can be implemented using static random access memory (SRAM), DRAM, flash memory, or some combination thereof, for example.

[0048] The system bus 218 interconnects the plurality of cores 110-1 through 110-n, the shared local cache 208, the frame compressor-decompressor controller 304, and other components and interfaces (e.g., the frame compressor-decompressor 108 can be directly coupled to the system bus 218). Each core 110 of the plurality of cores 110-1 through 110-n can store data at or retrieve data from the shared local cache 208 using the system bus 218. Similarly, each core 110 of the plurality of cores 110-1 through 110-n can store data at or retrieve data from the main memory 210 using the external bus 222, e.g., also by using the system bus 218 or the shared local cache 208. For example, a first core 110-1 can store data in the shared local cache 208, and then a second core 110-2 can retrieve the stored data from the shared local cache 208.

[0049] In example operations, a frame compressor-decompressor 108 (FCD) processes (e.g., compresses or decompresses) a plurality of unprocessed frames 306 to produce a plurality of processed frames 308. A frame compressor-decompressor controller 304 (FCD controller) is coupled to the frame compressor-decompressor 108 and arbitrates access to the frame compressor-decompressor 108 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame compressor-decompressor 108 and the frame compressor-decompressor controller 304 can be logically integrated together. At least two of the plurality of cores 110-1 to 110-n are coupled to the frame compressor-decompressor controller 304. Thus, each core 110 can obtain, via the frame compressor-decompressor controller 304, a processed frame 308 produced by the frame compressor-decompressor 108 from an unprocessed frame version thereof. The processed frame 308 can be obtained using, for example, a request 116 and a corresponding response 118.

[0050] The frame compressor-decompressor 108 can include a frame compressor 308-1 (e.g., as shown Figure 3-1 depicted), a frame decompressor 108-2 (e.g., as shown Figure 3-2 ), or both. Similarly, the frame compressor-decompressor controller 304 can include a frame compressor controller 304-1 (e.g., as shown Figure 3-1 depicted), a frame decompressor controller 304-2 (e.g., as shown Figure 3-2 ), or both. Depending on whether the processing operation of the frame compressor-decompressor 108 is a compression or a decompression operation, one of the unprocessed frames 306 or the processed frames 308 corresponds to a compressed frame and the other corresponds to a decompressed frame. If a given implementation includes both a frame compressor 108-1 and a frame decompressor 108-2 (and / or both a frame compressor controller 304-1 and a frame decompressor controller 304-2), these components can be located in the same place, adjacent to each other, or at different locations on the IC. For example, each can be located closer to the core that is likely to be the most frequent client of the corresponding compression or decompression service. Moreover, if the chip includes both a frame compressor 108-1 and a frame decompressor 108-2 located in the same place, each can include completely separate circuitry or they can share circuitry. In some implementations, the frame compressor-decompressor 108 can be implemented as a lossless frame data manipulator that adjusts the storage size of data for a frame as part of a lossless compression operation or a lossless decompression operation according to the request 116. Example implementations of compression and decompression operations are described below with respect to Figure 3-1 and Figure 3-2 .

[0051] Figure 3-1A SoC is shown having an example implementation of a video frame codec architecture that includes a frame compressor 108-1 (FC), a frame compressor controller 304-1 (FC controller), and a plurality of cores. The frame compressor 108-1 compresses uncompressed frames (e.g., decompressed frames 114 or frames that have not yet been compressed) to produce compressed frames 112. The frame compressor controller 304-1 is coupled to the frame compressor 108-1 and arbitrates access to the frame compressor 108-1 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame compressor 108-1 and the frame compressor controller 304-1 can be logically integrated together.

[0052] At least two cores 110 of the plurality of cores 110-1 to 110-n are coupled to the frame compressor controller 304-1. Here, each core 110 can include a component or block that produces visual data. Thus, each core 110 can obtain, via the frame compressor controller 304-1, a compressed frame 112 produced by the frame compressor 108-1 from a decompressed frame 114 version thereof. Thus, the compressed frame 112 can be obtained using, for example, a request 116 and a corresponding response 118. In operation, the frame compressor controller 304-1 can grant access to the compression engine of the frame compressor 108-1 to some requesting core 110 on a system bus 218 in a pipelined manner to avoid increasing traffic to an external bus 222 that provides access to a main memory 210. In some implementations, the frame compressor controller 304-1 can temporarily grant exclusive access to a requesting core 110 to access compression resources.

[0053] Figure 3-2 A SoC is shown having an example implementation of a video frame codec architecture that includes a frame decompressor 108-2 (FD), a frame decompressor controller 304-2 (FD controller), and a plurality of cores. The frame decompressor controller 304-2 is coupled to the frame decompressor 108-2 and arbitrates access to the frame decompressor 108-2 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame decompressor 108-2 and the frame decompressor controller 304-2 can be logically integrated together. At least two cores 110 of the plurality of cores 110-1 to 110-n are coupled to the frame decompressor controller 304-2. Thus, each core 110 can obtain, via the frame decompressor controller 304-2, a decompressed frame 114 produced by the frame decompressor 108-2 from a compressed frame 112 version thereof. Thus, the decompressed frame 114 can be obtained using, for example, a request 116 and a corresponding response 118.

[0054] Some of the following description focuses on implementations that can at least perform decompression operations. Thus, each can explicitly include a frame decompressor 108-2 and a frame compressor-decompressor controller 304 or a frame decompressor controller 304-2. However, each implementation can additionally or alternatively include a frame compressor 108-2 for performing compression operations. Moreover, how compression operations and movement of compressed frames 112 can be performed in at least similar fashion to that described with respect to decompression operations is apparent. Figure 4 to Figure 6 It is apparent how the described manner for decompression operations can be implemented. Moreover, the described request processing and reference Figure 7 The described time-sharing protocol can likewise be used for compression operations in similar fashion. Figure 8

[0055] At least some of the cores 110 can operate on video data for video-related purposes. Video-related purposes include video decoding, video display, video transcoding (e.g., decoding from one encoding format using one algorithm and encoding into another encoding format using another algorithm), video compositing (e.g., a graphics processing unit (GPU) operating on video frames to improve or enhance the image), and so forth. Reference is made to Figure 5 Example implementations are described below in which a first core implements a video decoder and a second core implements a display controller. However, reference is first made to Figure 4 An example implementation of the frame decompressor controller 304 is described.

[0056] Figure 4 An example frame compressor-decompressor controller 304 in conjunction with a frame decompressor 108-2 and a core 110 is shown generally at 400. The core 110 includes a FCD client interface 410 and the frame decompressor controller 304 includes a FCD service interface 408. The FCD client interface 410 and the FCD service interface 408 together enable communication between the core 110 and the frame compressor-decompressor controller 304. Thus, using these two interfaces, the core 110 can send requests 116 to the frame compressor-decompressor controller 304 and the frame compressor-decompressor controller 304 can send responses 118 to the core 110.

[0057] The FCD client interface 410 and the FCD service interface 408 can communicate over a system bus 218 (not shown), a video bus 218, and so forth. The requests 116 can be communicated using a bit line, one or more bits of a register, a message formulated as at least one packet, and so forth. The responses 118 including the decompressed frames 114 can be communicated using a bus coupling the core 110 to the frame decompressor 108-2, using a memory accessible by both the core 110 and the frame decompressor 108-2, some combination thereof, and so forth. Reference is made to Figure 5 and Figure 6 ​Examples of different interface communications are described.

[0058] As shown, frame compressor-decompressor controller 304 includes request manager 402, request queue 404, priority manager 406, and time- sharing protocol handler 412. FCD service interface 408 is coupled to at least request manager 402, priority manager 406, and time-sharing protocol handler 412. Reference is made below to Figure 7 Examples of schemes for managing requests, including prioritizing service to requests, are described. Reference is made below to Figure 8 Examples of schemes using time-sharing protocol handler 412 to establish a time-sharing protocol to access frame decompressor 108-2 are described.

[0059] Figure 5 An example approach 500 to implementing a video frame codec architecture is shown, in which a plurality of cores of a SoC include a video decoder 502 and a display controller 504. On the left, video decoder 502 includes a stream decoder 506, which includes frame compressor 108-1. However, frame compressor 108-1 can be implemented separately from stream decoder 506 or video decoder 502. On the right, display controller 504 includes a frame buffer 518. Frame decompressor 108-2, frame compressor-decompressor controller 304 (FCD controller), and video buffer 206 are associated with video decoder 502. For example, these three components can be co-located with video decoder 502 (e.g., disposed next to or physically mixed with video decoder 502), can be at least partially logically integrated with video decoder 502, or some combination thereof. Although described with reference to video decoder 502 and display controller 504, these principles are generally applicable to core 110. Moreover, although video buffer 206 is described separately from shared local cache 208, video buffer 206 can also be incorporated as part of shared local cache 208 or allocated from shared local cache 208.

[0060] In an example implementation, an electronic device receives a video stream 514, which may include a bitstream encoded to represent video. The video stream 514 consists of a plurality of raw frames 516 encoded using some codec such as H.264. A stream decoder 506 receives the video stream 514 via a system bus 218 and decodes it to produce a plurality of decoded frames 512, one of which is explicitly shown. A frame compressor 108-1 compresses the plurality of decoded frames 512 to produce a plurality of compressed frames 112. As shown, the frame compressor 108-1 is integrated with the stream decoder 506. Therefore, the frame compressor 108-1 can compress the decoded frames 512 as part of the decoding process; however, frame compression can be implemented in an alternative manner. These decoded and compressed frames 112 can be stored in a video buffer 206, a shared local cache 208, or main memory 210 for subsequent use by the video decoder 502 or another core.

[0061] The decoded and compressed frame 112 can be used for different video-related purposes. For example, if the compressed frame is to be used for further video decoding purposes, the compressed frame 112 may include a compressed reference frame 112R. Alternatively, if another compressed frame is used for video display purposes, the other compressed frame 112 may include a compressed display frame 112D. For the video decoding function of the video decoder 502, the frame decompressor 108-2 decompresses the compressed reference frame 112R to produce a decompressed reference frame 114R. The video decoder 502 can use the decompressed reference frame 114 to continue decoding the video stream 514.

[0062] For the video display function of the display controller 504, the frame decompressor 108-2 decompresses the compressed display frame 112D to generate the decompressed display frame 114D. The display controller 504 can be configured via the frame decompressor controller client interface 410 and the frame decompressor controller service interface 408. Figure 4 The two) from the frame compressor-decompressor controller 304 (e.g., which includes at least Figure 3-2 The frame compressor-decompressor controller 304-2 requests the decompression service. The frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via routing mechanism 510 (indicated by the solid wavy arrow). The display controller 504 can store the decompressed display frame 114D in the frame buffer 518. The display controller 504 can then use the stored decompressed display frame 114D to present the image of the video frame on the display screen. Although... Figure 5The frame compressor-decompressor controller 304 is described in terms of providing decompression services to multiple cores using the frame decompressor 108-2, but these principles also apply to the frame compressor-decompressor controller 304 providing compression services to multiple cores using the frame compressor 108-1.

[0063] Figure 5 A plurality of example routing mechanisms 510 are indicated using small dashed lines. The routing mechanisms 510 represent different pathways by which the frame compressor-decompressor controller 304 provides decompressed frames 114 to requesting cores, such as the video decoder 502 or the display controller 504. In a first routing mechanism 510-1, the frame compressor-decompressor controller 304 provides decompressed display frames 114D to the frame buffer 518 of the display controller 504 via the system bus 218. In a second routing mechanism 510-2, the frame compressor-decompressor controller 304 provides decompressed display frames 114D to the display controller 504 via the system bus 218 and the shared local cache 208. For example, the frame compressor-decompressor controller 304 can store decompressed display frames 114D in the shared local cache 208, and the display controller 504 can retrieve the stored decompressed display frames 114D from the shared local cache 208.

[0064] In a third routing mechanism 510-3, the frame compressor-decompressor controller 304 provides decompressed display frames 114D to the display controller 504 via the system bus 218 and the main memory 210. For example, the frame compressor-decompressor controller 304 can store decompressed display frames 114D off-chip in the main memory 210 via the system bus 218 and the external bus 222. The display controller 504 can retrieve the stored decompressed display frames 114D from the main memory 210 using both buses, and then store the frames in the frame buffer 518.

[0065] A fourth routing mechanism 510-4 involves the video decoder 502 being a client of the frame decompression service. In the fourth routing mechanism 510-4, the frame compressor-decompressor controller 304 provides decompressed reference frames 114R to the video decoder 502. To do so, the frame compressor-decompressor controller 304 transfers the decompressed reference frames 114R to the video buffer 206. The video decoder 502 can access the decompressed reference frames 114R from the video buffer 206.

[0066] In the fifth routing mechanism 510-5, if the integrated circuit includes a video bus 220 in addition to the system bus 218, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video bus 220, which is at least partially separate from the system bus 218. As shown, the video bus 220 is dedicated to propagating video data and extends at least between the frame compressor-decompressor controller 304 and the display controller 504. However, the video bus 220 can be implemented differently, such as by transferring other video data to and from other cores.

[0067] In the sixth routing mechanism 510-6, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video buffer 206. The frame compressor-decompressor controller 304 stores at least a portion of the decompressed display frame 114D in the video buffer 206. In one approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the system bus 218. In another approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the video bus 220 (e.g., if present separately from the system bus 218). Regardless of the approach, the sixth routing mechanism 510-6 can further require a memory mapping mechanism for the display controller 504 to access the video buffer 206. Reference is next made to Figure 6 An example implementation of the memory mapping mechanism is described.

[0068] Figure 6 An example technique 600 is shown for routing the decompressed display frame 114D from the video decoder 502 to the display controller 504 using the sixth routing mechanism 510-6. In this example, the video decoder 502 is integrated with the frame decompressor 108-2 and the frame compressor-decompressor controller 304. In addition to the video buffer 206, the video decoder 502 includes a synchronization register 602. In this example, the video buffer 206 is used as an output buffer. Accordingly, the frame compressor-decompressor controller 304 causes the frame decompressor 108-2 to store at least a portion of the decompressed display frame 114D in the video buffer 206.

[0069] The example technique 600 of the sixth routing mechanism 510-6 involves at least one status communication 606. A first status communication 606-1 occurs between the frame compressor-decompressor controller 304 and the synchronization register 602. A second status communication 606-2 occurs between the display controller 504 and the synchronization register 602. Each status communication 606 can include a polling operation or an update operation, and can be performed over the system bus 218 or the video bus 220.

[0070] In response to storing at least a portion of the decompressed display frame 114D in the video buffer 206, the frame compressor-decompressor controller 304 writes a value 604 into the synchronization register 602 as part of an update operation of the first status communication 606-1. The value 604 indicates a storage status of the contents of the video buffer 206. Here, the value 604 indicates that particular video data (e.g., one or more display lines 608 of the decompressed display frame 114D) has been newly placed in the video buffer 206. Thus, the video buffer 206 can function as a line buffer holding the one or more display lines 608.

[0071] Meanwhile, the display controller 504 extracts the value 604 from the synchronization register 602 as part of a polling operation of the second status communication 606-2. Based on the value 604, the display controller 504 determines that the requested new display line or lines 608 of the decompressed display frame 114D are present and available in the video buffer 206. The display controller 504 therefore retrieves the at least one display line 608 from the video buffer 206 via the system bus 218 or the video bus 220 in accordance with the value 604. In response to the retrieval, the display controller 504 writes another value 604 into the synchronization register 602 as part of an update operation of the second status communication 606-2 to indicate a retrieval status of the corresponding display line or lines 608. The other value 604 thus indicates that the at least one corresponding display line 608 has been read out of the video buffer 206.

[0072] Periodically, or as part of certain operations, the frame compressor-decompressor controller 304 polls the synchronization register 602 as part of the first status communication 606-1. The frame compressor-decompressor controller 304 checks the other value 604 to determine whether the corresponding display line or lines 608 have been accessed and retrieved by the display controller 504. If so, the frame compressor-decompressor controller 304 knows that the storage location of the corresponding display line or lines 608 can be overwritten with new data.

[0073] The value 604 or another value 604 can include, for example, one or more bits. The value 604 and the other value 604 can correspond to different storage locations or the same storage location of the synchronization register 602. Although described with reference to the video decoder 502 and the display controller 504, these principles are generally applicable to the core 110. By polling and updating the value 604 in the synchronization register 602, a memory-mapped transfer mechanism can be used to control the routing of the decompressed display frame 114D.

[0074] Figure 7 An example scheme 700 is shown for managing requests 116 issued by a plurality of cores for decompressed frames by the frame compressor-decompressor controller 304. In this example, the video decoder 502 is integrated with the frame decompressor 108-2 and the frame compressor-decompressor controller 304. The frame compressor-decompressor controller 304 is represented by the request manager 402, the request queue 404, and the priority manager 406 (also shown in Figure 4

[0075] Certain requests for decompressed frames can be more important or time- relevant than other requests. For example, a request related to real-time presentation of streaming video can be more important to an end user than a request for a video transcoding operation that can be performed in the background at a best-efforts rate. Thus, the former video-related purpose request can take priority over the latter video-related purpose request. To this end, the priority 706 can be associated with the request, the core, the frame, and so on.

[0076] In example implementations, the requestor identification 702 identifies the requesting core 110, such as the video decoder 502 or the display controller 504 of Figure 5 and 6 The frame identification 704 identifies the requested decompressed frame 114, for example, by a frame number, a time index, or an offset thereof. The requestor identification 702 can be implemented using a specific voltage value on one or more bit lines based on the origin or bus used to provide the request 116 (e.g., if a dedicated bus is provided for each core accordingly), an assigned identification value transferred with the request 116. The frame identification 704 can be explicitly included in each request 116, can be implicitly included in the request 116 (e.g., if the next consecutive frame is needed), can be included in a previous request as part of a frame range, and so on.

[0077] ​In operation, the request manager 402 receives a request 116 from the core 110, e.g., via the frame decompressor controller service interface 408. The request manager 402 inserts the request 116 into the request queue 404. In one approach, the request manager 402 operates the request queue 404 in a first-in, first-out (FIFO) manner. Alternatively, the request manager 402 can insert each request 116 into the request queue 404 based on a corresponding priority 706. The corresponding priority 706 of each individual request 116 can be included or can accompany the corresponding individual request 116. Alternatively, the priorities 706 can be explicitly assigned in bulk to a group of requests 116, or each core 110 can be associated with a particular priority 706 that is implicitly assigned to each request 116 issued by the associated core 110.

[0078] As shown, each request 116 is stored in the request queue 404 in association with a corresponding priority 706, such that requests 116 having different priorities 706 are stored in the same request queue 404. Alternatively, the frame compressor-decompressor controller 304 can implement the illustrated request queue 404 using multiple request queues 404, each request queue 404 being associated with a corresponding priority 706. Thus, each request 116 is stored in a respective request queue 404 associated with the respective priority 706 corresponding to the request 116.

[0079] Regardless of the approach used to implement the illustrated request queue 404, the priority manager 406 orders service to the plurality of requests 116 according to the plurality of priorities 706 that respectively correspond to the plurality of requests 116. Thus, the priority manager 406 extracts the next request 116 using a priority scheme that is based on the relative priorities between two or more requests 116. The priority manager 406 forwards the next request 116 to the frame decompressor 108-2. The frame decompressor 108-2 retrieves the compressed frame 112 corresponding to the requested decompressed frame 114. Continuing with the example scenario of Figure 5 and 6 The frame decompressor 108-2 retrieves the compressed display frame 112D and performs a decompression operation to produce the decompressed display frame 114D. The frame compressor-decompressor controller interface 408 causes the frame decompressor 108-2 to route the decompressed display frame 114D to the video decoder 502 as part of the response 118.

[0080] Figure 8 The frame compressor-decompressor controller 304 is shown establishing a shared frame compression resource (e.g., the frame compressor 108-1 (e.g., Figure 3-1 )) or a frame decompression resource (e.g., the frame decompressor 108-2 (e.g., Figure 3-2Example schemes of at least one time-sharing protocol 800. The time-sharing protocol handler 412 of the frame compressor-decompressor controller 304 (not explicitly shown) can establish at least one example time-sharing protocol 800-1, 800-2, or 800-3. As shown by arrow 812, time increases in the right direction. Each example time-sharing protocol 800 is described in terms of sharing frame decompression resources between two cores; however, the disclosed time-sharing protocols 800 can be applied to sharing frame decompression resources among more than two cores. While Figure 8 While described in terms of time-sharing frame decompression, these principles are also applicable to implementing time-sharing protocols for frame compression.

[0081] With the first time-sharing protocol 800-1, time is divided into a plurality of time slots. These time slots have a fixed time, and each respective time slot is assigned to a respective core. Time slot 802-1 corresponds to a first core, e.g., the video decoder 502. Time slot 802-2 corresponds to a second core, e.g., the display controller 504. In operation, the time-sharing protocol handler 412 rotates access to the frame decompression resources between the two cores over the prescribed length of two different time slots. In the first time-sharing protocol 800-1, interrupts are not enabled.

[0082] In contrast, interrupts 806 are enabled for the second time-sharing protocol 800-2. Here, the frame decompression resources are generally reserved for one of the two clients, in this case the first core. Thus, time slots 804-1 correspond to the first core and can extend indefinitely. These time slots 804-1 extend until the time-sharing protocol handler 412 receives an interrupt 806. The core 110 can signal the interrupt 806 via a message on the system bus, using a dedicated video bus, by driving an interrupt line high, etc. In response to detecting the interrupt 806, the time-sharing protocol handler 412 initiates a context switch to transfer access to the frame decompression resources to a different core and starts another time slot.

[0083] As shown in the second time-sharing protocol 800-2, the second core issues an interrupt 806a. The time-sharing protocol processor 412 detects the interrupt 806a and transfers access to the frame decompressor 108-2 to the second core that initiated the interrupt. Therefore, the frame decompressor 108-2 performs at least one decompression operation to generate a decompressed frame 114 for the requesting second core during time slot 804-2 following the interrupt 806a. The length of each time slot 804-2 can be fixed, depending on the number of requests queued for the second core, the priority of each queued request, etc. After the first core is re-authorized to access the frame decompression resources for another time slot 804-1, the second core can subsequently issue another interrupt 806b. Alternatively, each time slot 804-2 allocated to the second core can continue until another core issues an interrupt (not shown). Therefore, in this alternative, a context switch can occur in response to interrupt 806 and is independent of elapsed time.

[0084] Using the third time-sharing protocol 800-3, time is divided into multiple time slots. The third time-sharing protocol 800-3 is a hybrid of the first and second time-sharing protocols. Some time slots, such as time slot 808-1, have fixed, uninterruptible durations. Other time slots, such as time slot 810-1, have variable durations that can be interrupted by another core. As shown, the first core is allocated to time slots 808-1 and 810-1. An interrupt 806c from the second core arrives during time slot 808-1. However, the time-sharing protocol handler 412 ignores (or queues) this interrupt 806c. On the other hand, an interrupt 806d arriving during time slot 810-1 triggers a context switch. Using the context switch, after interrupt 806d, frame decompression resources are allocated to the second core's persistent time slot 804-2. If interrupt 806c is queued (instead of being dequeued) during time slot 808-1, then in response to interrupt 806c, a context switch can be initialized at the end of time slot 808-1 or at the beginning of time slot 810-1.

[0085] Having already described in general the technologies and environments in which video frame codec architectures can be implemented, the discussion now turns to example methods.

[0086] Example Method

[0087] The following discussion describes methods for operating video frame codec architectures. These methods can be implemented using the examples described above, such as... Figure 1 and Figure 3 The electronic device 102 and PCB 104 shown, and Figure 10 The described electronic device 1000. Aspects of these methods are... Figure 9The flowchart is described in terms of operations 902-912 performed by one or more components of at least one IC chip, as shown. The order in which the operations of these methods are illustrated and / or described is not intended to be construed as a limitation, and any number or combination of the described method operations can be combined in any order to implement the methods, or alternate methods. Although Figure 9 The flowchart is described in terms of operations 902-912 performed by one or more components of at least one IC chip, as shown. The order in which the operations of these methods are illustrated and / or described is not intended to be construed as a limitation, and any number or combination of the described method operations can be combined in any order to implement the methods, or alternate methods. Although

[0088] Figure 9 An example method 900 for operating a video frame codec architecture is shown, which can be implemented by circuitry providing frame decompression services to a plurality of different cores. At 902, a first request for a first decompressed frame is accepted from a first core. For example, frame compressor-decompressor controller 304 (e.g., its frame decompressor controller 304-2) can accept first request 116 for first decompressed frame 114 from first core 110-1. The accepting can be performed via system bus 218, video bus 220, synchronization register 602, etc.

[0089] At 904, a first compressed frame is decompressed to produce the first decompressed frame. For example, frame decompressor 108-2 can decompress first compressed frame 112 to produce first decompressed frame 114. The servicing of first request 116 can be performed according to the associated priority 706.

[0090] At 906, the first decompressed frame is provided to the first core in response to the first request. For example, frame compressor-decompressor controller 304 can provide first decompressed frame 114 to first core 110-1 in response to first request 116. Routing mechanism 500 can be used to transfer first decompressed frame 114 from frame decompressor 108-2 to first core 110-1.

[0091] At 908, a second request for a second decompressed frame is accepted from a second core. For example, frame compressor-decompressor controller 304 can accept second request 116 for second decompressed frame 114 from second core 110-2. First core 110-1 can include display controller 504, and second core 110-2 can include video decoder 502.

[0092] At 910, the second compressed frame is decompressed to produce a second decompressed frame. For example, frame decompressor 108-2 can decompress the second compressed frame 112 to produce the second decompressed frame 114. Decompression can be performed using a lossless compression / decompression algorithm. At 912, the second decompressed frame is provided to the second core in response to the second request. For example, frame compressor-decompressor controller 304 can provide the second decompressed frame 114 to the second core 110-2 in response to the second request 116. After obtaining the second decompressed frame 114, the second core 110-2 can process the frame according to the corresponding video-related function.

[0093] The foregoing discussion describes methods related to a video frame codec architecture. Aspects of these methods can be implemented in, for example, hardware (e.g., fixed logic circuitry), firmware, or some combination thereof. These techniques can employ Figure 1 to Figure 8 and Figure 10 one or more entities or components (electronic device 1000 is described below in Figure 10 FIGS. 1-3) that can be further divided, combined, or the like. Accordingly, these figures illustrate some of the many possible systems or apparatuses that can employ the described techniques. The entities and components of these figures generally represent firmware, hardware, IC chips, circuitry, or a combination thereof.

[0094] Example Electronic Device

[0095] Figure 10 Various components of an example electronic device 1000 are shown, which can implement a video frame codec architecture in accordance with one or more implementations described with reference to any of the previous Figure 1 to Figure 9 Electronic device 1000 can be implemented as any one of a fixed or mobile device, or a combination thereof, in any form of a consumer, computer, portable, user, server, communication, telephone, navigation, gaming, audio, camera, messaging, media playback, and / or other type of electronic device 102 (e.g., a smartphone as shown in FIG. 1). Figure 1

[0096] Electronic device 1000 includes a communication transceiver 1002 that allows wired and / or wireless communication of device data 1004, such as receiving data, transmission data, or other information as described above. Example communication transceiver 1002 includes an NFC transceiver, a WPAN radio compliant with various IEEE 802.15 (Bluetooth TM ) standards, a WLAN radio compliant with various IEEE 802.11 (WiFi TM ​WLAN radios of any of the standards, WWAN (compliant with 3GPP) radios for cellular telephony, wireless metropolitan area network (WMAN) radios compliant with various IEEE 802.16 (WiMAX TM ) standards, and wired local area network (LAN) Ethernet transceivers.

[0097] The electronic device 1000 can also include one or more data input ports 1006 via which any type of data, media content, and / or other inputs can be received, such as user-selectable inputs, messages, applications, music, television content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source. The data input ports 1006 can include USB ports for flash memory, DVD, CD, and the like, a coaxial cable port, and other serial or parallel connectors, including internal connectors. These data input ports 1006 can be used to couple the electronic device to components, peripherals, or accessories, such as a keyboard, microphone, or camera.

[0098] The electronic device 1000 of this example includes at least one processor 1008 (e.g., any one or more of an application processor, a microprocessor, a digital signal processor (DSP), a controller, etc.) that can include a combined processor and memory system (e.g., implemented as part of an SoC) that processes (e.g., executes) computer-executable instructions stored in a computer-readable medium to control the operation of the device. The processor 1008 can be implemented as an application processor, an embedded controller, a microcontroller, an SoC, etc. In general, a processor or processing system can be implemented with hardware having circuitry such as integrated circuits or

[0099] Alternatively, or additionally, the electronic device 1000 can be implemented with any one or combination of electronic circuits that can include software, hardware, firmware, or fixed logic circuitry that combine on a processing and control circuit to implement, which is generally designated 1010 (as electronic circuit 1010). This electronic circuit 1010 can implement executable or hardware-based modules (not shown) through logic circuitry and / or hardware, such as FPGAs, etc.

[0100] Although not shown, the electronic device 1000 can include a system bus, interconnect, crossbar, or data transfer system coupling the various components within the device. The system bus or interconnect can include any of a variety of bus structures, including a memory bus or memory controller, a peripheral bus, a serial bus, a general purpose serial bus, and / or a processor or local bus that utilizes any of a variety of bus architectures including a bus architecture owned by or licensed to Apple Inc.

[0101] The electronic device 1000 also includes one or more storage devices 1012, examples of which include random access memory (RAM), non-volatile memory (e.g., read-only memory (ROM), flash memory, EPROM, and EEPROM), and a disk storage device. The storage device(s) 1012 provide data storage mechanisms to store the device data 1004, other types of code and / or data, and device applications 1020 (e.g., software applications or programs) such as an operating system 1014. For example, the operating system 1014 can be saved as software instructions in the memory device 1012 and executed by the processor 1008.

[0102] As shown, the electronic device 1000 also includes an audio and / or video processing system 1016 that processes audio data and / or transmits audio and video data to an audio system 1018 and / or a display system 1022 (e.g., a video buffer or screen of a smartphone or camera). The audio system 1018 and / or the display system 1022 can include any device that processes, stores and / or displays audio, video, display and / or image data. Display data and audio signals can be communicated to an audio component and / or a display component via an RF (radio frequency) link, S-video link, HDMI (high-definition multimedia interface), composite video link, component video link, DVI (digital video interface), analog audio connection, or other similar communication link, such as a media data port 1024. In some embodiments, the audio system 1018 and / or the display system 1022 are external or separate components to the electronic device 1000. Alternatively, the display system 1022 can be an integrated component of the example electronic device 1000, such as part of an integrated touch interface.

[0103] Figure 10 The electronic device 1000 is an example of an electronic device 102 Figure 1 The processor 1008 is an example of an integrated circuit 106, a processor 202, or a SoC 302 (e.g., respectively, an Figure 1 , Figure 2 and Figure 3 ). Thus, as Figure 10As shown, the processor 1008 can include a frame compressor-decompressor 108, a plurality of cores 110-1 through 110-n, and a frame compressor-decompressor controller 304. In Figure 10 each of the video buffer 206, shared local cache 208, and main memory 210 (e.g., also Figure 2 , Figure 3 , Figure 5 or Figure 6 each of which are depicted as forming at least a portion of a memory system, e.g., by being part of one or more memory devices 1012. As noted above, one or more of these memory components can be part of the same integrated circuit as the processor 1008 (e.g., the video buffer 206 and shared local cache 208 are part of the SoC 302 in Figure 3 ), or alternatively located on a separate die, or even disposed on a different printed circuit board. Thus, the principles of the video frame codec architecture described herein can be implemented by an electronic device 1000 of Figure 10 , or in conjunction with an electronic device 1000 of Figure 10 .

[0104] Although implementations of the video frame codec architecture have been described in language specific to certain features and / or methods, the subject of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of the video frame codec architecture.

Claims

1. A method for sharing frame decompression circuitry, the method performed by the frame decompression circuitry and comprising: accepting, from a video decoder, a first request for a first decompressed frame; decompressing a first compressed frame to produce the first decompressed frame; providing, to the video decoder, the first decompressed frame in response to the first request; accepting, from a display controller, a second request for a second decompressed frame; decompressing a second compressed frame to produce the second decompressed frame; and providing, to the display controller via a video buffer, the second decompressed frame in response to the second request, providing the second decompressed frame including propagating one or more display lines from the video buffer to the display controller based on at least one value in a synchronization register.

2. The method of claim 1, wherein: the first decompressed frame comprises a decompressed reference frame; and the second decompressed frame comprises a decompressed display frame.

3. The method of claim 2, wherein: the frame decompression circuitry comprises a frame decompressor and a frame decompressor controller. providing the second decompressed frame to the display controller comprises:

4. The method of claim 1, wherein, providing the second decompressed frame to the display controller via a video bus extending at least between the frame decompression circuitry and the display controller.

5. The method of claim 1, further comprising: decoding, by a stream decoder of the video decoder, a video stream using the video buffer. providing the second decompressed frame to the display controller via the video buffer further comprises:

6. The method of claim 1, wherein, updating, by the frame decompression circuitry, the at least one value in the synchronization register to indicate that at least one display line has been stored in the video buffer; and polling, by the display controller, the at least one value in the synchronization register to determine that the at least one display line has been newly stored in the video buffer. providing the second decompressed frame to the display controller via the video buffer further comprises:

7. The method of claim 1, wherein, updating, by the display controller, the at least one value in the synchronization register to indicate that at least one display line has been retrieved from the video buffer; and polling, by the frame decompression circuitry, the at least one value in the synchronization register to determine that the at least one display line can be overwritten by new data in the video buffer.

8. The method of claim 1, further comprising: decoding, by a stream decoder of the video decoder, a video stream to produce a plurality of decoded frames; and compressing, by a frame compressor, the plurality of decoded frames to produce a plurality of compressed frames, the plurality of compressed frames including the first compressed frame and the second compressed frame.

9. The method of claim 8, wherein: decompressing the first compressed frame to produce the first decompressed frame comprises decompressing, by a frame decompressor, the first compressed frame at a frame level to produce the first decompressed frame; and ​ ​ decompressing the second compressed frame to produce a second decompressed frame includes decompressing, by the frame decompressor, the second compressed frame at a frame level to produce the second decompressed frame.

10. The method of claim 1, further comprising: sharing the frame decompression circuitry by ordering decompression of the first compressed frame and decompression of the second compressed frame based on a first priority associated with the first request and a second priority associated with the second request.

11. The method of claim 1, further comprising: sharing the frame decompression circuitry between at least the video decoder and the display controller using time slots.

12. The method of claim 11, further comprising: rotating access to the frame decompression circuitry between at least the video decoder and the display controller based on respective prescribed lengths of respective time slots.

13. The method of claim 11, further comprising: switching access to the frame decompression circuitry between at least the video decoder and the display controller in response to receiving at least one interrupt.

14. The method of claim 3, further comprising: managing the first request and the second request by the frame decompressor controller using at least one request queue.

15. An electronic device for sharing frame decompression circuitry, the electronic device comprising: the frame decompression circuitry configured for: means for accepting, from a video decoder, a first request for a first decompressed frame; means for decompressing a first compressed frame to produce the first decompressed frame; means for providing the first decompressed frame to the video decoder in response to the first request; means for accepting, from a display controller, a second request for a second decompressed frame; means for decompressing a second compressed frame to produce the second decompressed frame; and means for providing the second decompressed frame to the display controller in response to the second request; means for buffering video to propagate the second decompressed frame to the display controller; and means for synchronizing propagation of the second decompressed frame.

16. The electronic device of claim 15, further comprising: means for decoding a video stream to produce a plurality of decoded frames; and means for compressing the plurality of decoded frames to produce a plurality of compressed frames, the plurality of compressed frames including the first compressed frame and the second compressed frame.

17. An electronic device for sharing frame decompression circuitry, the electronic device comprising: at least one integrated circuit configured to: decode, by a video decoder, a video stream to produce a plurality of decoded frames, the video decoder including a frame decompressor and a frame decompressor controller; accept, at the frame decompressor controller, from the video decoder, a first request for a first decompressed frame; decompress, by the frame decompressor, a first compressed frame to produce the first decompressed frame, the first compressed frame produced from the plurality of decoded frames; provide the first decompressed frame to the video decoder in response to the first request; and provide the first decompressed frame to the video decoder in response to the first request. ​ ​ at the frame decompressor controller, accepting a second request for a second decompressed frame from a display controller; decompressing, by the frame decompressor, a second compressed frame to produce the second decompressed frame, the second compressed frame being produced from the plurality of decoded frames; in response to the second request, providing the second decompressed frame to the display controller; storing, by the frame decompressor, at least a portion of the second decompressed frame in a video buffer; and writing, by the frame decompressor controller, at least one value indicative of a storage status of the portion of the second decompressed frame into a synchronization register.

18. The electronic device of claim 17, wherein, the at least one integrated circuit is configured to: provide the second decompressed frame from the video buffer to the display controller using a video bus coupled to the display controller.

19. The electronic device of claim 17, wherein, the at least one integrated circuit is configured to: retrieve, by the display controller, the portion of the second decompressed frame from the video buffer in response to the at least one value indicative of the storage status of the portion of the second decompressed frame; and write, by the display controller, at least another value indicative of a retrieval status of the portion of the second decompressed frame into the synchronization register.

20. A method for sharing frame decompression circuitry, the method comprising: accepting a first request for a first decompressed frame from a video decoder; decompressing, by the frame decompression circuitry, a first compressed frame to produce the first decompressed frame; in response to the first request, providing the first decompressed frame to the video decoder; accepting a second request for a second decompressed frame from a display controller; decompressing, by the frame decompression circuitry, a second compressed frame to produce the second decompressed frame; in response to the second request, providing the second decompressed frame to the display controller; sharing the frame decompression circuitry between at least the video decoder and the display controller using time slots; and in response to receiving at least one interrupt, switching access to the frame decompression circuitry between at least the video decoder and the display controller.

Images