Video Frame Codec Architecture

Abstract

To provide methods and electronic devices for video frame codec architectures.SOLUTION: A frame decompressor (FCD: Frame Compressor-Decompressor) included in a printed circuit board 104 decompresses compressed frames to produce decompressed frames. An FCD controller arbitrates shared access to the FCD. Multiple cores of a SoC request to receive a decompressed frame from the FCD via the FCD controller. The FCD controller can implement a request queue and can order servicing of requests on the basis of priority of the requests or requesting cores. The FCD controller can also establish a time-sharing protocol for access by the multiple cores. In some implementations, a video decoder is logically integrated with the FCD and stores portions of the decompressed frame in a video buffer, and a display controller retrieves the portions for display using a synchronization mechanism.SELECTED DRAWING: Figure 3

Description

[Background technology]

[0001] Inventor Ville-Mikko Rautio Aki Kuusela background Electronic devices play an essential role in manufacturing, communications, healthcare, commerce, social interaction, and entertainment. For example, electronic devices power server farms that provide cloud-based distributed computing functionality for commerce and communications. Computationally capable devices are also embedded in many different kinds of modern equipment, from medical instruments to machines and from vehicles to industrial tools. Moreover, one electronic device - the smartphone - has become a necessity that is literally always at hand.

[0002] Many electronic devices, such as those with cameras or display screens, manipulate video data. For example, video may be captured using a security camera and then enhanced to improve certain visual aspects such as clarity or contrast. Existing video data may also be manipulated to improve the appearance of individual video frames and presented on the display screen of a smartphone or television monitor. For example, video data from a movie may be processed to make artificial graphics images more realistic or to increase the display resolution. Video image data is also manipulated in industrial and medical environments. For example, image data from a three-dimensional body scan may be stitched together and presented as a video for review and analysis by a physician.

[0003] In any of these situations, manipulating the video data is a processing-intensive task. This is due in part to the size or amount of information typically found in the video data. As a result, the area of ​​an integrated circuit (IC) chip devoted to handling video data can be greater than that for other types of data. The difficulty of handling video data is exacerbated by the ever-increasing display resolutions of video that electronic devices are expected to handle. For example, high definition (HD) video contains approximately four times more video data than standard definition (SD) video, and ultra HD (UHD) or 4K video contains approximately four times more video data than HD video.

[0004] Thus, the amount of video data that electronic devices are expected to process has increased dramatically over the past decade. The demand for video data processing is expected to grow even more in the coming years as the use of virtual reality (VR) and artificial reality (AR) becomes more common. Thus, manufacturers of electronic devices are striving to improve the capabilities of electronic devices to handle the ever-increasing amounts of video data.

[0005] This background discussion is provided to generally present the context of the disclosure. Unless otherwise stated herein, no admission is made, expressly or impliedly, that any material described herein is prior art to the present disclosure or the appended claims. Summary of the Invention [Problem to be solved by the invention]

[0006] summary Techniques and apparatus for a video frame codec architecture are described that allow integrated circuit (IC) chips to perform fewer frame compressions while also facilitating an efficient workflow for upgrading to newer frame compression / decompression techniques, including lossless techniques. This allows high bandwidth video data to be processed using a small amount of circuitry resources, eliminating the need to include multiple separate frame reconstruction units in multiple different cores of an IC chip. Instead, the frame reconstructor can provide frame reconstruction services to multiple different cores that function as frame reconstructor client circuits. The frame reconstructor controller facilitates sharing of reconstruction services using a queue or priority mechanism that orders reconstruction requests received from one or more of the multiple cores. The frame reconstructor controller allocates frames according to a time-sharing protocol. The frame decompressor may also arbitrate access to the frame decompressor. In an exemplary implementation, the frame decompressor is co-located with the video decoder client circuitry, with the frame decompressor being shared with at least the display controller. Similarly, the frame compressor may provide frame compression services to multiple different cores functioning as frame compressor client circuits. A frame compressor controller facilitates sharing the compression services with multiple cores. Additionally, both the frame compression and frame decompression services may be provided in a single IC chip and shared across multiple cores. [Means for solving the problem]

[0007] Aspects described below include an electronic device including a frame restorer and a frame restorer controller. The frame restorer is configured to restore a plurality of compressed frames to generate a plurality of restored frames. The frame restorer controller is coupled to the frame restorer and configured to arbitrate access to the frame restorer for a plurality of cores. The plurality of cores includes a first core and a second core. The first core is coupled to the frame restorer controller and configured to obtain, via the frame restorer controller, a certain restored frame of a plurality of restored frames generated by the frame restorer. The second core is coupled to the frame restorer controller and configured to obtain, via the frame restorer controller, another restored frame of a plurality of restored frames generated by the frame restorer. For example, the first core may be configured to obtain, via the frame restorer controller, a first restored frame of a plurality of restored frames generated by the frame restorer. The second core may be configured to obtain, via the frame restorer controller, a second restored frame of a plurality of restored frames generated by the frame restorer.

[0008] Aspects described below also include a method for sharing frame decompression circuitry among multiple cores. The method comprises accepting a first request for a first decompressed frame from a first core. The method also comprises decompressing the first compressed frame to generate the first decompressed frame. The first decompressed frame is provided to the first core in response to the first request. The method additionally comprises accepting a second request for a second decompressed frame from a second core. The method further comprises decompressing the second compressed frame to generate a second decompressed frame. The second decompressed frame is provided to the second core in response to the second request.

[0009] An aspect described below includes another electronic device including a video decoder and a display controller. The video decoder is configured to decode a video stream to generate a plurality of decoded frames. The video decoder includes a frame compressor, a frame decompressor, and a frame decompressor controller. The frame compressor is configured to compress the plurality of decoded frames to generate a plurality of compressed frames. The frame decompressor is configured to decompress the plurality of compressed frames to generate a plurality of decompressed frames. The frame decompressor controller is coupled to the frame decompressor and configured to arbitrate access to the frame decompressor. The display controller is coupled to the frame decompressor controller. The display controller is configured to obtain, via the frame decompressor controller, a decompressed frame of the plurality of decompressed frames generated by the frame decompressor.

[0010] Aspects described below also include a system including a frame restorer and a plurality of cores. The frame restorer is configured to restore a plurality of compressed frames to generate a plurality of restored frames. The plurality of cores includes a first core and a second core. The first core is coupled to the frame restorer and configured to obtain a restored frame of the plurality of restored frames. The second core is coupled to the frame restorer and configured to obtain another restored frame of the plurality of restored frames. For example, the first core may be configured to obtain a first restored frame of the plurality of restored frames. The second core may be configured to obtain a second restored frame of the plurality of restored frames. The system also includes control means for controlling the frame restorer to arbitrate access to the frame restorer for the plurality of cores including the first core and the second core. Additionally or alternatively, the system may include a frame compressor configured to compress a plurality of uncompressed frames to generate a plurality of compressed frames. The first and second cores may each obtain a respective compressed frame of the plurality of compressed frames. Thus, the system may also include control means for controlling the frame compressor to arbitrate access to the frame compressor for a plurality of cores, including the first core and the second core.

[0011] Apparatus and techniques for implementing a video frame codec architecture are described with reference to the following drawings, in which like numbers are used throughout to reference like features and components. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an exemplary environment including a printed circuit board in which a video frame codec architecture can be implemented. [Diagram 2] FIG. 2 illustrates another aspect of an exemplary environment in which a video frame codec architecture can be implemented. [Diagram 3]FIG. 2 illustrates a system-on-chip (SoC) having an exemplary implementation of a video frame codec architecture including a frame compressor-decompressor, a frame compressor-decompressor controller, and multiple cores. [Figure 3-1] A diagram showing an SoC having an example implementation of a video frame codec architecture including a frame compressor, a frame compressor controller, and multiple cores. [Figure 3-2] A diagram showing an SoC having an example implementation of a video frame codec architecture including a frame restorer, a frame restorer controller, and multiple cores. [Figure 4] FIG. 2 illustrates an exemplary frame compressor-decompressor controller associated with a frame decompressor and a core. [Diagram 5] FIG. 1 illustrates an example approach to implementing a video frame codec architecture in which multiple cores include a video decoder and a display controller. [Figure 6] 1 illustrates an example technique for routing reconstructed display frames from a video decoder to a display controller. [Figure 7] FIG. 2 illustrates an exemplary scheme by a frame decompressor controller for managing requests for decompressed frames issued by multiple cores. [Figure 8] FIG. 1 illustrates an exemplary scheme by a frame compressor-decompressor controller to establish a time-sharing protocol for sharing frame compression or frame decompression resources. [Figure 9] FIG. 2 illustrates an exemplary method for operating a video frame codec architecture as described herein. [Figure 10] A diagram illustrating various components of an example electronic device that can implement a video frame codec architecture in accordance with one or more implementations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Detailed Description overview Movie or other video data consumes a significant amount of bandwidth, both for storage at rest and for transmission as it propagates between electronic devices or their internal components. As video display resolutions increase, so does the bandwidth requirement. For example, Ultra HD (UHD) or 4K video has about 15-20 times more video data to handle than the SD video commonly used only a decade ago. This increased amount of data makes it difficult to manage the video even within a single integrated circuit (IC) chip. For example, a typical system-on-chip (SoC) has a system bus that can become overloaded if the system bus is used to carry raw decoded video data between different SoC components.

[0014] As is known, the video bandwidth can be reduced by encoding the video using some lossy codec, for example H.264. The encoded video can then be streamed from a cloud server to a tablet computer, for example, or stored on a Blu-ray disc or in a flash memory. For example, an end-user electronic device is then responsible for decoding the video and presenting it on a display screen. Decoding requires converting a stream of binary 1's and 0's or bits into individual decoded video frames that can be displayed in sequence to represent the video. As part of this video encoding and decoding procedure, the video data is decompressed. With this video-level procedure (for example, using the H.264 codec), the amount of compression is high, but the compression is also lossy. Thus, due to the video-level compression / decoding procedure, some video information is lost to the extent that the picture quality may be noticeably altered.

[0015] This loss of video information is acceptable to allow large amounts of video data to be transmitted between devices or stored using a significant amount of memory. After the electronic device decodes the video stream to generate decoded video frames, this large amount of video data is recreated frame by frame, either as a group of frames or as a number of individual frames. Each of these individual frames is still a significant amount of data. To handle this amount of video data in the electronic device, if the decoded video frame is not currently being processed or displayed, it is compressed to generate a compressed video frame. Since a finite number of frames are decoded from the stream at any given time, in some implementations a lossless compression algorithm can be applied to the decoded frames. A lossless decompression frame-level procedure can prevent any further image degradation. The compressed video frames take up less space in memory and consume less bandwidth on wiring such as the system bus of the SoC. Thus, compressed video frames can be transferred between different chips on a printed circuit board (PCB), between the SoC and main memory, or even between different components of a single SoC.

[0016] To enable existing systems to use per-video-frame or frame-level compression, each respective component or core of an SoC that works with video data includes a respective decompression unit. Using this simple conventional approach, each core can independently generate decompressed video frames from compressed video frames and then process the decompressed video frames according to the video-related purpose of the given core. However, this simple approach has several associated costs. First, significant area on the IC die is devoted to dual video frame decompression or compression circuitry. Second, the display path for presenting the video on the display screen includes a separate decompression unit. Third, the workflow for developing advanced video frame compression and decompression techniques for an SoC is highly complex. This complexity is due to the fact that , which stems from multiple different cores, each containing a separate decompression unit or a separate compression unit (and in some cases both). In other words, to upgrade to a newer, more efficient decompression algorithm, each core containing a compression or decompression unit must be modified simultaneously and then reintegrated with the rest of the SoC. Thus, the complexity of this workflow upgrade slows the adoption of improved compression / decompression algorithms in multicore chips.

[0017] In contrast, certain implementations described herein use a shared frame compressor-decompressor. The frame compressor-decompressor may include a frame compressor or a frame decompressor, or both. Multiple cores of an IC chip, such as an SoC, may obtain a decompressed version of a compressed frame from the shared frame decompressor, for example. A given core may make a request for a decompressed frame, and the frame decompressor may provide a response that includes the requested decompressed frame. In this manner, the frame decompressor may function to provide a frame decompression service, and each core may function as a client to the frame decompression service. Similarly, the frame compressor may function to provide a frame compression service, and each core may function as a client to the frame compression service. These frame-level compression and decompression services may use lossless algorithms.

[0018] In one implementation, the SoC includes different components, such as a video decoder and a display controller operating as different cores. The video decoder includes a stream decoder that decodes a video stream to generate decoded frames. The video decoder also includes a frame compressor that generates compressed frames from the decoded frames. The different components of the SoC are configured to process the reconstructed frames. For example, the video decoder may use the reconstructed reference frames for further decoding of the video stream. In addition, the display controller may present a video on a display screen using the reconstructed display frames.

[0019] To obtain any of these exemplary types of reconstructed frames, the frame reconstructor reconstructs the corresponding compressed frame. The frame reconstructor may route the requested reconstructed frame to the requesting core using one or more buses, at least one buffer, or some other routing mechanism. The frame reconstructor controller arbitrates shared access to the frame reconstructor between at least two cores, such as a video decoder and a display controller. The frame reconstructor controller may manage a request queue of compressed frame requests. The management may require ordering of frame request processing according to a priority scheme. The frame reconstructor controller may also establish a time-sharing protocol for access to the frame reconstructor. The time-sharing protocol may include time slots assigned to different cores, acceptance of interrupts issued by the cores to seize control of the frame reconstructor, etc.

[0020] In these aspects, frame compressor or frame decompressor resources can be shared between two or more client cores that process video data. For example, by sharing a frame decompressor, less area of ​​the SoC is devoted to frame decompression. In addition, there is no need to insert a separate frame decompressor along the video display path. Furthermore, the compression / decompression algorithms are decoupled from the overall SoC architecture. As a result, the compression / decompression algorithms can be updated more easily and therefore more frequently because there are fewer or even one frame compressor or frame decompressor included on the SoC and thus subject to the update workflow.

[0021] Exemplary implementations at various levels of detail are discussed below with reference to the associated figures. The following discussion first describes an exemplary operating environment, and then describes exemplary systems and hardware. , followed by a description of an example method, and concluding with an example electronic device and related example aspects.

[0022] Example Environment FIG. 1 illustrates an exemplary environment 100 including a printed circuit board 104 (PCB) in which a video frame codec architecture may be implemented. As shown, the environment 100 includes an electronic device 102. The electronic device 102 includes at least one PCB 104. The PCB 104 includes one or more integrated circuits, such as an integrated circuit 106 (IC). As described below with reference to FIG. 2 and FIG. 3, the PCB 104 may include other integrated circuits, such as at least one memory, separate from the IC 106. The IC 106 includes at least one frame restorer 108-2, a number of cores 110-1, 110-2...110-n (n represents an integer greater than 1), and a number of video frames, such as at least one compressed frame 112 and at least one restored frame 114. While describing the exemplary environment 100, FIG. 1 illustrates only the frame restorer 108-2 for simplicity. However, frame compressor-decompressor 108 and frame compressor 108-1 are described below with reference to Figures 3 and 3-1, respectively.

[0023] In an exemplary implementation, the frame decompressor 108-2 is communicatively coupled to at least a portion of the plurality of cores 110-1 through 110-n. The frame decompressor 108-2 includes circuitry for decompressing compressed video frames. In this manner, the frame decompressor 108-2 may decompress a compressed frame 112 to generate a decompressed frame 114. In operation, the cores 110 use the frame decompressor 108-2 to obtain a decompressed version of the compressed frame. For example, the nth core 110-n may send a request 116 to the frame decompressor 108-2 indicating that a decompressed version of the identified compressed frame 112 is requested. The frame decompressor 108-2 provides a response 118 that includes the decompressed frame 114. Although reference is made to a video frame as an exemplary context, a frame as described herein may include any frame having visual data, including computer-generated graphics frames, video frames, combined frames, and the like.

[0024] In this manner, frame decompressor 108-2 provides decompression services to individual cores of multiple cores 110-1-110-n. Similarly, frame compressor 108-1 (e.g., of FIG. 3-1) can provide compression services to individual cores of multiple cores 110-1-110-n. In this manner, frame decompressor 108-2 or frame compressor 108-1 (or both) implement at least a portion of frame compression-decompression (FCD) server circuitry 122 of IC 106. Similarly, multiple cores 110-1-110-n implement multiple frame compression-decompression (FCD) client circuitry 120. With this client-server architecture, a frame compressor unit or frame decompressor unit can be shared between two or more cores 110 to conserve space on IC 106 and simplify the workflow for IC 106 to enhance the compression / decompression techniques used to compress / decompress video frames. A more detailed example architecture for IC 106 and PCB 104 is described below with reference to Figures 3, 3-1, and 3-2. However, additional aspects of an example implementation are now described with reference to Figure 2.

[0025] 2 illustrates another aspect of an example environment 200 in which a video frame codec architecture as described herein may be implemented. Shown are various non-limiting example devices, namely electronic devices 102, including a smartphone 102-1, a notebook computer 102-2, a television 102-3, a desktop computer 102-4, a tablet 102-5, and a wearable device 102-6. As shown on the right, the electronic devices 102 include one or more processors 202, one or more computer-readable media 204, and at least one wiring 216. The computer-readable The media 204 may store, retain, or otherwise include code, data, instructions, other information, etc. The electronic device 102 may also include an operating system 212. Although depicted separately, the operating system 212 may be stored on one or more computer-readable media 204.

[0026] An operating system 212, embodied as an application (not shown) or computer readable instructions on computer readable medium 204, is executable by processor 202. The operating system 212 or basic input / output system (BIOS) may include a frame codec parameters module 214. The frame codec parameters module 214 may set one or more parameters to enable, qualify, tune, or otherwise facilitate performance of the shared frame compression and decompression functionality described herein.

[0027] As shown, the computer-readable medium 204 may include at least one video buffer 206, at least one shared local cache 208, and at least one main memory 210. In one implementation, the video buffer 206 and the shared local cache 208 are separate memory blocks on the IC. In another implementation, the video buffer 206 and the shared local cache 208 are part of the same memory block, such as when a portion of the shared local cache 208 is used as the video buffer 206 in a manner where the allocation is dynamically changing or fixed. The wiring 216 may include at least one system bus 218, at least one video bus 220, and at least one external bus 222. In one implementation, the video bus 220 and the system bus 218 are different buses. In another implementation, there is no separate video bus, and thus the video data is propagated in the vicinity of the IC using the system bus 218. Exemplary implementations of these computer-readable medium 204 and these wiring 216 are described below with reference to Figures 4-6.

[0028] Exemplary Components and Techniques 3 illustrates a portion of the PCB 104. The PCB 104 includes a system-on-chip 302 (SoC) and a main memory 210. The SoC 302 illustrates an example implementation of a video frame codec architecture including a frame compressor-decompressor 108 (FCD), a frame compressor-decompressor controller 304, and multiple cores 110-1 through 110-n. The SoC 302 also includes a shared local cache 208 and a system bus 218. The main memory 210 is coupled to the SoC 302 via an external bus 222 that is included as part of (e.g., disposed on) the PCB 104.

[0029] The PCB 104 can be implemented with rigid or flexible materials for mounting or attaching multiple IC chips, wiring, interfaces, etc. The main memory 210 can be implemented with, for example, dynamic random access memory (DRAM), which is periodically refreshed to maintain memory contents, or flash memory, which can maintain memory contents even without power. In general, more energy is consumed to access data stored on the main memory 210 than is consumed to access data stored on the SoC 302, such as the shared local cache 208. The shared local cache 208 can be implemented with, for example, static random access memory (SRAM), DRAM, flash memory, or some combination thereof.

[0030] The system bus 218 interconnects the multiple cores 110-1 to 110-n, the shared local cache 208, the frame compressor-decompressor controller 304, and other components and interfaces (e.g., the frame compressor-decompressor 108 is connected to the system bus). Each core 110 of the multiple cores 110-1 through 110-n may store data in or retrieve data from the shared local cache 208 using the system bus 218. Similarly, each core 110 of the multiple cores 110-1 through 110-n may store data in or retrieve data from the main memory 210 using the external bus 222, such as by also using the system bus 218 or the shared local cache 208. For example, a first core 110-1 may store data in the shared local cache 208 and a second core 110-2 may then retrieve the stored data from the shared local cache 208.

[0031] In an exemplary operation, a frame compressor-decompressor 108 (FCD) processes (e.g., compresses or decompresses) a plurality of unprocessed frames 306 to generate a plurality of processed frames 308. A frame compressor-decompressor controller 304 (FCD controller) is coupled to the frame compressor-decompressor 108 to arbitrate access to the frame compressor-decompressor 108 for the plurality of cores 110-1 to 110-n. Although shown separately, the frame compressor-decompressor 108 and the frame compressor-decompressor controller 304 may be logically integrated together. At least two of the cores 110 of the plurality of cores 110-1 to 110-n are coupled to the frame compressor-decompressor controller 304. In this manner, each core 110 can obtain a processed frame 308 generated by the frame compressor-decompressor 108 from its unprocessed frame version via the frame compressor-decompressor controller 304. The processed frame 308 may be obtained, for example, using the request 116 and the corresponding response 118 .

[0032] The frame compressor-decompressor 108 may include a frame compressor 308-1 (e.g., as depicted in FIG. 3-1), a frame restorer 108-2 (e.g., as depicted in FIG. 3-2), or both. Similarly, the frame compressor-decompressor controller 304 may include a frame compressor controller 304-1 (e.g., as depicted in FIG. 3-1), a frame restorer controller 304-2 (e.g., as depicted in FIG. 3-2), or both. Depending on whether the processing operation by the frame compressor-decompressor 108 is a compression or decompression operation, one of the unprocessed frames 306 or the processed frames 308 corresponds to a compressed frame and the other corresponds to a decompressed frame. If a given implementation includes both a frame compressor 108-1 and a frame restorer 108-2 (and / or both a frame compressor controller 304-1 and a frame restorer controller 304-2), such components may be co-located, located in close proximity to each other, or located at different locations on the IC. For example, each may be located closer to the cores that are likely to be the most common clients of the corresponding compression or decompression service. Furthermore, if a chip includes a co-located frame compressor 108-1 and frame decompressor 108-2, each may include completely separate circuitry, or they may share circuitry. In one implementation, the frame compressor-decompressor 108 can be implemented as a lossless frame data manipulator that adjusts the memory size of a frame's data as part of a lossless compression or lossless decompression operation depending on the requirements 116. Exemplary implementations of the compression and decompression operations are described below with reference to Figures 3-1 and 3-2, respectively.

[0033] FIG. 3-1 illustrates an SoC having an example implementation of a video frame codec architecture including a frame compressor 108-1 (FC), a frame compressor controller 304-1 (FC controller), and multiple cores. The frame compressor 108-1 compresses uncompressed frames (e.g., decompressed frames 114 or frames not yet compressed) to generate compressed frames 112. The frame compressor controller 304-1 is coupled to the frame compressor 108-1 and arbitrates access to the frame compressor 108-1 for the multiple cores 110-1 through 110-n. Although shown separately, the frame compressor 108-1 and the frame compressor controller 304-1 are connected to each other via a frame compressor controller 304-1. The frame compressor controller 304-1 and the frame compressor controller 304-2 may be logically integrated together.

[0034] At least two of the cores 110 of the plurality of cores 110-1 to 110-n are coupled to the frame compressor controller 304-1, where each core 110 may include a component or block that generates visual data. In this manner, each core 110 may obtain a compressed frame 112 generated by the frame compressor 108-1 from its decompressed frame 114 version via the frame compressor controller 304-1. In this manner, the compressed frame 112 may be obtained, for example, using a request 116 and a corresponding response 118. In operation, the frame compressor controller 304-1 grants access to the compression engine of the frame compressor 108-1 to a requesting core 110 on the system bus 218 in a pipelined manner to avoid adding traffic to the external bus 222 that provides access to the main memory 210. In one implementation, the frame compressor controller 304-1 may temporarily grant exclusive access to a requesting core 110 to access the compression resources.

[0035] FIG. 3-2 illustrates an SoC having an exemplary implementation of a video frame codec architecture including a frame restorer 108-2 (FD), a frame restorer controller 304-2 (FD controller), and multiple cores. The frame restorer controller 304-2 is coupled to the frame compressor 108-2 and arbitrates access to the frame restorer 108-2 for multiple cores 110-1-110-n. Although shown separately, the frame compressor 108-2 and the frame compressor controller 304-2 may be logically integrated together. At least two of the cores 110 of the multiple cores 110-1-110-n are coupled to the frame restorer controller 304-2. In this manner, each core 110 can obtain a reconstructed frame 114 generated by the frame restorer 108-2 from its compressed frame 112 version via the frame restorer controller 304-2. In this manner, the reconstructed frame 114 can be obtained, for example, using a request 116 and a corresponding response 118.

[0036] Part of the following description focuses on implementations that can perform at least decompression operations. Thus, each may explicitly include a frame decompressor 108-2 and a frame compressor-decompressor controller 304 or a frame decompressor controller 304-2. However, each implementation may additionally or alternatively include a frame compressor 108-2 that performs compression operations. Furthermore, it will become apparent how compression operations and movement of compressed frames 112 can be accomplished in a manner at least similar to that described for decompression operations with reference to Figures 4-6. Also, the request handling described with reference to Figure 7 and the time sharing protocol described with reference to Figure 8 can be similarly implemented in a similar manner for compression operations.

[0037] At least some of the cores 110 can operate on video data for video-related purposes, including video decoding, video display, video transcoding (e.g., decoding from one coding format using one algorithm and encoding to another coding format using another algorithm), video compensation (e.g., operations on video frames by a graphics processing unit (GPU) to improve or enhance the image), etc. An example implementation in which a first core is implemented as a video decoder and a second core is implemented as a display controller is described below with reference to FIG. 5. However, an example implementation of the frame restorer controller 304 is described first with reference to FIG. 4.

[0038] 4 illustrates, generally at 400, an exemplary frame compressor-decompressor controller 304 associated with a frame decompressor 108-2 and a core 110. The core 110 is an FCD client. The core 110 includes an FCD client interface 410, and the frame decompressor controller 304 includes an FCD service interface 408. The FCD client interface 410 and the FCD service interface 408 together enable communication between the core 110 and the frame compressor-decompressor controller 304. Thus, using these two interfaces, the core 110 can send requests 116 to the frame compressor-decompressor controller 304, and the frame compressor-decompressor controller 304 can send responses 118 to the core 110.

[0039] The FCD client interface 410 and the FCD service interface 408 may communicate over a system bus 218 (not shown), a video bus 218, or the like. The request 116 may be communicated using bit lines, one or more bits in a register, a message organized into at least one packet, or the like. The response 118 including the reconstructed frame 114 may be communicated using a bus coupling the core 110 to the frame reconstructor 108-2, using a memory accessed by both the core 110 and the frame reconstructor 108-2, and some combination thereof. Examples of different interface communications are described below with reference to Figures 5 and 6.

[0040] As shown, the frame compressor-decompressor controller 304 includes a request manager 402, a request queue 404, a priority manager 406, and a time-sharing protocol handler 412. The FCD service interface 408 is coupled to at least the request manager 402, the priority manager 406, and the time-sharing protocol handler 412. An exemplary technique for managing requests, including prioritizing request processing, is described below with reference to Figure 7. An exemplary manner in which the time-sharing protocol handler 412 is used to establish a time-sharing protocol to access the frame decompressor 108-2 is described below with reference to Figure 8.

[0041] 5 illustrates an example approach 500 for implementing a video frame codec architecture in which multiple cores of an SoC include a video decoder 502 and a display controller 504. On the left, the video decoder 502 includes a stream decoder 506 that includes a frame compressor 108-1. However, the frame compressor 108-1 may be implemented separately from the stream decoder 506 or the video decoder 502. On the right, the display controller 504 includes a frame buffer 518. The frame decompressor 108-2, the frame compressor-decompressor controller 304 (FCD controller), and the video buffer 206 are associated with the video decoder 502. For example, these three components may be co-located (e.g., located next to or physically intermixed with) the video decoder 502, may be at least partially logically integrated with the video decoder 502, or they may be some combination thereof. Although described with reference to video decoder 502 and display controller 504, the principles apply generally to core 110. Also, although video buffer 206 is depicted as separate from shared local cache 208, video buffer 206 may alternatively be incorporated as part of or allocated from shared local cache 208.

[0042] In an exemplary implementation, an electronic device obtains a video stream 514, which may include a stream of bits encoded to represent a video. The video stream 514 is comprised of a number of raw frames 516 that are encoded using some codec, such as H.264. A stream decoder 506 receives the video stream 514 via the system bus 218 and decodes the video stream 514 to generate a number of decoded frames 512, one of which is explicitly shown. A frame compressor 108-1 compresses the number of decoded frames 512 to generate a number of compressed frames 112. As shown, the frame compressor 108-1 is integrated with the stream decoder 506. In this manner, the frame compressor 108-1 may be used to decode the number of decoded frames 512. 08-1 may compress the decoded frames 512 as part of the decoding process. However, frame compression may be implemented in alternative manners. These decoded and compressed frames 112 may be stored in the video buffer 206, the shared local cache 208, or main memory 210 for subsequent use by the video decoder 502 or another core.

[0043] The decoded compressed frame 112 may be used for different video related purposes. For example, if the compressed frame is used for further video decoding purposes, the compressed frame 112 may comprise a compressed reference frame 112R. Alternatively, if the other compressed frame is used for video display purposes, the other compressed frame 112 may comprise a compressed display frame 112D. For the video decoding functionality of the video decoder 502, the frame decompressor 108-2 decompresses the compressed reference frame 112R to generate a reconstructed reference frame 114R. The video decoder 502 can continue decoding the video stream 514 using the reconstructed reference frame 114R.

[0044] For the video display functionality of the display controller 504, the frame decompressor 108-2 decompresses the compressed display frame 112D to generate the decompressed display frame 114D. The display controller 504 can request this restoration service from the frame compressor-decompressor controller 304 (e.g., including at least the frame decompressor controller 304-2 of FIG. 3-2) via the frame decompressor controller client interface 410 and the frame decompressor controller service interface 408 (both of FIG. 4). The frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via a routing mechanism 510 (represented by a solid wavy arrow). The display controller 504 can store the decompressed display frame 114D in a frame buffer 518. Thus, the display controller 504 can present an image of the video frame on a display screen using the stored decompressed display frame 114D. Although the frame compressor-decompressor controller 304 of FIG. 5 is described in terms of providing decompression services using frame decompressor 108-2 to multiple cores, the principles also apply to a frame compressor-decompressor controller 304 that provides compression services using frame compressor 108-1 to multiple cores.

[0045] Several exemplary routing mechanisms 510 are illustrated in FIG. 5 using thin dashed lines. The routing mechanisms 510 represent different paths by which the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to a requesting core, such as the video decoder 502 or the display controller 504. In a first routing mechanism 510-1, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to a frame buffer 518 of the display controller 504 via the system bus 218. In a second routing mechanism 510-2, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and the shared local cache 208. For example, the frame compressor-decompressor controller 304 can store the decompressed display frame 114D in the shared local cache 208, and the display controller 504 can retrieve the stored decompressed display frame 114D from the shared local cache 208.

[0046] In the third routing mechanism 510-3, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the system bus 218 and the main memory 210. For example, the frame compressor-decompressor controller 304 may store the decompressed display frame 114D off-chip in the main memory 210 via the system bus 218 and the external bus 222. The display controller 504 may retrieve the stored decompressed display frame 114D from the main memory 210 using the two buses and then store the frame in the frame buffer 518.

[0047] The fourth routing mechanism 510-4 is related to serving the video decoder 502 as a client of the frame reconstruction service. In the fourth routing mechanism 510-4, the frame compressor-decompressor controller 304 provides the reconstructed reference frame 114R to the video decoder 502. To that end, the frame compressor-decompressor controller 304 forwards the reconstructed reference frame 114R to the video buffer 206. The video decoder 502 can access the reconstructed reference frame 114R from the video buffer 206.

[0048] In a fifth routing mechanism 510-5, where the integrated circuit includes a video bus 220 and a system bus 218, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via a video bus 220 that is at least partially separate from the system bus 218. As shown, the video bus 220 is dedicated to the propagation of video data and extends at least between the frame compressor-decompressor controller 304 and the display controller 504. However, the video bus 220 can be implemented differently, such as by carrying other video data to and from other cores.

[0049] In the sixth routing mechanism 510-6, the frame compressor-decompressor controller 304 provides the decompressed display frame 114D to the display controller 504 via the video buffer 206. The frame compressor-decompressor controller 304 stores at least a portion of the decompressed display frame 114D in the video buffer 206. In one approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the system bus 218. In another approach, the display controller 504 retrieves the decompressed display frame 114D from the video buffer 206 using at least the video bus 220 (e.g., if separate from the system bus 218). With either approach, the sixth routing mechanism 510-6 may further require a memory-mapped mechanism for the display controller 504 to access the video buffer 206. An exemplary implementation of the memory-mapped mechanism will now be described with reference to FIG. 6.

[0050] 6 illustrates an example technique 600 for routing a reconstructed display frame 114D from a video decoder 502 to a display controller 504 using a sixth routing mechanism 510-6. In this example, the video decoder 502 is integrated with a frame decompressor 108-2 and a frame compressor-decompressor controller 304. The video decoder 502 includes a synchronization register 602 in addition to a video buffer 206. The video buffer 206 functions as an output buffer in this example. In this manner, the frame compressor-decompressor controller 304 causes the frame decompressor 108-2 to store at least a portion of the reconstructed display frame 114D in the video buffer 206.

[0051] The exemplary technique 600 for the sixth routing mechanism 510-6 involves at least one status communication 606. A first status communication 606-1 occurs between the frame compressor-decompressor controller 304 and the synchronization register 602. A second status communication 606-2 occurs between the display controller 504 and the synchronization register 602. Each status communication 606 may include a polling or update operation and may occur over the system bus 218 or the video bus 220.

[0052] In response to storing at least a portion of the reconstructed display frame 114D in the video buffer 206, the frame compressor-decompressor controller 304 writes a value 604 to a synchronization register 602 as part of an update operation of a first status communication 606-1, the value 604 indicating the status of the storage contents of the video buffer 206, where the value 604 indicates the status of a particular video data, such as one or more display lines 608 of the reconstructed display frame 114D. This indicates that the image has been newly placed in the image buffer 206. 6 can function as a line buffer to hold one or more display lines 608.

[0053] Meanwhile, the display controller 504 extracts a value 604 from the synchronization register 602 as part of the polling operation of the second status communication 606-2. Based on the value 604, the display controller 504 determines that one or more new display lines 608 of the requested reconstructed display frame 114D are present and available in the video buffer 206. Thus, the display controller 504 retrieves at least one display line 608 from the video buffer 206 via the system bus 218 or the video bus 220 according to the value 604. In response to the retrieval, the display controller 504 writes another value 604 to the synchronization register 602 as part of the update operation of the second status communication 606-2 to indicate the status of the corresponding retrieved display line 608. Thus, the other value 604 indicates that at least one display line 608 has been read from the video buffer 206.

[0054] Periodically, or as part of some operation, the frame compressor-decompressor controller 304 polls the synchronization register 602 as part of a first status communication 606-1. The frame compressor-decompressor controller 304 looks at the other value 604 to determine whether the corresponding display line 608 has been accessed and retrieved by the display controller 504. If so, the frame compressor-decompressor controller 304 knows that the memory location of the corresponding display line 608 can be overwritten with new data.

[0055] The value 604 or other values ​​604 may comprise, for example, one or more bits. The value 604 and the other values ​​604 may correspond to different memory locations or the same memory location of the synchronization register 602. Although described with reference to the video decoder 502 and the display controller 504, these principles apply generally to the core 110. By polling and updating the value 604 in the synchronization register 602, the routing of the reconstructed display frame 114D can be controlled using a memory mapped forwarding mechanism.

[0056] 7 illustrates an exemplary scheme 700 by a frame compressor-decompressor controller 304 for managing requests 116 made by multiple cores for decompressed frames. In this example, the video decoder 502 is integrated with the frame decompressor 108-2 and the frame compressor-decompressor controller 304. The frame compressor-decompressor controller 304 is represented by a request manager 402, a request queue 404, and a priority manager 406 (also shown in FIG. 4). As shown, the request queue 404 includes a number of requests 116. Each request 116 corresponds to a priority 706. Each request 116 also includes or corresponds to a requestor identification 702 and a frame identification 704.

[0057] Some requests for restored frames may be more important or time-related than other requests. For example, a request for real-time presentation of streaming video is likely to be more important to an end user than a request for a video transcoding operation that can be performed in the background at a best-effort speed. Thus, requests for the former video-related purpose may be prioritized over requests for the latter video-related purpose. To that end, a priority 706 may be associated with a request, a core, a frame, etc.

[0058] In an exemplary implementation, the requestor identification 702 identifies the core 110 making the request, such as the video decoder 502 or the display controller 504 (of FIGS. 5 and 6). The frame identification 704 identifies the reconstructed frame 114 that is being requested, such as by frame number, time index, or offset thereof. The requestor identification 702 may identify the source or (if a dedicated bus is provided for each core, for example) the core that provided the request 116. This can be accomplished with a particular voltage value on one or more bit lines based on the bus used, etc., by an assigned identification value that is transferred with the request 116. The frame identification 704 can be explicitly included in each request 116, can be implicitly included in the request 116 (e.g., if the next sequential frame is desired), can be included in a previous request as part of a range of frames, etc.

[0059] In operation, the request manager 402 receives requests 116 from the cores 110, such as via the frame decompressor controller services interface 408. The request manager 402 inserts the requests 116 into the request queue 404. In one approach, the request manager 402 operates the request queue 404 in a first-in, first-out (FIFO) manner. Alternatively, the request manager 402 can insert each request 116 into the request queue 404 based on a corresponding priority 706. The corresponding priority 706 of each individual request 116 can be included in or accompanied by the corresponding individual request 116. Alternatively, the priority 706 can be explicitly assigned collectively to a group of requests 116, or each core 110 can be associated with a particular priority 706 that is implicitly assigned to each request 116 issued by the associated core 110.

[0060] As shown, each request 116 is associated with a corresponding priority 706 and stored in a request queue 404 such that requests 116 having different priorities 706 are stored in the same request queue 404. Alternatively, the frame compressor-decompressor controller 304 may implement the illustrated request queue 404 using multiple request queues 404, with each request queue 404 associated with a corresponding priority 706. Thus, each request 116 is stored in a respective request queue 404 associated with the respective priority 706 corresponding to that request 116.

[0061] Regardless of the technique used to implement the illustrated request queue 404, the priority manager 406 orders the processing of the multiple requests 116 according to multiple priorities 706 corresponding to the multiple requests 116, respectively. Thus, the priority manager 406 extracts the next request 116 using a priority scheme based on the relative priorities between two or more requests 116. The priority manager 406 forwards the next request 116 to the frame decompressor 108-2. The frame decompressor 108-2 retrieves the compressed frame 112 corresponding to the requested decompressed frame 114. Continuing with the exemplary scenario of Figures 5 and 6, the frame decompressor 108-2 retrieves the compressed display frame 112D and performs a decompression operation to generate a decompressed display frame 114D. The frame compressor-decompressor controller interface 408 causes the frame decompressor 108-2 to route the decompressed display frame 114D to the video decoder 502 as part of the response 118.

[0062] FIG. 8 illustrates an exemplary scheme by the frame compressor-decompressor controller 304 to establish at least one time-sharing protocol 800 for sharing frame compression resources, such as the frame compressor 108-1 (e.g., of FIG. 3-1) or frame reconstruction resources, such as the frame decompressor 108-2 (e.g., of FIG. 3-2). A time-sharing protocol handler 412 of the frame compressor-decompressor controller 304 (not explicitly shown) can establish at least one exemplary time-sharing protocol 800-1, 800-2, or 800-3. Time progresses to the right, as indicated by arrow 812. Each exemplary time-sharing protocol 800 is described in terms of sharing frame reconstruction resources between two cores. However, the disclosed time-sharing protocols 800 are applicable to sharing frame reconstruction resources between more than two cores. Although FIG. 8 is described in terms of time-sharing frame reconstruction, the principles also apply to implementing time-sharing protocols for frame compression.

[0063] In the first time sharing protocol 800-1, time is divided into multiple time slots. These time slots have a fixed duration, and each respective time slot is allocated to a respective core. Time slot 802-1 corresponds to a first core, such as video decoder 502. Time slot 802-2 corresponds to a second core, such as display controller 504. In operation, the time sharing protocol handler 412 rotates access to the frame reconstruction resource between the two cores for a given length of two different time slots. No interrupts are enabled in the first time sharing protocol 800-1.

[0064] In contrast, an interrupt 806 is enabled for a second time-sharing protocol 800-2, where frame recovery resources are typically reserved for one of the two clients, the first core in this example. Thus, time slots 804-1 correspond to the first core and can extend indefinitely. These time slots 804-1 extend until the time-sharing protocol handler 412 receives an interrupt 806. The core 110 can signal the interrupt 806 via a message on the system bus, using a dedicated video bus, such as by driving an interrupt line high. In response to detecting the interrupt 806, the time-sharing protocol handler 412 initiates a context switch to transfer access to the frame recovery resources to a different core and initiate another time slot.

[0065] As shown in the second time sharing protocol 800-2, the second core issues an interrupt 806a. The time sharing protocol handler 412 detects the interrupt 806a and transfers access to the frame restorer 108-2 to the interrupting second core. Thus, the frame restorer 108-2 performs at least one restore operation to generate a restored frame 114 for the requesting second core during the time slot 804-2 following the interrupt 806a. The length of each time slot 804-2 can be fixed, dependent on how many requests are queued for the second core, dependent on the priority level of each queued request, etc. The second core can then issue another interrupt 806b after the first core is again granted access to the frame restore resources for another time slot 804-1. In an alternative approach, each time slot 804-2 allocated to the second core may last until another core issues an interrupt (not shown). Thus, in this alternative approach, a context switch may occur in response to an interrupt 806 and independent of the time that has elapsed.

[0066] For the third time sharing protocol 800-3, time is divided into multiple time slots. The third time sharing protocol 800-3 is a hybrid of the first and second time sharing protocols. Some time slots, such as time slot 808-1, have a fixed time during which they cannot be interrupted. Other time slots, such as time slot 810-1, have a variable time during which they can be interrupted by another core. As shown, a first core is allocated to time slots 808-1 and 810-1. An interrupt 806c from a second core arrives during time slot 808-1. However, the time sharing protocol handler 412 ignores (or queues) this interrupt 806c. Meanwhile, an interrupt 806d arriving during time slot 810-1 triggers a context switch. Due to the context switch, frame recovery resources are allocated to the second core during time slot 804-2 after the interrupt 806d. If interrupt 806c is queued (instead of being cleared) during time slot 808-1, then a context switch can be initiated at the end of time slot 808-1 or at the beginning of time slot 810-1 in response to interrupt 806c.

[0067] Having generally described an environment in which techniques and apparatus for a video frame codec architecture can be implemented, the discussion now turns to an exemplary method.

[0068] Exemplary Methods The following discussion describes methods for operating a video frame codec architecture. These methods can be implemented utilizing the previously described examples such as electronic device 102 and PCB 104 as shown in Figures 1 and 3, as well as electronic device 1000 depicted in Figure 10. Aspects of these methods are illustrated in Figure 9, which are shown as operations 902-912 performed by one or more components of at least one IC chip. The order in which these method operations are illustrated and / or described is not intended to be construed as limiting, and any number of the described method operations or combinations thereof can be combined in any order to implement a method or alternating method. Although the flow chart of Figure 9 is described in terms of operating frame restorer 108-2, the principles apply to operating frame compressor 108-1 as well.

[0069] 9 illustrates an example method 900 for operating a video frame codec architecture that may be implemented by circuitry that provides frame reconstruction services to multiple different cores. At 902, a first request for a first reconstructed frame is received from a first core. For example, the frame compressor-reconstructor controller 304 (e.g., its frame reconstructor controller 304-2) may receive a first request 116 for a first reconstructed frame 114 from a first core 110-1. The reception may occur via the system bus 218, the video bus 220, the synchronization registers 602, etc.

[0070] At 904, the first compressed frame is decompressed to generate a first reconstructed frame. For example, the frame decompressor 108-2 may decompress the first compressed frame 112 to generate the first reconstructed frame 114. Processing of the first request 116 may be performed according to an associated priority 706.

[0071] At 906, the first reconstructed frame is provided to the first core in response to the first request. For example, the frame compressor-decompressor controller 304 may provide the first reconstructed frame 114 to the first core 110-1 in response to the first request 116. The first reconstructed frame 114 may be forwarded from the frame decompressor 108-2 to the first core 110-1 using the routing mechanism 500.

[0072] At 908, a second request for the second decompressed frame is accepted from the second core. For example, the frame compressor-decompressor controller 304 may accept a second request 116 for the second decompressed frame 114 from the second core 110-2. The first core 110-1 may include a display controller 504, and the second core 110-2 may include a video decoder 502.

[0073] At 910, the second compressed frame is decompressed to generate a second decompressed frame. For example, the frame decompressor 108-2 may decompress the second compressed frame 112 to generate a second decompressed frame 114. The decompression may be performed using a lossless compression / decompression algorithm. At 912, the second decompressed frame is provided to a second core in response to the second request. For example, the frame compressor-decompressor controller 304 may provide the second decompressed frame 114 to the second core 110-2 in response to the second request 116. After obtaining the second decompressed frame 114, the second core 110-2 may process the frame according to the corresponding video-related functionality.

[0074] The above discussion describes the methodology for the video frame codec architecture. Aspects of the methods may be implemented, for example, in hardware (e.g., fixed logic circuitry), firmware, or some combination thereof. These techniques may be implemented using one or more of the entities or components shown in Figures 1-8 and 10 (electronic device 1000 is described in Figure 10 below), which may be further divided, combined, etc., and thus these figures illustrate some of the many possible systems or apparatuses in which the described techniques may be used. The entities and components in these figures generally represent firmware, hardware, IC chips, circuits, or combinations thereof.

[0075] Exemplary Electronic Devices Figure 10 illustrates various components of an exemplary electronic device 1000 capable of implementing a video frame codec architecture according to one or more implementations as described with reference to any of Figures 1-9 above. The electronic device 1000 may be implemented as any one or combination of fixed or mobile devices in the form of any of the following: consumer, computer, portable, user, server, communication, telephone, navigation, gaming, audio, camera, messaging, media playback, and / or other types of electronic devices 102, such as a smartphone depicted in Figure 1.

[0076] The electronic device 1000 includes a communications transceiver 1002 that enables wired and / or wireless communication of device data 1004, such as received data, transmitted data, or other information as described above. Exemplary communications transceivers 1002 include NFC transceivers, WPAN radios conforming to various IEEE 802.15 (Bluetooth®) standards, WLAN radios conforming to any of the various IEEE 802.11 (Wi-Fi®) standards, WWAN (3GPP compliant) radios for cellular telephone communications, Wireless Metropolitan Area Network (WMAN) radios conforming to various IEEE 802.16 (WiMAX®) standards, and wired local area network (LAN) Ethernet transceivers.

[0077] The electronic device 1000 may also include one or more data input ports 1006 through which any type of data, media content, and / or other inputs, such as user-selectable inputs, messages, applications, music, television content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source, may be received. The data input ports 1006 may include USB ports, coaxial cable ports, and other serial or parallel connectors (including internal connectors) for flash memory, DVDs, CDs, and the like. These data input ports 1006 may be used to couple the electronic device to components, peripherals, or accessories, such as a keyboard, microphone, or camera.

[0078] The electronic device 1000 of this example includes at least one processor 1008 (e.g., any one or more of an application processor, microprocessor, digital signal processor (DSP), controller, etc.), which may include a combination of a processor and memory system (e.g., implemented as part of an SoC) that processes (e.g., executes) computer-executable instructions stored on a computer-readable medium to control operation of the device. The processor 1008 may be implemented as an application processor, an embedded controller, a microcontroller, an SoC, etc. In general, a processor or processing system may be implemented at least partially in hardware. This may include components of integrated circuits or on-chip systems, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon and / or other hardware.

[0079] Alternatively or additionally, electronic device 1000 may be implemented using any one or combination of electronic circuitry, which may include software, hardware, firmware, or fixed logic circuitry implemented in association with processing and control circuitry generally designated 1010 (as electronic circuitry 1010). This electronic circuitry 1010 may implement executable or hardware-based modules (not shown), such as through logic circuitry and / or hardware (e.g., FPGAs).

[0080] Although not shown, electronic device 1000 may include a system bus, wiring, crossbar, or data transfer system that couples various components within the device. The system bus or wiring may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of a variety of bus architectures.

[0081] The electronic device 1000 also includes one or more memory devices 1012 that allow for data storage, examples of which include random access memory (RAM), non-volatile memory (e.g., read only memory (ROM), flash memory, EPROM, and EEPROM), and disk storage devices. The memory devices 1012 provide a data storage mechanism to store device data 1004, other types of code and / or data, as well as various device applications 1020 (software applications or programs). For example, an operating system 1014 may be maintained as software instructions in the memory device 1012 and executed by the processor 1008.

[0082] As shown, the electronic device 1000 also includes an audio and / or video processing system 1016 that processes audio data and / or passes audio and video data to an audio system 1018 and / or a display system 1022 (e.g., a video buffer or a smartphone or camera screen). The audio system 1018 and / or the display system 1022 may include any device that processes, displays, and / or otherwise renders audio, video, display, and / or image data. The display data and audio signals can be communicated to the audio and / or display components via other similar communication links, such as an RF (radio frequency) link, an S-video link, an HDMI (High-Definition Multimedia Interface), a composite video link, a component video link, a DVI (Digital Video Interface), an analog audio connection, or a media data port 1024. In some implementations, the audio system 1018 and / or the display system 1022 are external or separate components of the electronic device 1000. Alternatively, the display system 1022 can be an integrated component of the exemplary electronic device 1000, such as part of an integrated touch interface.

[0083] The electronic device 1000 of FIG. 10 is an example implementation of the electronic device 102 of FIG. 1. Thus, the processor 1008 is an example of an integrated circuit 106, a processor 202, or a SoC 302 (e.g., of FIGS. 1, 2, and 3, respectively). Thus, as shown in FIG. 10, the processor 1008 may include a frame compressor-decompressor 108, a number of cores 110-1 through 110-n, and a frame compressor-decompressor controller 304. In FIG. 10, the video buffer 206, the shared local cache 208, and the main memory 210 (e.g., also of FIG. 2, 3, 5, or 6, respectively) are depicted as forming at least a portion of a memory system, such as by being part of one or more memory devices 1012. As mentioned above, one or more of these memory components may be part of the same integrated circuit (e.g., a memory controller 1016, a memory controller 1018, a memory controller 1019, a memory controller 1020, a memory controller 1022, a memory controller 1024, a memory controller 1026, a memory controller 1028 ... The video buffer 206 and the shared local cache 208 may be part of the SoC 302 of FIG. 3 ), or may instead be located on separate dies, or even disposed on different printed circuit boards. In this manner, the principles of the video frame codec architecture as described herein may be implemented by or in conjunction with the electronic device 1000 of FIG.

[0084] Although implementations of the video frame codec architecture have been described in language specific to certain features and / or methods, the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations of the video frame codec architecture.

Claims

1. 1. An electronic device comprising: a frame restorer configured to restore a plurality of compressed frames individually at a frame level to generate a plurality of restored frames, each of the plurality of compressed frames being derived from a plurality of decoded frames; and a frame restorer controller coupled to the frame restorer and configured to arbitrate access to the frame restorer for a plurality of cores; a first core of the plurality of cores coupled to the frame restorer controller, the first core configured to obtain, via the frame restorer controller, one reconstructed frame of the plurality of reconstructed frames generated by the frame restorer at the frame level; and a second core of the plurality of cores coupled to the frame restorer controller, the second core configured to obtain, via the frame restorer controller, another reconstructed frame of the plurality of reconstructed frames generated by the frame restorer at the frame level; 11. An electronic device, wherein the first core and the second core are configured to perform video transcoding.

2. the first core is configured to decode a video stream to generate the plurality of decoded frames, at least one of the plurality of decoded frames being generated using the reconstructed frame; 2. The electronic device of claim 1, wherein the second core is configured to encode one or more decoded frames of the plurality of decoded frames using other reconstructed frames for transcoding the video.

3. the first core comprises a video decoder including a stream decoder configured to decode the video stream to generate the plurality of decoded frames; the frame restorer is logically integrated with the video decoder; The electronic device of claim 2 , wherein the second core comprises an encoder configured to encode the plurality of decoded frames to at least partially transcode the video.

4. The frame decompressor controller includes: a request queue configured to store a plurality of requests, each request corresponding to at least one reconstructed frame, the frame reconstructor controller further comprising: An electronic device according to any preceding claim, comprising a request manager configured to receive the requests from the cores and to insert the requests into the request queue.

5. The electronic device of claim 4 , wherein the frame decompressor controller comprises a priority manager configured to order processing of the plurality of requests according to a plurality of priorities corresponding to the plurality of requests.

6. 6. The electronic device of claim 1, wherein the frame restorer controller comprises a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame restorer among the multiple cores.

7. The time sharing protocol handler includes: establishing the time sharing protocol to include a number of time slots assigned to at least one of the plurality of cores; establishing the time-sharing protocol to permit receipt of an interrupt signal from a core of the plurality of cores; 7. The electronic device of claim 6, configured to at least one of: a.

8. a frame compressor configured to individually compress one or more uncompressed frames at the frame level to generate one or more compressed frames; 8. The electronic device of claim 1, further comprising a frame compressor controller coupled to the frame compressor and configured to arbitrate access to the frame compressor for the multiple cores.

9. The electronic device of claim 8 , wherein the frame compressor is configured to individually compress the decoded frames at the frame level to generate the compressed frames.

10. 10. The electronic device of claim 1, wherein the frame reconstructor is configured to implement a lossless frame-level compression algorithm to generate the plurality of reconstructed frames respectively from the plurality of compressed frames.

11. 1. A method for sharing frame decompression circuitry among multiple cores, comprising: compressing the first decoded frame at a frame level to generate a first compressed frame; compressing the second decoded frame at a frame level to generate a second compressed frame; accepting a first request for a first recovery frame from a first core; individually reconstructing the first compressed frames at the frame level to generate the first reconstructed frames; providing the first recovery frame to the first core in response to the first request; accepting a second request for a second reconstructed frame from a second core; individually reconstructing the second compressed frames at the frame level to generate the second reconstructed frames; providing the second recovery frame to the second core in response to the second request; performing video transcoding by the first core and the second core.

12. The above-mentioned actions are as follows: decoding, by the first core, a video stream to generate a plurality of decoded frames including the first decoded frame and the second decoded frame; encoding, by the second core, the plurality of decoded frames to generate another video stream; The method of claim 11 , comprising:

13. The decoding and generating includes: decoding, by the first core, the video stream to generate at least one decoded frame of the plurality of decoded frames using the first reconstructed frame; 13. The method of claim 12, wherein the encoding and generating comprises encoding, by the second core, one or more decoded frames of the plurality of decoded frames to generate another video stream using the second reconstructed frames.

14. accepting a third request for a third reconstruction frame from a third core; individually reconstructing the third compressed frame at the frame level to generate the third reconstructed frame; providing the third recovery frame to the third core in response to the third request; the first request indicates the first core; the second request indicates the second core; The method according to any one of claims 11 to 13, wherein the third request indicates the third core.

15. The method of any of claims 11 to 14, further comprising establishing a time-sharing protocol for access to the frame recovery circuitry by at least the first core and the second core.

16. receiving an interrupt from the second core during a time slot corresponding to the first core; 16. The method of claim 15, comprising: in response to receiving the interrupt, switching from performing a restore for the first core to performing a restore for the second core.

17. 17. The method of claim 11, further comprising providing the first recovery frame and providing the second recovery frame in an order that depends at least in part on a relative priority between the first request and the second request.

18. accepting a plurality of requests for a plurality of compressed frames from a plurality of cores; compressing a plurality of decompressed frames to generate the plurality of compressed frames, respectively; The method of any of claims 11 to 17, further comprising: providing the plurality of compressed frames to the plurality of cores in response to the plurality of requests.

19. 1. An electronic device comprising: a frame decompressor configured to decompress the plurality of compressed frames to generate a plurality of reconstructed frames; a frame restorer controller coupled to the frame restorer and configured to arbitrate access to the frame restorer for a plurality of cores, the frame restorer controller comprising a time-sharing protocol handler configured to establish a time-sharing protocol to share the frame restorer among the plurality of cores, the time-sharing protocol handler comprising: establishing the time sharing protocol to include a number of time slots assigned to at least one of the plurality of cores; establishing the time-sharing protocol to permit receipt of an interrupt signal from a core of the plurality of cores; wherein the interrupt signal indicates a request for at least one reconstruction frame; and a first core of the plurality of cores coupled to the frame restorer controller, the first core configured to obtain, via the frame restorer controller, a reconstructed frame of the plurality of reconstructed frames generated by the frame restorer; a second core of the plurality of cores coupled to the frame restorer controller, the second core configured to obtain, via the frame restorer controller, another reconstructed frame of the plurality of reconstructed frames generated by the frame restorer; 11. An electronic device, wherein the first core and the second core are configured to perform video transcoding.

20. the first core is configured to decode a video stream using the reconstructed frame to generate at least one decoded frame of a plurality of decoded frames; 20. The electronic device of claim 19, wherein the second core is configured to encode one or more decoded frames of the plurality of decoded frames using other reconstructed frames for transcoding the video.

Images