Intra-block copy decoding using dynamic remapping of on-chip memory

Abstract

A device includes a decoder configured to, during an intra-block copy (IBC) decoding process for at least a portion of a decoding unit of video data, identify a target virtual address for data access associated with a specific operation of the IBC decoding process. The target virtual address is generated based on an addressing scheme of a virtual memory used by the IBC decoding process. The decoder is configured to dynamically map the target virtual address to a specific memory address of a portion of on-chip memory. The on-chip memory is configured to store reconstructed blocks of the video data and has a second size that is smaller than a first size of the virtual memory. The decoder is further configured to access the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

Description

[0001] I. Cross-reference to Related Applications

[0002] This application claims the benefit of priority from commonly owned U.S. non-provisional patent application No. 17 / 655,393, filed on March 18, 2022, the contents of which are expressly incorporated herein by reference in their entirety.

[0003] II. Technical Field

[0004] This disclosure generally relates to decoding video data.

[0005] III. Related Fields

[0006] Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite wireless telephones (so-called "smartphones"), video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 (Part 10), Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), ITU-T H.266 / Versatile Video Coding (VVC), and extensions of such standards. By implementing such video coding techniques, such video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0007] Video coding techniques include spatial (intra-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) can be divided into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded slice of a picture are encoded using spatial prediction relative to reference samples in neighboring blocks in the same picture. Generally speaking, in intra block copy (IBC) coding mode, a video encoder may determine a prediction block for a current block, where the prediction block is in the same frame (picture) as the current block. The prediction block may be identified by a motion vector (also known as a block vector).

[0008] During decoding, previously decoded pixels identified by one or more motion vectors are used to generate predicted pixels for the current block. For example, a VVC IBC implementation specifies access to a relatively large number of previously decoded pixels (e.g., 256x128 pixels or 61 kilobytes (Kbytes)) decoded before the current decoding unit is decoded, so that they can be retrieved after reconstruction. This access requirement can be met by making the previously decoded pixel data locally available on-chip, such as by buffering the previously decoded pixel data in static random access memory (SRAM) or storing the previously decoded pixel data in off-chip dynamic random access memory (DRAM) using cached direct memory access (DMA). Such on-chip and off-chip solutions are expensive in terms of area, and an on-chip 61Kbyte SRAM implementation may be too expensive. However, off-chip storage using cached DMA also results in higher power consumption and increased access latency compared to on-chip storage, and thus reduces performance.

[0009] IV. Summary of the Invention

[0010] According to a particular implementation of the technology disclosed herein, a device includes a decoder configured to, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of video data, identify a target virtual address for a data access associated with a specific operation of the IBC decoding process. The target virtual address is generated based on an addressing scheme of a virtual memory used by the IBC decoding process. The decoder is configured to dynamically map the target virtual address to a specific memory address of a portion of on-chip memory. The on-chip memory is configured to store reconstructed blocks of the video data and has a second size that is smaller than a first size of the virtual memory. The decoder is further configured to access the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

[0011] According to a particular implementation of the technology disclosed herein, a method for decoding video data includes, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of the video data, identifying, at one or more processors, a target virtual address for a data access associated with a particular operation of the IBC decoding process, the target virtual address generated according to an addressing scheme of a virtual memory used by the IBC decoding process. The method includes dynamically mapping, at the one or more processors, the target virtual address to a particular memory address of a portion of on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory. The method also includes accessing the on-chip memory using the particular memory address to perform the particular operation of the IBC decoding process.

[0012] According to a specific implementation of the technology disclosed herein, a non-transitory computer-readable medium includes instructions that, when executed by one or more processors, cause the one or more processors to: identify a target virtual address for data access associated with a specific operation of an intra block copy (IBC) decoding process performed on at least a portion of a decoding unit of video data, the target virtual address being generated according to an addressing scheme of a virtual memory used by the IBC decoding process. When executed by the one or more processors, the instructions cause the one or more processors to dynamically map the target virtual address to a specific memory address of a portion of on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory. When executed by the one or more processors, the instructions further cause the one or more processors to access the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

[0013] Other implementations, advantages, and features of the present disclosure will become apparent upon review of the entire application, including the Brief Description of the Drawings, Detailed Description, and Claims.

[0014] V. Description of the Figures

[0015] Figure 1 is a block diagram illustrating an example of a specific implementation of a system operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0016] Figure 2 is an example of dynamically mapping virtual memory addresses to Figure 1 A diagram of a specific implementation of various portions of on-chip memory used in a system.

[0017] Figure 3 is an example of some examples of the present disclosure that can be used in Figure 1 A block diagram of an example of components of a decoder implemented in a system.

[0018] Figure 4 is an example of some examples of the present disclosure that can be used in Figure 1 A block diagram of another example of components of a decoder implemented in a system.

[0019] Figure 5 is a block diagram illustrating a specific implementation of an integrated circuit operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0020] Figure 6is a diagram of a specific implementation of a portable electronic device operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0021] Figure 7 is a diagram of a camera operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0022] Figure 8 is a diagram of a wearable electronic device operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0023] Figure 9 is a diagram of a voice-controlled speaker system operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0024] Figure 10 is a diagram of a head-mounted device (such as a virtual reality or augmented reality head-mounted device) that is operable to use dynamic mapping to access on-chip memory to perform IBC decoding according to some examples of the present disclosure.

[0025] Figure 11 is a diagram of a first example of a vehicle operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0026] Figure 12 is a diagram of a second example of a vehicle operable to access on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0027] Figure 13 is a diagram of a specific implementation of a method for accessing on-chip memory using dynamic mapping to perform IBC decoding according to some examples of the present disclosure.

[0028] Figure 14 is a block diagram of a specific illustrative example of a device operable to access on-chip memory using dynamic mapping to perform IBC decoding in accordance with some examples of the present disclosure.

[0029] VI. Specific Implementation Methods

[0030] Disclosed are systems and methods for accessing on-chip memory using dynamic mapping to perform IBC decoding. Conventional video decoding techniques using intra-block copy (IBC) decoding mode (such as VVC IBC implementations) specify buffering a relatively large number of previously decoded pixels (e.g., 256x128 pixels or 61 kilobytes (Kbytes)) that are decoded before the current decoding unit is decoded so that they are available for retrieval after reconstruction. While on-chip and off-chip solutions for this access requirement are expensive in terms of area, and an on-chip 61Kbyte SRAM implementation may be too expensive, off-chip storage using cached DMA also results in higher power consumption and increased access latency. Therefore, off-chip storage using cached DMA may also result in reduced performance compared to on-chip storage.

[0031] The disclosed systems and methods include techniques for dynamically remapping the virtual addresses of unused segments of an IBC decoded pixel buffer during decoding based on a window of reconstructed pixels eligible for retrieval as decoding progresses, enabling on-chip storage to be used to access reconstructed pixels. In a VVC IBC implementation, this remapping can reduce the buffer's memory requirements by 50% (e.g., reducing the required memory to 30.5Kbytes), enabling IBC buffering to be performed entirely on-chip. Performing IBC buffering entirely on-chip eliminates the need for one or more additional buffers associated with conventional approaches that do not use dynamic remapping to on-chip memory, such as a buffer (e.g., a 15Kbyte buffer) used to support round-trip DMA writes of reconstructed pixels in conventional approaches that use cached DMA with larger (e.g., 61Kbyte) memory requirements. Furthermore, according to some aspects, one or more additional buffers (e.g., a buffer for reconstructing loop filter data) can be merged into the disclosed on-chip buffer, further reducing the overall amount of memory used to implement video decoding.

[0032] By implementing dynamic mapping to reduce IBC buffer memory requirements, the IBC buffer can be implemented entirely on-chip, enabling 1-cycle latency access to buffered pixel data from SRAM and avoiding latency bottlenecks associated with off-chip accesses that would otherwise inhibit decoder performance. Furthermore, area reduction can be achieved by consolidating other data buffers into the on-chip buffer. Furthermore, using on-chip SRAM saves power compared to using cached DMA, which involves larger memory and additional I / O accesses.

[0033] The following describes specific aspects of the present disclosure with reference to the accompanying drawings. In this specification, shared features are designated by shared reference numerals. As used herein, various terms are used only for the purpose of describing a specific implementation and are not intended to limit the implementation. For example, the singular forms "one", "an" and "the" are intended to also include the plural forms, unless the context clearly indicates otherwise. It can be further understood that the term "including" can be used interchangeably with "comprising". In addition, it should be understood that the term "wherein" can be used interchangeably with "wherein". As used herein, "exemplary" can indicate examples, specific implementations and / or aspects, and should not be understood as restrictive or indicating preferences or preferred specific implementations. As used herein, ordinal terms (e.g., "first", "second", "third", etc.) used to modify elements (such as, structures, components, operations, etc.) do not themselves indicate any priority or order of the element relative to another element, but only distinguish the element from another element with the same name (but using ordinal terms). As used herein, the term "set" refers to one or more specific elements, while the term "plurality" refers to multiple (e.g., two or more) specific elements.

[0034] As used herein, "coupling" may include "communicatively coupled," "electrically coupled," or "physically coupled," and may also (or alternatively) include any combination thereof. Two devices (or components) may be coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) directly or indirectly via one or more other devices, components, wires, buses, networks (e.g., a wired network, a wireless network, or a combination thereof), etc. As illustrative, non-limiting examples, two electrically coupled devices (or components) may be included in the same device or in different devices, and may be connected via electronics, one or more connectors, or inductive coupling. In some implementations, two devices (or components) that are communicatively coupled (such as electrically connected) may transmit and receive signals (e.g., digital signals or analog signals) directly or indirectly via one or more wires, buses, networks, etc. As used herein, "direct coupling" may include two devices that are coupled (e.g., communicatively coupled, electrically coupled, or physically coupled) without intervening components.

[0035] In this disclosure, terms such as "determine," "calculate," "estimate," "shift," "adjust," etc. may be used to describe how to perform one or more operations. It should be noted that such terms should not be interpreted as limiting, and other techniques may be used to perform similar operations. In addition, as mentioned herein, "generate," "calculate," "estimate," "use," "select," "access," and "determine" may be used interchangeably. For example, "generating," "calculating," "estimating," or "determining" a parameter (or signal) may refer to actively generating, estimating, calculating, or determining the parameter (or signal), or may refer to using, selecting, or accessing a parameter (or signal) that has already been generated (e.g., by another component or device).

[0036] refer to Figure 1 , depicts certain illustrative aspects of a system 100 that includes a device 102 coupled to a display device 104 and configured to access on-chip memory using dynamic mapping to perform IBC decoding. The device 102 includes a memory 110 coupled to one or more processors 116 and configured to store instructions 112. The one or more processors 116 are configured to execute the instructions 112 to perform operations associated with decoding the encoded video data 122 at a decoder 124. In various implementations, some or all of the functionality associated with the decoder 124 is performed via execution of the instructions 112 by the processor 116, performed by processing circuitry of the processor 116 in a hardware implementation, or a combination thereof.

[0037] The one or more processors 116 include a decoder 124 coupled to an encoded data source 120. The decoder 124 is configured to obtain encoded video data 122 from the encoded data source 120. For example, the encoded data source 120 may correspond to a portion of one or more of a media file (e.g., a media file including the encoded video data 122 retrieved from the memory 110), a game engine, one or more other sources of video information (such as a remote media server), or a combination thereof. In a particular implementation, the encoded video data 122 corresponds to VVC IBC encoded video data.

[0038] Device 102 also includes a modem 118 coupled to decoder 124 and configured to communicate with one or more other devices, such as via one or more wireless networks. According to some aspects, modem 118 is configured to receive encoded video data 122 from a second device, such as video data streamed from a remote server for playback at device 102.

[0039] The decoder 124 is configured to access the on-chip memory 150 via one or more operations 170, such as a write operation 172 or a read operation 174. In certain implementations, the on-chip memory 150 and the decoder 124 are integrated into a single substrate 190 (e.g., a single chip). Although the on-chip memory 150 is illustrated as distinct from and coupled to the decoder 124, in other implementations, the on-chip memory 150 is integrated into the decoder 124 as a local buffer. According to one aspect, the on-chip memory 150 includes a static random access memory (SRAM) IBC ring buffer.

[0040] According to one aspect, the decoder 124 is configured to perform an IBC decoding process 132 for each IBC-encoded decoding unit of the encoded video data 122 to generate a reconstructed block of the video data. The IBC decoding process 132 may be performed in conjunction with the IBC prediction engine 130 of the decoder 124. For illustration, the IBC prediction engine 130 may be configured to receive data extracted from the encoded video data 122 indicating one or more motion vectors associated with a current decoding unit 134. The IBC prediction engine 130 may initiate the IBC decoding process 132 to retrieve pixel data for one or more currently reconstructed blocks indicated by the one or more motion vectors, merge the reconstructed pixels to generate predicted pixels for the current decoding unit, and buffer the predicted pixels for further processing. The predicted pixels may be stored (e.g., buffered) in the on-chip memory 150 and may be accessed by the decoder 124 for further processing, such as to generate reconstructed pixels, to perform loop filtering, etc., as further referenced. Figure 3 described.

[0041] The decoder 124 is configured to identify a target virtual address 138 for data access associated with a particular operation 170 of the IBC decoding process 132 (such as a read operation 174 for retrieving currently reconstructed pixel data or a write operation 172 for storing predicted pixel data) during an IBC decoding process 132 of at least a portion of a decoding unit 134 of the encoded video data 122. The target virtual address 138 is generated according to an addressing scheme 136 of a virtual memory 140 used by the IBC decoding process 132 (such as a VVC addressing scheme for IBC decoding).

[0042] In a particular implementation, the virtual memory 140 corresponds to a virtual buffer having a first size 142 according to the VVC specification that is sufficient to store eight 64x64 blocks of pixel data in a rectangular layout having a row width of 256 pixels and a column height of 128 pixels, arranged as two block rows and four block columns (also referred to as "sub-rows" and "sub-columns," respectively). In an example where 15 bits of combined luma and chroma data are associated with each pixel, storing the pixel data associated with 256x128 pixels requires a first size 142 of at least 61,440 bytes, which may be prohibitively expensive in terms of on-chip area. However, reading and writing pixel data at off-chip memory (e.g., via DMA to DRAM) introduces significantly greater access latency than using an on-chip buffer, resulting in reduced decoder performance.

[0043] The decoder 124 includes a dynamic mapper 160 that is configured to dynamically map the target virtual address 138 to a particular memory address 168 of a portion of the on-chip memory 150. The on-chip memory 150 is configured to store reconstructed blocks 154 of the video data 122 and is operable as an SRAM IBC ring buffer having a second size 152 that is smaller than the first size 142 of the virtual memory 140. In a particular implementation, the first size 142 of the virtual memory 140 is at least twice the second size 152 of the on-chip memory 150.

[0044] For illustration, although the VVC specification may specify access to eight blocks of virtual memory 140 via the virtual memory addressing scheme 136, one or more other aspects of the VVC specification may limit the motion vector size for intra block prediction such that a maximum of four blocks of pixel data (e.g., the current block and the three most recently reconstructed blocks) are accessed during the IBC decoding process 132. Accordingly, the dynamic mapper 160 may map portions of the virtual memory 140 corresponding to the four accessible blocks of pixel data to four portions of the on-chip memory 150. As illustrated, the dynamic mapper 160 is configured to perform a dynamic mapping 144 that maps a target virtual address 138 associated with a particular block (e.g., "block 6") of the eight blocks of the virtual memory 140 to a particular memory address 168 associated with another particular block (e.g., "block 2") of the four illustrated blocks in the on-chip memory 150. Reference Figure 2 An example of the operation of the dynamic mapper 160 is further described.

[0045] Dynamic mapper 160 is configured to populate and update a data structure 162 that associates one or more virtual memory start addresses 164 with one or more portions 166 of on-chip memory 150. For example, as an illustrative, non-limiting example, data structure 162 may include one or more arrays, lists, or tables that associate the virtual memory start address of “block 6” of virtual memory 140 with the “block 2” portion of on-chip memory 150. Figure 2 Examples of populating and updating data structure 162 in conjunction with writing data to on-chip memory 150 are further described in .

[0046] The decoder 124 is configured to access the on-chip memory 150 using a specific memory address 168 to perform a specific operation 170 of the IBC decoding process 132. For example, in response to the specific operation 170 corresponding to a write operation 172 for writing a block of pixel data to an unused portion of the virtual memory 140, the decoder 124 (e.g., the dynamic mapper 160) is configured to store the virtual memory start address of the block of pixel data in the data structure 162. As another example, in response to the specific operation 170 corresponding to a read operation 174 for reading a block of pixel data from the virtual memory 140, the decoder 124 is configured to locate the virtual memory start address of the block of pixel data in the data structure 162 to identify the corresponding portion of the on-chip memory 150 from which the pixel data can be retrieved.

[0047] The decoder 124 may be configured to perform additional processing on the reconstructed blocks of the encoded video data 122 to generate the video data 126, such as further described in Figure 3 According to one aspect, video data 126 can be provided to display device 104 for viewing by a user of device 102 .

[0048] During operation, the encoded video data 122 is processed by the decoder 124 to generate video data 126. The intra-block coded portion of the encoded video data 122 is processed by the IBC prediction engine 130 to generate predicted pixels for the current block based on pixels of a previously reconstructed block in the same frame as the current block. Each target virtual address 138 for the virtual memory 140 generated by the virtual memory addressing scheme 136 (such as for retrieving a reconstructed pixel or for storing a predicted pixel) is mapped by the dynamic mapper 160 to a corresponding specific memory address 168 in the on-chip memory 150. The dynamic mapper 160 can update the data structure 162 as each block is processed so that the on-chip memory 150 operates as a ring buffer that maintains pixel data corresponding to the current block and blocks within the permitted motion vector range for IBC decoding, and overwrites pixel data for reconstructed blocks that are no longer within the permitted motion vector range for the current block.

[0049] Sizing on-chip memory 150 to include reconstructed blocks accessible by the motion vector of the current block, while ignoring reconstructed blocks no longer accessible by the motion vector, allows on-chip memory 150 to be significantly smaller than virtual memory 140 (e.g., half the size of the virtual memory). Furthermore, using dynamic mapper 160 to map virtual memory start address 164 provided by virtual memory addressing scheme 136 to portion 166 of on-chip memory 150 enables read and write access for IBC operations without modifying virtual memory addressing scheme 136 (as modifying virtual memory addressing scheme 136 could cause decoder 124 to no longer comply with the video coding standard). Thus, on-chip memory 150 can provide on-chip storage for accessible reconstructed blocks based on virtual memory addressing scheme 136 without consuming excessive chip area and with reduced latency and power consumption compared to off-chip storage.

[0050] According to some aspects, decoder 124 and on-chip memory 150 are integrated into at least one of a mobile phone or a tablet computer device, such as Figure 6 According to some aspects, the decoder 124 and the on-chip memory 150 are integrated into Figure 8 The wearable electronic device shown in the example Figure 9 The illustrated voice controlled speaker system, or Figure 7 In the illustrated camera device. According to some aspects, one or more processors 116 are integrated into Figure 10 The virtual reality head mounted device, mixed reality head mounted device, or augmented reality head mounted device, or Figure 11 and Figure 12 in the illustrated vehicle.

[0051] Although display device 104 is illustrated as included in (e.g., integrated with) device 102, in other implementations, display device 104 is coupled to device 102 but not included therein. While specific examples of the sizes and layouts of virtual memory 140 and on-chip memory 150 are described for illustrative purposes, it should be understood that the techniques described herein can be used with virtual memory 140 of other sizes, on-chip memory 150 of other sizes, or both. For example, while a 50% reduction in memory size is described using dynamic mapping 114 in conjunction with certain aspects of a VVC IBC implementation, other amounts of memory size reduction can be achieved in implementations using other IBC coding standards (e.g., which may have different ranges of prediction blocks accessible via the motion vector of the current block). Therefore, the examples presented herein should be considered illustrative rather than limiting. Furthermore, while a graphical representation of virtual memory 140 is depicted, it should be understood that virtual memory 140 represents a logical model of memory used by virtual memory addressing scheme 136 and, in accordance with the present disclosure, does not represent physical memory or any other physical device for data storage.

[0052] Figure 2 An example 200 is illustrated depicting the states of virtual memory 140, on-chip memory 150, and data structure 162 after performing each write operation in a sequence of IBC write operations (e.g., when decoding and reconstructing a sequence of Quad64x64 blocks to obtain a CTB size of 128x128 pixels). The state of virtual memory 140 illustrates the location of stored data in virtual memory as specified by virtual memory addressing scheme 136, and the state of on-chip memory 150 illustrates the location of stored data in physical memory using mapping performed by dynamic mapper 160. In example 200, each block in virtual memory 140 and on-chip memory 150 represents a quad 64x64 of a 128x128 sized CTB. However, in other implementations, the illustrated state sequence also represents CTB sizes of 64x64 and 32x32, but at the CTB level rather than the Quad 64x64 level.

[0053] Blocks in virtual memory 140 and on-chip memory 150 are illustrated as having one of three possible states: a "-1" state, indicating that the block cannot be accessed for retrieval (e.g., the block is outside the motion vector range); a shaded (diagonal) state, indicating that the current Quad 64x64 is valid (e.g., prediction and reconstruction are in progress); and a dark state, indicating that the Quad 64x64 for which reconstruction is complete and can be accessed for retrieval.

[0054] The data structure 162 includes two current buffer indexes 202: CurrTop, which indicates which subcolumn of the top subrow of the on-chip memory 150 is current (or most recently used), where "0" indicates the left subcolumn and "1" indicates the right subcolumn; and CurrBot, which indicates which subcolumn of the bottom subrow of the on-chip memory 150 is current (or most recently used), where "0" indicates the left subcolumn and "1" indicates the right subcolumn. The data structure 162 also includes two start address arrays 204: TopStartAddr and BotStartAddr. TopStartAddr is a two-element array in which the left array element stores the starting address of the Quad64x64 stored in the left subcolumn of the top subrow of the on-chip memory 150 (as indicated in the addressing scheme of the virtual memory 140), and the right array element stores the starting address of the Quad64x64 stored in the right subcolumn of the top subrow of the on-chip memory 150. Similarly, BotStartAddr is a two-element array in which the left array element stores the starting address of the Quad 64x64 stored in the left subcolumn of the bottom subrow of on-chip memory 150 (as indicated in the addressing scheme of virtual memory 140), and the right array element stores the starting address of the Quad 64x64 stored in the right subcolumn of the bottom subrow of on-chip memory 150.

[0055] At slice initialization, all blocks of virtual memory 140 and on-chip memory 150 are set to -1, current buffer index 202 is set to 0, and elements of start address array 204 are set to 0. During processing of Quad #0 (e.g., the first Quad 64x64, also denoted as Q0), the top left block at row 0, column 0 of virtual memory 140 is indicated as valid, and the left block of the top sub-row of on-chip memory 150 is indicated as valid. Current buffer index 202 and start address array 204 remain unchanged because the left sub-column of the top sub-row of on-chip memory 150 is valid (e.g., CurrTop=0), and the start address of Quad #0 in the top sub-row of virtual memory 140 is 0 (e.g., TopStartAddr(0)=0).

[0056] During processing of Quad #1 (e.g., the second Quad 64x64, also denoted as Q1), the block in the top subrow, second subcolumn (row 0, column 64) of virtual memory 140 is indicated as valid, and the right block in the top subrow of on-chip memory 150 is indicated as valid. CurrTop is set to 1, and the starting address of Quad #1 in the top subrow of virtual memory 140 (e.g., column 64) is stored in TopStartAddr(1) (e.g., TopStartAddr(1)=64). Storing the block of Quad #0 is indicated as complete in virtual memory 140 and on-chip memory 150.

[0057] During processing of Quad #2, the block in the bottom subrow, first subcolumn (row 64, column 0) of virtual memory 140 is indicated as valid, and the left block in the bottom subrow of on-chip memory 150 is indicated as valid. CurrBot remains 0, and the starting address (e.g., column 0) of Quad #2 in the bottom subrow of virtual memory 140 is stored in BotStartAddr(0) (e.g., BotStartAddr(0)=0). Storing the block of Quad #1 is indicated as complete in virtual memory 140 and on-chip memory 150.

[0058] During processing of Quad #3, the block in the bottom subrow, second subcolumn (row 64, column 64) of virtual memory 140 is indicated as valid, and the right block in the bottom subrow of on-chip memory 150 is indicated as valid. CurrBot is set to 1, and the starting address of Quad #3 in the bottom subrow of virtual memory 140 (e.g., column 64) is stored in BotStartAddr(1) (e.g., BotStartAddr(1)=64). Storing the block of Quad #2 is indicated as complete in virtual memory 140 and on-chip memory 150.

[0059] During processing of Quad #4, the block in the top sub-row, third sub-column (row 0, column 128) of virtual memory 140 is indicated as valid, and the left block in the top sub-row of on-chip memory 150 is indicated as valid, where the reconstructed data of Quad #0 is replaced by Quad #4 in on-chip memory 150. CurrTop is set to 0, and the starting address of Quad #4 in the top sub-row of virtual memory 140 (e.g., column 128) is stored in TopStartAddr(0) (e.g., TopStartAddr(0)=128). The block storing Quad #3 is indicated as complete in virtual memory 140 and on-chip memory 150. Because Quad #0 is outside the motion vector range of Quad #4, the block associated with Quad #0 is set to -1 in virtual memory 140 (e.g., outside the acquisition range and therefore no longer used).

[0060] The states of virtual memory 140, on-chip memory 150, and data structure 162 are illustrated for continuing the pattern described above to process Quad #5 through Quad #11.

[0061] After processing Quad #7 associated with the lower right block of virtual memory 140 (e.g., row 64, column 192), the virtual memory addressing scheme 136 wraps around so that processing of Quad #8 is performed in the block in the top sub-row, first sub-column of virtual memory 140. The block in the top sub-row, first sub-column (row 0, column 0) of virtual memory 140 is indicated as valid, and the left block in the top sub-row of on-chip memory 150 is indicated as valid, where the reconstructed data of Quad #4 is replaced by Quad #8 in on-chip memory 150. CurrTop is set to 0, and the starting address of Quad #8 in the top sub-row of virtual memory 140 (e.g., column 0) is stored as TopStartAddr(0)=0. Because Quad #4 is outside the motion vector range of Quad #8, a portion 224 of virtual memory 140 associated with Quad #4 is set to -1 (e.g., outside the acquisition range and therefore no longer used).

[0062] Thus, in response to performing a write operation to write a pixel data block 210 (e.g., Quad #8) to an unused portion 212 of the virtual memory 140, the decoder 124 is configured to store a virtual memory start address 208 of the pixel data block 210 in the data structure 162 (e.g., in the TopStartAddr array). In a particular implementation, the virtual memory start address 208 of the pixel data block 210 is based on another stored virtual memory start address 218 associated with another portion 222 of the virtual memory 140 that has an increased block size (e.g., a block width of 64 pixels) (e.g., the start address (192) of the pixel data block 220 of Quad #5 in the top right block of the virtual memory 140 and the on-chip memory 150) and a modulus associated with the row size (e.g., 256) of the virtual memory 140. For illustration, the virtual memory starting address 208 of Quad #8 may be calculated as: virtual memory starting address 208 = ((virtual memory starting address 218) + 64) % 256, where "%" indicates a modulo operator.

[0063] Figure 3 An illustrative example 300 of components that may be implemented in the decoder 124 is depicted. In the particular example 300, the decoder 124 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction engine 310, a loop filter 312, on-chip memory 150, and a decoded picture buffer (DPB) 314. In particular implementations, the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction engine 310, the loop filter 312, the DPB 314, or any combination thereof may be implemented in one or more processors or in processing circuitry.

[0064] The prediction processing unit 304 includes the IBC prediction engine 130 and the dynamic mapper 160. The prediction processing unit 304 may include additional units that perform prediction according to other prediction modes. As an example, the prediction processing unit 304 may include a motion compensation unit, an intra prediction unit, a palette unit, an affine unit, a linear model (LM) unit, one or more other units configured to perform prediction, or a combination thereof.

[0065] CPB memory 320 may store video data to be decoded, such as encoded video data 122 (e.g., an encoded video bitstream). CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) and may also store video data other than syntax elements for coded pictures, such as temporary data representing outputs from various units of decoder 124. DPB 314 typically stores decoded pictures, which decoder 124 may output as video data 126, use as reference video data when encoding subsequent data or pictures in the encoded video bitstream, or both. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. CPB memory 320 and DPB 314 may include the same memory device or separate memory devices.

[0066] Illustrated Figure 3 The various units shown are provided to aid in understanding the operations performed by the decoder 124 according to some specific implementations. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functionality and are preset on executable operations. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the executable operations. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function circuit blocks or programmable circuit blocks), and in some examples, the one or more units may be integrated circuits.

[0067] Entropy decoding unit 302 receives the encoded video data from CPB memory 320 and entropy decodes the encoded video data 122 to reproduce syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction engine 310, and loop filter 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0068] Generally speaking, the decoder 124 reconstructs a picture block by block. The decoder 124 may perform a reconstruction operation on each block individually (wherein the block currently being reconstructed (ie, decoded) may be referred to as the "current block").

[0069] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block and transform information, such as a quantization parameter (QP) and / or a transform mode indication. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization and determine the degree of inverse quantization to be applied by the inverse quantization unit 306. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thereby form a transform coefficient block comprising the transform coefficients.

[0070] After inverse quantization unit 306 forms a transform coefficient block, inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse discrete cosine transform (DCT), an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the coefficient block.

[0071] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, a motion compensation unit (not shown) may generate a prediction block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which to retrieve the reference block, and a motion vector that identifies the position of the reference block in the reference picture relative to the position of the current block in the current picture.

[0072] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit (not shown) may generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. The intra-prediction unit 318 may retrieve data of neighboring samples of the current block from the DPB 314.

[0073] The prediction processing unit 304 may also determine to use an intra block copy (IBC) mode to decode the video data block, which uses the IBC prediction engine 130. Generally speaking, in IBC mode, the decoder 124 may determine a prediction block for a current block, where the prediction block is located in the same frame as the current block. The prediction block may be identified by a block vector (e.g., a motion vector) and is limited to the location of the decoded block. As previously described, the IBC prediction engine 130 may use the virtual memory addressing scheme 136 to generate a target virtual address 138 for the prediction block, and the dynamic mapper 160 may map the target virtual address 138 to a specific memory address 168 corresponding to a portion of the on-chip memory 150.

[0074] The reconstruction engine 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction engine 310 may add samples of the residual block to corresponding samples of the prediction block in the on-chip memory 150 to reconstruct the current block in the on-chip memory 150.

[0075] The loop filter 312 may perform one or more filtering operations on the reconstructed block. For example, the loop filter 312 may access the reconstructed block in the on-chip memory 150 and perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. Therefore, the decoder 124 is configured to access the on-chip memory 150 during the execution of the loop filtering operations performed by the loop filter 312 and associated with the IBC decoding process. The operations of the loop filter 312 are not necessarily performed in all examples.

[0076] The decoder 124 may store the reconstructed blocks in the DPB 314, which may be implemented as an off-chip memory to have a larger storage capacity than the on-chip memory 150. As discussed above, the DPB 314 may provide reference information (such as samples of the current picture for intra prediction and previously decoded pictures for subsequent motion compensation) to the prediction processing unit 304. In addition, the decoder 124 may output decoded pictures from the DPB 314 for subsequent presentation on a display device, such as to a video display. Figure 1 The video data 126 is displayed to the display device 104.

[0077] Figure 4 is a block diagram illustrating an example 300 of components that may be implemented in decoder 124, depending on a particular implementation. IBC prediction engine 130 is illustrated as including a fetch command generator 402 and an IBC data fetcher 404. IBC prediction engine 130 is coupled to a reconstruction engine 410 that includes on-chip memory 150.

[0078] The fetch command generator 402 is configured to receive a prediction command 408 (IBC_Pred) and generate one or more fetch commands 412 for the IBC data fetcher 404. The IBC data fetcher 404 includes a dynamic mapper 160 for mapping a virtual address received via the fetch command 412 to a memory address for reading reconstructed pixel data from the on-chip memory 150. The reconstructed pixel data is retrieved from the on-chip memory 150 via one or more IBC reconstruction fetch operations 420. The IBC data fetcher 404 is further configured to generate predicted pixels based on the reconstructed pixel data and write the predicted pixels to the on-chip memory 150 as IBC predicted pixels 422.

[0079] Reconstruction engine 410 includes on-chip memory 150 and is configured to generate reconstructed pixels for a particular block based on its IBC predicted pixels 422 and the residual of the particular block. In certain implementations, reconstruction engine 410 operates in a manner substantially similar to that described for reconstruction engine 310.

[0080] Although the dynamic mapper 160 is illustrated in the IBC data fetcher 404 , in other implementations, the dynamic mapper 160 may be implemented in the fetch command generator 402 to perform mapping of virtual memory addresses to memory addresses for accessing the on-chip memory 150 , which may be included in the fetch command 412 to the IBC data fetcher 404 .

[0081] Figure 5 1 is a block diagram illustrating an implementation 500 of a device 102 as an integrated circuit 502 for accessing on-chip memory using dynamic mapping to perform IBC decoding. The integrated circuit 502 includes one or more processors 116, which include a decoder 124. The decoder 124 includes an on-chip memory 150 and a dynamic mapper 160. The integrated circuit 502 also includes a signal input 504 (such as a bus interface) to enable reception of encoded video data 122. The integrated circuit 502 includes a signal output 506 (such as a bus interface) to enable output of decoded video data 126. The integrated circuit 502 enables accessing on-chip memory using dynamic mapping to perform IBC decoding as a component in a system that performs video decoding and playback, such as a video playback system. Figure 1 Described in.

[0082] As an illustrative, non-limiting example, Figure 6 6. An embodiment 600 is depicted in which the device 102 includes a mobile device 602, such as a phone or tablet. The mobile device 602 includes a display 604. Components of the one or more processors 116, including the decoder 124 and the on-chip memory 150, are integrated into the mobile device 602 and are illustrated using dashed lines to indicate internal components that are generally not visible to a user of the mobile device 602. In a specific example, the decoder 124 operates to access the on-chip memory 150 using dynamic mapping to perform IBC decoding. For example, the mobile device 602 may receive encoded video data from a remote device (e.g., a phone or computer device of another participant in a video conference), decode the encoded video data using the decoder 124 (including or coupled to the on-chip memory 150), and display the resulting decoded video at the display 604.

[0083] Figure 7An implementation 700 is depicted in which the device 102 includes a portable electronic device corresponding to a camera device 702. The decoder 124, the on-chip memory 150, a display device (e.g., not shown), or a combination thereof are included in the camera device 702. During operation, the decoder 124 uses dynamic mapping to access the on-chip memory 150 to perform IBC decoding during playback of video data (such as video data captured by the camera device 702 and stored as encoded video data at the memory of the camera device 702) via the display device.

[0084] Figure 8 A specific implementation 800 of a wearable electronic device 802 (illustrated as a "smart watch") is depicted. In a particular aspect, the wearable electronic device 802 includes the device 102. The decoder 124 is integrated into the wearable electronic device 802 and is coupled to or includes the on-chip memory 150. In a particular aspect, the wearable electronic device 802 is coupled to or includes a display screen 804 to display video data decoded by the decoder 124. In a particular example, the decoder 124 operates to access the on-chip memory 150 using dynamic mapping to perform IBC decoding. In a particular example, the wearable electronic device 802 includes a haptic device that provides a tactile notification (e.g., a vibration) associated with playback of the decoded video data via the display screen 804. For example, the tactile notification can cause a user to view the wearable electronic device 802 to view video playback, such as a video announcement of an incoming video phone call or a video message received at the wearable electronic device 802.

[0085] Figure 9 is a specific implementation 900 of a wireless speaker and voice activated device 902. In certain aspects, the wireless speaker and voice activated device 902 includes Figure 1The wireless speaker and voice-activated device 902 may have wireless network connectivity and be configured to perform auxiliary operations. One or more processors 116 are included in the wireless speaker and voice-activated device 902 and include a decoder 124, which includes (or is coupled to) on-chip memory 150. In certain aspects, the wireless speaker and voice-activated device 902 includes one or more microphones 938 and one or more speakers 936, and also includes or is coupled to a display device 904 for playback of video output by the decoder 124. During operation, the decoder 124 uses dynamic mapping to access the on-chip memory to perform IBC decoding during playback of video data via the display device 904. In response to receiving verbal commands via the one or more microphones 938, the wireless speaker and voice-activated device 902 may perform auxiliary operations, such as by executing a voice activation system (e.g., an integrated auxiliary application). The auxiliary operations may include adjusting the temperature, playing media content (such as stored or streamed audio and video content), turning on lights, and the like. For example, an auxiliary operation is performed in response to receiving a command following a keyword or key phrase (eg, "Hello, Assistant").

[0086] Figure 10 An implementation 1000 of a portable electronic device corresponding to a virtual reality, augmented reality, or mixed reality head mounted device 1002 is depicted. In certain aspects, the head mounted device 1002 includes Figure 1 The decoder 124 is integrated into the head-mounted device 1002. In certain aspects, the decoder 124 includes or is coupled to an on-chip memory 150 and is configured to access the on-chip memory 150 using dynamic mapping to perform IBC decoding during playback of video data via the visual interface device 1004 (e.g., the display device 104). The visual interface device 1004 is positioned in front of the user's eyes to enable display of video associated with an augmented reality, mixed reality, or virtual reality scene to the user while wearing the head-mounted device 1002.

[0087] Figure 11 An implementation 1100 is depicted in which the device 102 corresponds to or is integrated within a vehicle 1102, illustrated as a manned or unmanned aerial vehicle (e.g., a package delivery drone). In some implementations, one or more processors 116 (e.g., including the decoder 124) are integrated into the vehicle 1102.

[0088] During playback of video data decoded by the decoder 124 and played back via the display device 1104, the decoder 124 uses dynamic mapping to access on-chip memory to perform IBC decoding. In some implementations, the vehicle 1102 is manned (e.g., carrying a pilot, one or more passengers, or both), the display device 1104 is located inside the cabin of the vehicle 1102, and IBC decoding is used during playback to the pilot or passengers of the vehicle 1102. In another implementation, the vehicle 1102 is unmanned, the display device 1104 is mounted to an exterior surface of the vehicle 1102, and IBC decoding is used during video playback to one or more viewers outside the vehicle 1102. For example, the vehicle 1102 may be moving (e.g., around an outdoor audience during a concert) while broadcasting a video (such as a commercial or streaming video of a concert stage), and the one or more processors 116 (e.g., including the decoder 124) may use dynamic mapping to access on-chip memory to perform IBC decoding to generate video from the encoded video stream.

[0089] Figure 12 An implementation 1200 is depicted in which the device 102 corresponds to or is integrated within a vehicle 1202 (illustrated as an automobile that also includes a decoder 124, a display device 1220, and speakers 1236). In some implementations, the decoder 124 and the on-chip memory 150 are integrated into the vehicle 1202, and the display device 1220 is configured to display video data output by the decoder 124. For example, the video data may correspond to streamed video data from a remote source (e.g., a remote media server), video stored at the vehicle 1202 (such as entertainment content or instructional videos regarding the operation of the vehicle 1202), or video captured via one or more camera sensors of the vehicle 1202 (such as a backup camera).

[0090] Figure 13 An example of a method 1300 for decoding video data is illustrated. As an illustrative, non-limiting example, one or more operations of the method 1300 may be performed by Figure 1 The system 100 (eg, the device 102, one or more processors 116, or the decoder 124) executes.

[0091] Method 1300 includes, at 1302, identifying, at one or more processors, a target virtual address for a data access associated with a particular operation of an intra block copy (IBC) decoding process during an intra block copy (IBC) decoding process on at least a portion of a decoding unit of video data. The target virtual address is generated based on an addressing scheme for a virtual memory used by the IBC decoding process. For example, a decoder 124 (e.g., an IBC prediction engine 130 executing an IBC decoding process 132) identifies a target virtual address 138 via a virtual memory addressing scheme 136 for performing one or more operations 170, such as a write operation 172 (e.g., storing a predicted pixel) or a read operation 174 (e.g., retrieving a reconstructed pixel for prediction). According to one aspect, the addressing scheme corresponds to a versatile video coding (VVC) addressing scheme for IBC decoding.

[0092] The method 1300 includes dynamically mapping, at one or more processors, a target virtual address to a specific memory address of a portion of on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory, at 1304. For example, the dynamic mapper 160 performs the dynamic mapping 144 to map the target virtual address 138 to a specific memory address 168 of the portion of on-chip memory 150. According to one aspect, the on-chip memory includes a static random access memory (SRAM) IBC ring buffer.

[0093] The method 1300 includes accessing the on-chip memory using the specific memory address to perform specific operations of the IBC decoding process at 1306. For example, the decoder 124 accesses the on-chip memory 150 to perform one or more operations 170 associated with the IBC decoding process 132.

[0094] In some implementations, before dynamically mapping the target virtual address to the specific memory address, a previously used virtual address in the virtual memory is mapped to the specific memory address. Figure 2 , before portion 212 of virtual memory 140 (e.g., row 0, column 0) is dynamically mapped to the upper left portion of on-chip memory 150 for storing Quad #8, the address of portion 224 of virtual memory 140 associated with Quad #4 (e.g., row 0, column 128) is previously mapped to the upper left portion of on-chip memory 150.

[0095] In a particular implementation, the particular operation corresponds to a write operation for writing a block of pixel data to an unused portion of virtual memory, and dynamically mapping the target virtual address to the particular memory address includes storing a virtual memory start address of the block of pixel data in a data structure that associates the virtual memory start address with a portion of on-chip memory. The virtual memory start address of the block of pixel data is based on another stored virtual memory start address associated with another portion of the on-chip memory that is increased in block size and a modulus associated with a row size of the virtual memory. For example, in writing Figure 2 During the reading of pixel data 210 for Quad #8 in the on-chip memory 150, a virtual memory start address 208 (e.g., column 0 of the top row) of a portion 212 of the virtual memory 140 is written to a start address (e.g., TopStartAddr(0)) in the start address array 204. The virtual memory start address 208 is associated with the virtual memory start address 218 of the pixel data block 220 for Quad #5 stored in the top right block of the on-chip memory 150 via the following formula: Virtual memory start address 208 = ((virtual memory start address 218) + 64) % 256, as previously described with respect to Figure 2 Explained.

[0096] In a particular implementation, the particular operation corresponds to a read operation for reading a block of pixel data from virtual memory, and dynamically mapping the target virtual address to the particular memory address includes locating the virtual memory start address of the pixel data block in a data structure that associates the virtual memory start address with a portion of on-chip memory. Figure 2 When pixel data block 220 associated with Quad #5 in the image processing unit 100 is mapped to the pixel data block 220 associated with Quad #5 in the image processing unit 100, the target memory address of portion 222 of virtual memory 140 (e.g., row 0, column 192) is located in the start address array 204 (e.g., "192" is located in the right element of the TopStartAddr array (TopStartAddr(1)=192)) and is therefore mapped to the upper right portion of on-chip memory 150 that stores the pixel data for Quad #5.

[0097] Figure 13 The method 1300 may be implemented by a field programmable gate array (FPGA) device, an application specific integrated circuit (ASIC), a processing unit (such as a central processing unit (CPU)), a digital signal processor (DSP), a controller, another hardware device, a firmware device, or any combination thereof. As an example, Figure 13 The method 1300 may be performed by a processor executing instructions, such as reference Figure 14 described.

[0098] refer to Figure 14, depicts a block diagram of a particular exemplary implementation of a device and generally designated 1400. In various implementations, the device 1400 may have more than Figure 14 More or less components than shown. In an exemplary implementation, the device 1400 may be connected to Figure 1 In an exemplary embodiment, the device 1400 may execute the reference Figures 1 to 13 One or more operations described.

[0099] In certain implementations, the device 1400 includes a processor 1406 (e.g., a CPU). The device 1400 may include one or more additional processors 1410 (e.g., one or more DSPs). Figure 1 The one or more processors 116 of the processor 1406 may correspond to the processor 1410, or a combination thereof. For example, the processor 1410 may include the decoder 124 and the speech and music coder-decoder (CODEC) 1408. The speech and music CODEC 1408 may include a speech coder ("vocoder") encoder 1436, a vocoder decoder 1438, or a combination thereof.

[0100] Device 1400 may include memory 1486 and codec 1434. Memory 1486 may include instructions 1456 that are executable by one or more additional processors 1410 (or processor 1406) to implement the functionality described with reference to decoder 124. In the specific example, memory 1486 corresponds to memory 110, and instructions 1456 correspond to Figure 1 The device 1400 may include a modem 118 coupled to an antenna 1452 via a transceiver 1450 .

[0101] Device 1400 may include a display 1428, such as display device 104, coupled to a display controller 1426. One or more speakers 1436, one or more microphones 1460, or a combination thereof may be coupled to codec 1434. Codec 1434 may include a digital-to-analog converter (DAC) 1402 and an analog-to-digital converter (ADC) 1404. In a particular implementation, codec 1434 may receive an analog signal from microphone 1460, convert the analog signal to a digital signal using analog-to-digital converter 1404, and transmit the digital signal to speech and music codec 1408. In a particular implementation, speech and music codec 1408 may provide the digital signal to codec 1434. Codec 1434 may convert the digital signal to an analog signal using digital-to-analog converter 1402 and may provide the analog signal to speaker 1436.

[0102] In a particular implementation, device 1400 can be included in a system-in-package or system-on-chip device 1422. In a particular implementation, memory 1486, processor 1406, processor 1410, display controller 1426, codec 1434, and modem 118 are included in the system-in-package or system-on-chip device 1422. In a particular implementation, input device 1430 (e.g., a keyboard, touch screen, or pointing device) and power supply 1444 are coupled to the system-in-package or system-on-chip device 1422. Additionally, in a particular implementation, as shown in FIG. Figure 14 As illustrated, the display 1428, input device 1430, speaker 1436, microphone 1460, antenna 1452, and power supply 1444 are external to the system-in-package or system-on-chip device 1422. In a particular implementation, each of the display 1428, input device 1430, speaker 1436, microphone 1460, antenna 1452, and power supply 1444 may be coupled to a component of the system-in-package or system-on-chip device 1422, such as an interface or a controller.

[0103] Device 1400 may include a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop, a computer, a tablet, a personal digital assistant, a display device, a television, a game console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a head-mounted device, an augmented reality head-mounted device, a mixed reality head-mounted device, a virtual reality head-mounted device, an aerial vehicle, a home automation system, a voice-activated device, a wireless speaker and voice-activated device, a portable electronic device, a car, a vehicle, a computing device, a communication device, an Internet of Things (IoT) device, a virtual reality (VR) device, a base station, a mobile device, or any combination thereof.

[0104] In conjunction with the described techniques, an apparatus includes means for identifying, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of video data, a target virtual address for a data access associated with a particular operation of the IBC decoding process, the target virtual address being generated based on an addressing scheme of a virtual memory used by the IBC decoding process. In one example, the means for identifying the target virtual address includes an IBC prediction engine 130, a decoder 124, one or more processors 116, a device 102, a prediction processing unit 304, one or more other circuits or devices for identifying, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of a video signal, a target virtual address for a data access associated with a particular operation of the IBC decoding process, or a combination thereof.

[0105] The apparatus includes means for dynamically mapping a target virtual address to a specific memory address of a portion of on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory. In one example, the means for dynamically mapping includes a dynamic mapper 160, an IBC prediction engine 130, a decoder 124, one or more processors 116, a device 102, a prediction processing unit 304, a fetch command generator 402, an IBC data fetcher 404, one or more other circuits or devices for dynamically mapping the target virtual address to the specific memory address of the portion of on-chip memory, or a combination thereof.

[0106] The apparatus further includes means for accessing the on-chip memory using a specific memory address to perform a specific operation of the IBC decoding process. In one example, accessing the on-chip memory includes the IBC prediction engine 130, the decoder 124, one or more processors 116, the device 102, the prediction processing unit 304, the reconstruction engine 310, the loop filter 312, the IBC data fetcher 404, the reconstruction engine 410, one or more other circuits or devices for accessing the on-chip memory using a specific memory address to perform a specific operation of the IBC decoding process, or a combination thereof.

[0107] In some implementations, a non-transitory computer-readable medium (e.g., a computer-readable storage device such as memory 110) includes instructions (e.g., instructions 112) that, when executed by one or more processors (e.g., one or more processors 116), cause the one or more processors to perform operations related to the referenced Figures 1 to 12 The technology described, Figure 13 The method of claim 1, or any combination thereof, or at least a portion of any corresponding operation.

[0108] Certain aspects of the disclosure are described below in the following set of related clauses:

[0109] According to clause 1, a device includes: a decoder configured to: identify a target virtual address for data access associated with a specific operation of an intra block copy (IBC) decoding process performed on at least a portion of a decoding unit of video data during the IBC decoding process, the target virtual address being generated according to an addressing scheme of a virtual memory used by the IBC decoding process; dynamically map the target virtual address to a specific memory address of a portion of an on-chip memory, the on-chip memory being configured to store a reconstructed block of the video data and having a second size that is smaller than a first size of the virtual memory; and access the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

[0110] Clause 2. A device according to clause 1, wherein in response to the particular operation corresponding to a write operation for writing a block of pixel data to an unused portion of the virtual memory, the decoder is configured to store the virtual memory start address of the block of pixel data in a data structure that associates the virtual memory start address with a portion of the on-chip memory.

[0111] Clause 3. An apparatus according to clause 2, wherein the virtual memory start address of the pixel data block is based on a virtual memory start address of another storage associated with another portion of the virtual memory with an increased block size and a modulus associated with a row size of the virtual memory.

[0112] Clause 4. A device according to clause 1, wherein in response to the particular operation corresponding to a read operation for reading a block of pixel data from the virtual memory, the decoder is configured to locate a virtual memory start address of the block of pixel data in a data structure that associates a virtual memory start address with a portion of the on-chip memory.

[0113] Clause 5. The apparatus of any one of clauses 1 to 4, wherein the on-chip memory and the decoder are integrated into a single substrate.

[0114] Clause 6. The apparatus of any one of clauses 1 to 5, wherein the first size is at least twice the second size.

[0115] Clause 7. The apparatus of any one of clauses 1 to 6, wherein the on-chip memory comprises a static random access memory (SRAM) IBC ring buffer.

[0116] Clause 8. Apparatus as recited in any one of clauses 1 to 7, wherein the addressing scheme corresponds to a Versatile Video Coding (VVC) addressing scheme for IBC decoding.

[0117] Clause 9. The apparatus of any one of clauses 1 to 8, wherein the decoder is further configured to access the on-chip memory during performance of loop filtering operations associated with the IBC decoding process.

[0118] Clause 10. The apparatus of any one of clauses 1 to 9, further comprising a modem coupled to the decoder, the modem configured to receive the video data from a second device.

[0119] Clause 11. A device according to any one of clauses 1 to 10, wherein the decoder and the on-chip memory are integrated into at least one of the following: a mobile phone, a tablet computer device, a wearable electronic device, a camera device, a virtual reality head-mounted device, a mixed reality head-mounted device, or an augmented reality head-mounted device.

[0120] Clause 12. The apparatus of any one of Clauses 1 to 10, wherein the decoder and the on-chip memory are integrated in a vehicle, the vehicle further comprising a display device configured to display video data output by the decoder.

[0121] According to clause 13, a method for decoding video data comprises: during an intra block copy (IBC) decoding process on at least a portion of a decoding unit of the video data, identifying, at one or more processors, a target virtual address for data access associated with a specific operation of the IBC decoding process, the target virtual address being generated based on an addressing scheme of a virtual memory used by the IBC decoding process; dynamically mapping, at the one or more processors, the target virtual address to a specific memory address of a portion of an on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory; and accessing the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

[0122] Clause 14. The method of clause 13, wherein prior to dynamically mapping the target virtual address to the specific memory address, a previously used virtual address in the virtual memory is mapped to the specific memory address.

[0123] Clause 15. A method according to clause 13 or clause 14, wherein: the specific operation corresponds to a write operation for writing a pixel data block to an unused portion of the virtual memory, and dynamically mapping the target virtual address to the specific memory address includes: storing the virtual memory start address of the pixel data block in a data structure that associates the virtual memory start address with a portion of the on-chip memory, and the virtual memory start address of the pixel data block is based on another stored virtual memory start address associated with another portion of the on-chip memory with an increased block size and a modulus associated with the row size of the virtual memory.

[0124] Clause 16. A method according to clause 13 or clause 14, wherein: the specific operation corresponds to a read operation for reading a pixel data block from the virtual memory, and dynamically mapping the target virtual address to the specific memory address includes: locating the virtual memory start address of the pixel data block in a data structure that associates the virtual memory start address with a portion of the on-chip memory.

[0125] Clause 17. The method of any one of Clauses 13 to 16, wherein the on-chip memory comprises a static random access memory (SRAM) IBC ring buffer.

[0126] Clause 18. The method of any one of clauses 13 to 17, wherein the addressing scheme corresponds to a Versatile Video Coding (VVC) addressing scheme for IBC decoding.

[0127] According to clause 19, an apparatus comprises: a memory configured to store instructions; and a processor configured to execute the instructions to perform the method according to any one of clauses 13 to 18.

[0128] According to clause 20, a non-transitory computer-readable medium comprises instructions that, when executed by a processor, cause the processor to perform the method according to any one of clauses 13 to 18.

[0129] According to clause 21 , an apparatus comprises means for performing the method according to any one of clauses 13 to 18.

[0130] According to clause 22, a non-transitory computer-readable medium includes instructions that, when executed by one or more processors, cause the one or more processors to: identify a target virtual address for data access associated with a specific operation of an intra block copy (IBC) decoding process during an IBC decoding process of at least a portion of a decoding unit of video data, the target virtual address being generated based on an addressing scheme of a virtual memory used by the IBC decoding process; dynamically map the target virtual address to a specific memory address of a portion of an on-chip memory, the on-chip memory having a second size that is smaller than a first size of the virtual memory; and access the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process.

[0131] Clause 23. A non-transitory computer-readable medium according to clause 22, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to update a data structure associating a virtual memory start address with a portion of the on-chip memory in response to the particular operation corresponding to a write operation.

[0132] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media or communication media, which corresponds to tangible media such as data storage media, which includes, for example, any media that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

[0133] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connector is appropriately referred to as a computer-readable medium. For example, if an instruction is sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connectors, carrier waves, signals or other transient media, but are directed to non-transient tangible storage media. As used herein, disks and optical disks include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein disks typically reproduce data magnetically, while optical discs utilize lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0134] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, as used herein, the term "processor" may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Additionally, these techniques may be fully implemented in one or more circuits or logic elements.

[0135] The techniques of this disclosure can be implemented in a variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Specifically, as described above, the various units may be combined in a codec hardware unit, or may be provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0136] Those skilled in the art will also recognize that the various exemplary logic blocks, configurations, circuits, and algorithmic steps described in conjunction with the specific implementations disclosed herein may be implemented as electronic hardware, computer software executed by a processing device (such as a hardware processor), or a combination of the two. Various exemplary components, blocks, configurations, circuits, and steps have been generally described above with respect to their functionality. Whether such functionality is implemented as hardware or executable software depends on the specific application and the design constraints imposed on the entire system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be interpreted as resulting in a departure from the scope of this disclosure.

[0137] The steps of the methods or algorithms described in conjunction with the specific implementations disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a memory device such as a random access memory (RAM), a magnetoresistive random access memory (MRAM), a spin torque transfer MRAM (STT-MRAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of non-transitory storage medium known in the art. An exemplary memory device is coupled to a processor so that the processor can read information from the memory device and write information to the memory device. In an alternative embodiment, the memory device may be integrated with the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). The ASIC may reside in a computing device or a user terminal. In an alternative embodiment, the processor and the storage medium may reside in a computing device or a user terminal as discrete components.

[0138] The preceding description of the disclosed implementations is provided to enable those skilled in the art to make or use the disclosed implementations. Various modifications to these implementations will be apparent to those skilled in the art, and the principles defined herein may be applied to other implementations without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the specific implementations shown herein, but should be accorded the widest scope possible consistent with the principles and novel features as defined by the appended claims.

Claims

1. A device, comprising: A decoder, the decoder being configured to: identifying, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of video data, a target virtual address in a virtual address space for data access associated with a particular operation of the IBC decoding process, the target virtual address being generated according to an addressing scheme of an on-chip memory used by the IBC decoding process; dynamically mapping the target virtual address to a specific memory address of a portion of the on-chip memory, The on-chip memory is configured to store reconstructed blocks of the video data and has a second size that is smaller than the first size of the virtual address space; as well as accessing the on-chip memory using the specific memory address to perform the specific operation of the IBC decoding process, In response to the particular operation corresponding to a read operation for reading a block of pixel data from the on-chip memory, the decoder is configured to locate an on-chip memory start address of the block of pixel data in a data structure associating a virtual start address with a portion of the on-chip memory.

2. The apparatus of claim 1 , wherein in response to the specific operation corresponding to a write operation for writing a block of pixel data to an unused portion of the on-chip memory, the decoder is configured to store a virtual starting address of the block of pixel data into the data structure.

3. The apparatus of claim 2 , wherein the virtual starting address of the block of pixel data is based on another stored virtual starting address associated with another portion of the on-chip memory that has an increased block size and a modulus associated with a row size of the virtual address space.

4. The apparatus of claim 1 , wherein the on-chip memory is configured as a ring buffer to store pixel data of a current block of the video data and a reconstructed block of the video data that is within a range of permitted motion vectors for IBC decoding, and wherein the ring buffer overwrites pixel data of the reconstructed block that is no longer within the range of permitted motion vectors for the current block.

5. The apparatus of claim 1, wherein the on-chip memory and the decoder are integrated into a single substrate. The device of claim 1 , wherein the first size is at least twice the second size.

7. The apparatus of claim 1, wherein the on-chip memory comprises a static random access memory (SRAM) IBC ring buffer.

8. The apparatus of claim 1, wherein the addressing scheme corresponds to a Versatile Video Coding (VVC) addressing scheme for IBC decoding.

9. The apparatus of claim 1, wherein the decoder is further configured to access the on-chip memory during execution of an in-loop filtering operation associated with the IBC decoding process.

10. The device according to claim 1, further comprising: A modem is coupled to the decoder, the modem being configured to receive the video data from a second device.

11. The device of claim 1 , wherein the decoder and the on-chip memory are integrated into at least one of: a mobile phone, a tablet computer device, a wearable electronic device, a camera device, a virtual reality head-mounted device, a mixed reality head-mounted device, or an augmented reality head-mounted device.

12. The apparatus of claim 1, wherein the decoder and the on-chip memory are integrated in a vehicle, the vehicle further comprising a display device configured to display video data output by the decoder.

13. A device comprising: A decoder, the decoder being configured to: identifying, during an intra block copy (IBC) decoding process for at least a portion of a decoding unit of video data, a target virtual address in a virtual address space for data access associated with a particular operation of the IBC decoding process, the target virtual address being generated according to an addressing scheme of an on-chip memory used by the IBC decoding process; dynamically mapping the target virtual address to a particular memory address of a portion of the on-chip memory based on at least one of a current buffer index and an array of start addresses associated with data in the on-chip memory, the on-chip memory being configured to store reconstructed blocks of the video data and having a second size that is less than the first size of the virtual address space; as well as The specific memory address is used to access the on-chip memory to perform the specific operation of the IBC decoding process.

14. The apparatus of claim 13 , wherein the reconstructed block of the video data comprises a sequence of groups of four blocks of the video data, wherein the on-chip memory is sized to store a single group of four blocks of the video data, and wherein the virtual address space is sized for multiple groups of four blocks of the video data.

15. The apparatus of claim 14 , wherein the on-chip memory is configured as a ring buffer to store pixel data of a current block of the video data and three reconstructed blocks of the video data that are within a range of permitted motion vectors for IBC decoding, and wherein the ring buffer overwrites pixel data of reconstructed blocks that are no longer within the range of permitted motion vectors for the current block.

16. The apparatus of claim 15, wherein the circular buffer comprises a top row for storing two blocks of the video data and a bottom row for storing two blocks of the video data.

17. The apparatus of claim 16 , wherein when the target virtual address corresponds to a block of video data in a top row of a 2×2 block group, the decoder is configured to select a storage location in the top row of the circular buffer based on a current top buffer index, and when the target virtual address corresponds to a block of video data in a bottom row of the 2×2 block group, the decoder is configured to select a storage location in the bottom row of the circular buffer based on a current bottom buffer index.

18. The apparatus of claim 17, wherein the decoder is configured to update one of the current top buffer index or the current bottom buffer index in conjunction with storing a reconstructed block to the circular buffer.

19. The apparatus of claim 15, wherein the decoder is configured to update the start address array to store an indicator of a virtual start address of the reconstructed block in conjunction with storing the reconstructed block to the ring buffer.

20. A method for decoding video data, the method comprising: During an intra block copy (IBC) decoding process on at least a portion of a decoding unit of video data, identifying, at one or more processors, a target virtual address in a virtual address space for a data access associated with a particular operation of the IBC decoding process, the target virtual address being generated according to an addressing scheme of an on-chip memory used by the IBC decoding process; dynamically mapping, at the one or more processors, the target virtual address to a particular memory address of a portion of the on-chip memory based on at least one of a current buffer index and an array of starting addresses associated with data in the on-chip memory, the on-chip memory being configured to store reconstructed blocks of the video data and having a second size that is less than a first size of the virtual address space; as well as The specific memory address is used to access the on-chip memory to perform the specific operation of the IBC decoding process.

Images