Method of operation and apparatus for multi-codec circuitry

Abstract

This application relates to methods of operating and apparatuses of multi-codec circuits. A method of operating a multi-codec circuit configured to support a plurality of codecs can include selecting a current coding tree unit (CTU) from a plurality of CTUs partitioned from an input image, selecting a current coding unit (CU) from a plurality of CUs partitioned from the current CTU, determining availability of neighboring CUs of the current coding unit based on a coding unit bitmap, and performing signal processing on the current CU using at least one of the neighboring CUs determined to be available. The coding unit bitmap can include a plurality of bits, each bit indicating whether signal processing has been performed on a corresponding coding unit among the plurality of CUs. Each of the plurality of bits can correspond to a minimum CU supported by each of the plurality of codecs.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2024-0174780, filed on November 29, 2024, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The embodiments of this disclosure described herein relate to image processing, and more specifically, to a method of operating a multi-codec circuit for processing image data. Background Technology

[0004] Recently, with the increase in the resolution of display devices, the size of image data displayed by these devices has also increased. In order to effectively manage large amounts of image data, various video codecs have been developed and are being actively used.

[0005] Each of the various video codecs is standardized in various forms. To support a variety of video codecs, it may be necessary to develop separate video codec encoders and video codec decoders that conform to each standard. Summary of the Invention

[0006] Embodiments of this disclosure provide a method for operating a multi-codec circuit that achieves reduced cost and improved performance.

[0007] According to embodiments of this disclosure, a method for a multi-codec circuit configured to support multiple codecs includes: selecting a current CTU from a plurality of coding tree units (CTUs) partitioned from an input image; selecting a current CU from a plurality of coding units (CUs) partitioned from the current CTU; determining the availability of neighboring CUs of the current CU based on a codec unit bitmap; and performing signal processing on the current CU using at least one available neighboring CU based on the result of determining the availability of neighboring CUs. The codec unit bitmap may include a plurality of bits, each of which indicates whether signal processing has been performed on a corresponding CU among the plurality of CUs. Each of the plurality of bits corresponds to the smallest CU supported by each of the plurality of codecs.

[0008] According to embodiments of this disclosure, a method for a multi-codec circuit configured to support multiple codecs includes: determining the availability of a first neighboring CU of a first CU among a plurality of codec units (CUs) divided from a first codec tree unit (CTU) based on a codec unit bitmap; performing first signal processing on the first CU using the first neighboring CU based on the first neighboring CU being determined to be available for first signal processing; setting a first current codec bit in the codec unit bitmap corresponding to the first CU to a first value based on the completion of the first signal processing; determining the availability of a second neighboring CU of a second CU among the plurality of CUs based on the codec unit bitmap; performing second signal processing on the second CU using the second neighboring CU based on the second neighboring CU being determined to be available for second signal processing; and setting a second current codec bit in the codec unit bitmap corresponding to the second CU to a first value based on the completion of the second signal processing. The codec unit bitmap may include a plurality of bits indicating whether signal processing has been performed on the plurality of CUs. Each of the plurality of bits may correspond to the smallest CU supported by each of the plurality of codecs.

[0009] According to embodiments of this disclosure, a method for a multi-codec circuit configured to support multiple codecs includes: clearing a codec unit bitmap to a reset value; determining the availability of a first adjacent CU of a first codec unit (CU) in a first codec tree unit (CTU) based on the codec unit bitmap; performing first signal processing on the first CU using the first adjacent CU based on the first adjacent CU being determined to be available for first signal processing; setting a first current codec bit in the codec unit bitmap corresponding to the first CU to a first value; clearing the codec unit bitmap to a reset value; determining the availability of a second adjacent CU of a second CU in a second CTU based on the codec unit bitmap; performing second signal processing on the second CU using the second adjacent CU based on the second adjacent CU being determined to be available for second signal processing; and setting a second current codec bit in the codec unit bitmap corresponding to the second CU to a first value. The size of the first CTU is different from the size of the second CTU. The codec unit bitmap may include multiple bits indicating whether signal processing has been performed on the plurality of CUs including the first CU, the second CU, the first adjacent CU, and the second adjacent CU. Each of the multiple bits corresponds to the minimum CU supported by each of the multiple codecs.

[0010] According to embodiments of this disclosure, an electronic device may include: a memory, storage instructions, and a codec unit bitmap; and at least one processor configured to execute instructions to: select a current CTU from a plurality of codec tree units (CTUs) partitioned from an input image, select a current CU from a plurality of codec units (CUs) partitioned from the current CTU, determine the availability of neighboring CUs of the current CU based on the codec unit bitmap including a plurality of bits, each of the plurality of bits indicating whether signal processing has been performed on the corresponding CU among the plurality of CUs, perform signal processing on the current CU using at least one of the neighboring CUs determined to be available, and update the codec bit in the codec unit bitmap corresponding to the current CU based on the completion of signal processing on the current CU. Attached Figure Description

[0011] The above and other objects and features of this disclosure will become apparent from the detailed description of its embodiments with reference to the accompanying drawings.

[0012] Figure 1 This is a diagram illustrating a system 1000 according to one or more embodiments of the present disclosure.

[0013] Figure 2 This illustrates one or more embodiments. Figure 1 Block diagram of a multi-codec circuit.

[0014] Figure 3 , Figure 4A and Figure 4B It is a diagram used to describe the codec tree unit and codec unit for input image according to one or more embodiments.

[0015] Figure 5 It is a diagram used to describe the codec unit partitioning method used in various codecs according to one or more embodiments.

[0016] Figure 6A and 6B This is a diagram illustrating the operations for determining the availability of adjacent codec units according to one or more embodiments.

[0017] Figure 7 This illustrates one or more embodiments. Figure 1 Block diagram of a multi-codec circuit.

[0018] Figure 8 This illustrates one or more embodiments. Figure 7 A block diagram of the encoder / decoder unit availability check block.

[0019] Figure 9 This illustrates one or more embodiments. Figure 7 The flowchart shows the operation of the multi-codec circuit.

[0020] Figure 10A , 10B 11A and 11B illustrate one or more embodiments Figure 8 The operation of the encoder / decoder unit availability check block.

[0021] Figure 12A and 12B An illustration is provided according to one or more embodiments. Figure 8 The operation of the encoder / decoder unit availability check block.

[0022] Figure 13A , 13B 14A and 14B illustrate embodiments according to one or more of the following. Figure 8 The operation of the encoder / decoder unit availability check block.

[0023] Figure 15A and 15B Illustrations according to one or more embodiments Figure 8 The operation of the encoder / decoder unit availability check block.

[0024] Figure 16A and 16B An illustration is provided according to one or more embodiments. Figure 8 The operation of the encoder / decoder unit availability check block.

[0025] Figure 17 The illustration shows a method for using one or more embodiments of a device. Figure 1 The input image is processed by a multi-codec circuit, consisting of a codec tree unit and a codec unit.

[0026] Figure 18 This is a block diagram schematically illustrating a multicodec circuit according to one or more embodiments of the present disclosure. Detailed Implementation

[0027] In the following, embodiments of the present disclosure will be described in detail and clearly to the extent that they can be readily implemented by those skilled in the art.

[0028] In the following detailed description, the components described, such as “part” (or “unit”), “block”, “module”, etc., and the functional blocks illustrated in the accompanying drawings, may be implemented in software, hardware, or a combination thereof. For example, hardware may include electrical circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, pressure sensors, inertial sensors, microelectromechanical systems (MEMS), passive components, or a combination thereof.

[0029] Figure 1 This is a diagram of a system 1000 including a storage device according to one or more embodiments. Figure 1System 1000 can be a mobile system, such as a portable communication terminal (e.g., a mobile phone), a smartphone, a tablet PC, a wearable device, a healthcare device, or an Internet of Things (IoT) device. However, Figure 1 The system 1000 is not limited to mobile systems and can be a PC, laptop computer, server, media player or automotive device (e.g., navigation device).

[0030] refer to Figure 1 System 1000 may include a main processor 1100, memories 1200a and 1200b, and storage devices 1300a and 1300b. Furthermore, system 1000 may include at least one of an image capture device 1410, a user input device 1420, a sensor 1430, a communication device 1440, a display 1450, a speaker 1460, a power supply device 1470, and a connection interface 1480.

[0031] The main processor 1100 can control all operations of the system 1000, and more specifically, control the operations of other components included in the system 1000. The main processor 1100 can be implemented as a general-purpose processor, a special-purpose processor, or an application processor.

[0032] The main processor 1100 may include at least one central processing unit (CPU) core 1110 and a controller 1120 configured to control memories 1200a and 1200b and / or storage devices 1300a and 1300b. In some embodiments, the main processor 1100 may also include an accelerator 1130, which is dedicated circuitry for high-speed data operations, such as artificial intelligence (AI) data operations. The accelerator 1130 may include a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU) and may be implemented as a chip physically separate from other components of the main processor 1100.

[0033] Memory 1200a and 1200b can be used as the main memory device of system 1000. Although each of memory 1200a and 1200b may include volatile memory, such as static random access memory (SRAM) and / or dynamic RAM (DRAM), each of memory 1200a and 1200b may include non-volatile memory, such as flash memory, phase-change RAM (PRAM), and / or resistive RAM (RRAM). Memory 1200a and 1200b can be implemented in the same package as main processor 1100.

[0034] Storage devices 1300a and 1300b can be used as non-volatile storage devices configured to store data regardless of whether they are powered on and having a larger storage capacity than memories 1200a and 1200b. Storage devices 1300a and 1300b may each include storage controllers (STRG CTRL) 1310a and 1310b and non-volatile memory (NVM) 1320a and 1320b configured to store data via the control of storage controllers 1310a and 1310b. While NVMs 1320a and 1320b may include flash memory with a two-dimensional (2D) or three-dimensional (3D) V-NAND structure, NVMs 1320a and 1320b may also include other types of NVMs, such as PRAM and / or RRAM.

[0035] Storage devices 1300a and 1300b can be physically separated from the main processor 1100 in system 1000, or implemented in the same package as the main processor 1100. Additionally, storage devices 1300a and 1300b can be of the type of solid-state drive (SSD) or memory card, and can be removably combined with other components of system 1000 via an interface such as connection interface 1480 described below. Storage devices 1300a and 1300b can operate according to standard protocols, such as Universal Flash Memory (UFS), embedded multimedia card (eMMC), or Non-Volatile Memory Fast (NVMe), but are not limited to these.

[0036] Image capture device 1410 can capture still images or moving images. Image capture device 1410 may include a camera, a portable video camera, and / or a webcam.

[0037] User input device 1420 can receive various types of data input by the user of system 1000, and includes a touchpad, keypad, keyboard, mouse and / or microphone.

[0038] Sensor 1430 can detect various types of physical quantities that can be obtained from outside the system 1000 and convert the detected physical quantities into electrical signals. Sensor 1430 may include temperature sensors, pressure sensors, illuminance sensors, position sensors, acceleration sensors, biosensors, and / or gyroscope sensors.

[0039] Communication device 1440 can send and receive signals between other devices outside system 1000 according to various communication protocols. Communication device 1440 may include an antenna, transceiver, and / or modem.

[0040] The display 1450 and the speaker 1460 can be used as output devices configured to output visual and auditory information to the user of the system 1000, respectively.

[0041] The power supply device 1470 can appropriately convert power supplied from a battery and / or an external power source embedded in the system 1000 and supply the converted power to each of the components of the system 1000.

[0042] The connection interface 1480 provides a connection between the system 1000 and an external device that connects to the system 1000 and can send and receive data from the system 1000. The connection interface 1480 can be implemented using various interface schemes, such as Advanced Technology Attachment (ATA), Serial ATA (SATA), External SATA (e-SATA), Small Computer Small Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI Express (PCIe), NVMe, IEEE 1394, Universal Serial Bus (USB) interface, Secure Digital (SD) card interface, Multimedia Card (MMC) interface, eMMC interface, UFS interface, Embedded UFS (eUFS) interface, and Compact Flash (CF) card interface.

[0043] In one or more embodiments, the main processor 1100 may be a system-on-a-chip (SoC) that includes function blocks or intellectual property (IP) blocks configured to implement various functions. As an example, the main processor 1100 may also include multi-codec circuitry 1140. Multi-codec circuitry 1140 may be logic or hardware circuitry configured to support various video / image codecs.

[0044] As an example, the multi-codec circuit 1140 can be configured to support various video / image codecs, such as H.264, High-Efficiency Video Codec (HEVC), High-Efficiency Image Codec (HEIC), AOMedia Video 1 (AV1), Universal Video Codec (VVC), Audio Video Codec Standard 3 (AVS3), etc. The multi-codec circuit 1140 can perform encoding and decoding operations corresponding to each of the multiple codecs. In one or more embodiments, the various video / image codecs described above can have their own appropriate standards, and the multi-codec circuit 1140 can perform encoding and decoding operations as defined in the corresponding codec standards.

[0045] For example, to meet standards defined by various codecs, each of the various codecs may require a separate component, and in this case, implementing the multi-codec circuit 1140 may be cumbersome. Additionally, when additional codecs are developed in addition to existing codecs, separate components (means) may be needed for applying the additional codecs.

[0046] According to one or more embodiments of the present disclosure, the multi-codec circuit 1140 may include logic circuitry configured to check the availability of adjacent codec units for each of various codecs (e.g., whether signal processing for adjacent codec units is complete). In this case, the availability of adjacent codec units can be easily determined even if the size, shape, order, or partitioning method of the codec units varies across different codecs. The configuration of the multi-codec circuit 1140 according to one or more embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings.

[0047] Figure 2 This illustrates one or more embodiments. Figure 1 A block diagram of a multi-codec circuit. For ease of description, Figure 2 Some configurations of the multicodec circuitry 1140 are shown, but the scope of this disclosure is not limited thereto. Figure 2 The multicoder circuit 1140 is described as performing encoding and decoding based on the High Efficiency Video Codec (HEVC, also known as H.265), but the scope of this disclosure is not limited thereto, and the multicoder circuit 1140 can perform or support encoding and decoding corresponding to various video / image codecs.

[0048] In the following text, the terms "signal processing for an input image," "code-decode tree unit," or "code-decode unit" will be used. In this context, signal processing can refer to the encoding or decoding of target information (e.g., an input image, a code-decode tree unit, or a code-decode unit) based on a corresponding codec. For example, signal processing for a code-decode unit can refer to performing operations such as intra-frame prediction, motion prediction, and motion compensation corresponding to the code-decode unit. However, the scope of this disclosure is not limited thereto.

[0049] In one or more embodiments, the multi-codec circuit 1140 can perform signal processing on the input image IMG_IN based on codec tree units (CTUs). For example, the multi-codec circuit 1140 can divide the input image IMG_IN into multiple codec tree units. A codec tree unit (CTU) can be a basic processing unit used for inter-frame prediction or other signal processing. The multi-codec circuit 1140 can further divide each of the multiple codec tree units into smaller codec units (CUs). The multi-codec circuit 1140 can perform inter-frame prediction or signal processing on a unit-by-unit basis.

[0050] In the following description, for ease of description, the multi-codec circuit 1140 is described as performing signal processing on a unit basis (CU). However, the scope of this disclosure is not limited thereto. For example, depending on the type or kind of codec supported, a codec tree unit (CTU) can be divided into codec units (CU) or prediction units (PU). Alternatively, a codec unit (CU) can be further divided into codec blocks or prediction blocks. In this case, the multi-codec circuit 1140 can perform signal processing on either the codec unit (CU) or the prediction unit (PU). As an example, when the multi-codec circuit 1140 performs signal processing on a prediction unit (PU), the embodiments described below can be applied to determine the availability of adjacent prediction units (PU). As described above, in the signal processing for the input image IMG_IN, terms such as codec tree unit (CTU), codec unit (CU), code block (CB), prediction unit (PU), prediction block (PB), and superblock can be used interchangeably depending on the type or kind of codec. In this case, it will be understood that the embodiments of this disclosure are not limited to the above terms, and the embodiments of this disclosure can be applied to check the availability of adjacent codec units, adjacent prediction units, or adjacent blocks.

[0051] refer to Figure 1 and Figure 2 The multi-codec circuit 1140 can encode the input image IMG_IN to generate a bitstream BST in which the input image IMG_IN is compressed. Alternatively, the multi-codec circuit 1140 can decode the bitstream BST to generate an output image IMG_OUT. The output image IMG_OUT can be displayed via an external display device (e.g., Figure 1 The display (1450) can be used for display, or the data can be stored in a separate memory.

[0052] The multicoder circuit 1140 may include a motion estimation block 1141a, a motion compensation block 1141b, an intra-frame prediction block 1142, a mode selection block SEL, a transform and quantization block 1143, an entropy coding block 1144, an entropy decoding block 1145, an inverse quantization and inverse transform block 1146, a deblocking filter 1147, and a memory interface block 1148.

[0053] Motion estimation block 1141a can estimate the motion between the input image IMG_IN and a reference image. Motion estimation block 1141a can output motion information corresponding to the estimated motion. In one or more embodiments, the reference image can be an image of a previous frame of the input image IMG_IN and can be stored in a separate buffer memory (e.g., memory connected to memory interface block 1148). Motion compensation block 1141b can receive motion information from motion estimation block 1141a and can perform motion compensation based on the motion information. In one or more embodiments, inter-frame prediction for the input image IMG_IN can be performed by motion estimation block 1141a and motion compensation block 1141b.

[0054] Intra-prediction block 1142 can be configured to perform intra-prediction on the input image IMG_IN. For example, intra-prediction block 1142 can use information from the signal-processed codec unit in a codec unit adjacent to the currently signal-processed codec unit to perform spatial prediction.

[0055] The mode selection block SEL can be configured to select whether the current input image IMG_IN, the current codec tree unit (CTU), or the current codec unit (CU) is in inter-frame prediction mode or intra-frame prediction mode. The result selected by the mode selection block SEL can be combined with the input image IMG_IN and can be provided to the transform and quantization block 1143.

[0056] Transform and quantization block 1143 can transform video information into frequency domain data. In one or more embodiments, transform and quantization block 1143 can transform received data into frequency domain data based on discrete cosine transform (DCT) or other various transform examples. Transform and quantization block 1143 can quantize frequency domain data.

[0057] Entropy coding block 1144 can perform entropy coding on the output of transform and quantization block 1143 to generate a bit stream BST. Entropy coding block 1144 can be configured to perform various coding methods, such as Huffman coding, run-length encoding, differential coding, and similar coding methods, but the scope of the invention is not limited thereto. In one or more embodiments, the bit stream BST can be stored in a separate external memory.

[0058] Entropy decoding block 1145 can perform entropy decoding on bit stream BST. In this case, bit stream BST can be an entropy-encoded bit stream. In one or more embodiments, entropy decoding block 1145 can perform the inverse transform of entropy encoding block 1144.

[0059] The inverse quantization and inverse transform block 1146 can inverse quantize quantized data and transform the domain of the inverse quantized data.

[0060] The deblocking filter block 1147 performs filtering operations to reduce discontinuities at the boundaries of an image or codec unit and attenuate ringing artifacts and sample distortion. In one or more embodiments, the deblocking filter block 1147 may also include sample adaptive offset (SAO).

[0061] The memory interface block 1148 can be configured to store the results of signal processing performed by the multi-codec circuit 1140 in external memory, or to transmit and receive the results of signal processing or frame images stored in external memory. In one or more embodiments, the encoded bitstream BST can be stored in external memory via the memory interface block 1148. Alternatively, the decoded output image IMG_OUT can be stored in external memory via the memory interface block 1148. The output image IMG_OUT stored in external memory can be displayed by a separate display device.

[0062] As described above, the multi-codec circuit 1140 can encode the input image IMG_IN to generate a compressed bitstream BST of the input image IMG_IN, or it can decode the bitstream BST to generate an output image IMG_OUT.

[0063] Figure 2 The configuration of the multicoder circuit 1140 is merely an example, and the scope of this disclosure is not limited thereto. For example, the multicoder circuit 1140 may include hardware configurations configured to support various video / image codecs such as H.264, HEVC, HEIC, AV1, VVC, AVS3, etc.

[0064] In one or more embodiments, the intra-prediction block 1142 of the multi-codec circuit 1140 can perform spatial prediction using the signal processing results of the current codec unit and neighboring codec units. In this case, in order to use the signal processing results of neighboring codec units, it should be determined whether the signal processing for the neighboring codec unit is complete. In this disclosure, whether the signal processing for the neighboring codec unit is complete is referred to as the "availability of the neighboring codec unit CU". That is, a neighboring codec unit considered "available" indicates that the signal processing for the neighboring codec unit has been completed and can be used for the signal processing operation (e.g., intra-prediction) of the current codec unit.

[0065] In one or more embodiments, codec tree units (CTUs) can be defined with different sizes for multiple codecs. For example, in an H.264 codec, a CTU may have a size of 16×16; in an HEVC codec, a CTU may have a size of 16×16, 32×32, or 64×64; and in a VVC, AV1, or AVS3 codec, a CTU may have a size of 32×32, 64×64, or 128×128. Alternatively, the size of the CTU can vary in each of various existing codecs or codecs to be developed in the future. Furthermore, the size or shape of the CU can vary depending on each codec. Additionally, multiple codecs may each partition their codec units in a different manner. In this case, the size or shape of the codec unit can be different in each of the multiple codecs.

[0066] As mentioned above, depending on the type of codec, the size or shape of the codec tree unit and the codec unit may vary, and in such cases, it may be difficult to determine the availability of neighboring codec units corresponding to the current codec unit. Furthermore, to determine the availability of neighboring codec units, each of the multiple codecs may require separately defined information or separate hardware components.

[0067] Figure 3 , Figure 4A and Figure 4B This is a diagram used to describe codec tree units and codec units for input images according to one or more embodiments. For ease of description, codec tree units and codec units based on the HEVC codec are described.

[0068] refer to Figures 2 to 4B The multi-codec circuit 1140 can perform encoding or decoding on the input image IMG_IN. In this case, it is assumed that the multi-codec circuit 1140 performs encoding or decoding based on HEVC.

[0069] The input image IMG_IN can include 1024 pixel information in the first direction DR1 and 768 pixel information in the second direction DR2. That is, the input image IMG_IN can have a size of 1024×768.

[0070] The multi-codec circuit 1140 can divide the input image IMG_IN into codec tree units (CTUs) of size 64×64. Specifically, for an input image IMG_IN of size 1024×768, the codec tree units (CTUs) of size 64×64 can be arranged in a 16×12 configuration on the first direction DR1 and the second direction DR2.

[0071] The multicoder circuit 1140 can perform signal processing on a row-by-row basis for codec tree units (CTUs). For example, the multicoder circuit 1140 can sequentially perform signal processing on the codec tree units (CTUs) located in the first row on the first direction DR1, and then sequentially perform signal processing on the codec tree units (CTUs) located in the second row adjacent to the first row on the second direction DR2 on the first direction DR1.

[0072] Each codec tree unit (CTU) can be divided into multiple codec units (CUs). For example, the a-th codec tree unit CTU_a can be divided into codec units 0a to 15a, Cu0a to CU15a (for simplicity, the "CU" in the reference numerals for codec units is omitted in each figure), and the b-th codec tree unit CTU_b can be divided into codec units 0b to 6b, Cu0b to CU6b. The division of codec units can be determined based on pixel information included in the codec tree unit.

[0073] As an example, such as Figure 4A As shown, the a-th codec tree unit CTU_a can be divided into four (4) codec units CU of depth "1" (i.e., divided into four 32×32 regions). Based on pixel information, some regions of depth "1" are divided into four regions of depth "2" (i.e., each region is divided into four 16×16 regions). Based on pixel information, some regions of depth "2" are divided into four regions of depth "3" (i.e., each region is divided into four 8×8 regions). In this case, the 7th and 8th codec units CU7a and CU8a of the a-th codec tree unit CTU_a have a depth of "1" and a size of 32×32, the 0th, 1st, 2nd, 9th, 14th, and 15th codec units CU0a, CU1a, CU2a, CU9a, CU14a, and CU15a have a depth of "2" and a size of 16×16, and the 3rd, 4th, 5th, 6th, 10th, 11th, 12th, and 13th codec units CU3a, CU4a, CU5a, CU6a, CU10a, CU11a, CU12a, and CU13a have a depth of "3" and a size of 8×8.

[0074] As described above, such as Figure 4BAs shown, the b-th codec tree unit CTU_b can be divided into regions of depth "2". For example, the b-th codec tree unit CTU_b is divided into four (4) codec units CU of depth "1" (i.e., divided into four 32×32 regions). Based on pixel information, some regions of depth "1" are divided into four regions of depth "2" (i.e., each region is divided into four 16×16 regions). In this case, the 0b, 1b and 6b codec units CU0b, CU1b and CU6b of the b-th codec tree unit CTU_b can have a depth of "1" and a size of 32×32, and the 2b, 3b, 4b and 5b codec units CU2b, CU3b, CU4b and CU5b can have a depth of "2" and a size of 16×16.

[0075] As mentioned above, in the input image IMG_IN, the codec tree units (CTUs) are divided into the same size, but the codec units can be divided into various sizes according to the pixel information or channel characteristics of each region of the input image IMG_IN.

[0076] The multi-codec circuit 1140 can perform signal processing on multiple codec units (CUs) according to the Z-scan order of each CU in the codec tree unit (CTU). For example, as Figure 3 As shown, when performing signal processing for the a-th codec tree unit CTU_a, the multi-codec circuit 1140 can sequentially perform signal processing on codec units 0a to 15 ...

[0077] Figure 5 This diagram illustrates the codec unit partitioning method used in various codecs. As described above, the multi-codec circuit 1140 can support various codecs. In this case, the size and shape of the codec units (CUs) used in various codecs can vary depending on the codec unit partitioning method.

[0078] For example, such as Figure 5As shown, in the AV1 codec, the codec tree unit (CTU) can be partitioned into a split structure (SPLIT), a vertical split structure (VERT), a vertical 4-partition structure (VERT_4), a vertical A-partition structure (VERT_A), a vertical B-partition structure (VERT_B), a horizontal split structure (HORZ), a horizontal 4-partition structure (HORZ_4), a horizontal A-partition structure (HORZ_A), or a horizontal B-partition structure (HORZ_B). Alternatively, in the VVC codec, the codec tree unit (CTU) can be partitioned into a quadtree structure, a binary tree structure, or a ternary tree structure. Alternatively, in the AVS3 codec, the codec tree unit (CTU) can be partitioned into a quadtree structure (QT), a binary tree structure (BT), or an extended quadtree structure (EQT).

[0079] As described above, in each codec, the size and shape of the codec unit (CU) can be set differently depending on the codec unit partitioning method of the corresponding codec. Alternatively, even in the same codec, the size and shape of the codec unit (CU) can be set differently depending on the codec unit partitioning method.

[0080] In this scenario, determining the availability of adjacent codec units for the current codec unit might require various information, such as the position, size, and shape of the current codec unit, and the positions, sizes, and shapes of adjacent codec units. This information should be set for each of the multiple codecs and each codec unit, potentially increasing the complexity of the multi-codec circuitry 1140. Furthermore, when a new codec is applied, there are questions regarding the implementation of additional information or circuitry for the codec units used in the new codec.

[0081] Figure 6A and 6B This diagram illustrates operations for determining the availability of adjacent codec units according to one or more embodiments. Hereinafter, for ease of description, embodiments of this disclosure are described focusing on signal processing for a single codec tree unit. However, the scope of this disclosure is not limited thereto, and the multi-codec circuit 1140 may sequentially perform signal processing on multiple codec tree units.

[0082] In one or more embodiments, Figure 6A The a-th codec tree unit CTU_a shown illustrates a structure in which codec units are divided based on a first codec (e.g., HEVC), and the b-th codec tree unit CTU_b shows a structure in which codec units are divided based on a second codec (e.g., VVC). In one or more embodiments, the a-th codec tree unit CTU_a and the b-th codec tree unit CTU_b may correspond to the same region in the input image IMG_IN.

[0083] In other words, the size or shape of the codec unit (CU) can change when different codecs are applied to the same region of the input image IMG_IN. In this case, the size and shape of adjacent codec units corresponding to the current codec unit can differ. For example, for the a-th codec tree unit CTU_a, suppose the 9a-th codec unit CU9a is the current codec unit. In this case, adjacent codec units may include CU6a, CU7a, CU8a, CU10a, CU12a, CU14a, and CU15a. Conversely, in the b-th codec tree unit CTU_b, suppose the 15b-th codec unit CU15b, located in a similar position, is the current codec unit. In this case, adjacent codec units may include CU5b, CU6b, CU7b, CU12b, CU14b, CU16b, and CU17b.

[0084] Therefore, even if the input images have the same IMG_IN, depending on the type of the corresponding codec, the size and shape of adjacent codec units can change in various ways when processing signals for codec units at similar or identical locations. In other words, depending on the type of the corresponding codec, various information is needed to determine the availability of adjacent codec units.

[0085] Alternatively, the availability of adjacent codec units can change when the processing order for the codec units changes. For example, as Figure 6B As shown, the d-th codec tree unit CTU_d-1 or CTU_d-2 can be divided into codec units CU0d to CU18d from 0d to 18d.

[0086] In this case, the processing order of codec tree units (CTU_d-1) and (d-2) can be different in some regions. For example, in... Figure 6B In region A, CTU_d-1 can be processed in the order CU3d→CU4d→CU5d→CU6d→CU7d→CU8d→CU9d→CU10d. Conversely, in region A, CTU_d-2 can be processed in the order CU3d→CU7d→CU4d→CU5d→CU6d→CU8d→CU9d→CU10d.

[0087] In one or more embodiments, it is assumed that the 4th codec unit CU4d is the current codec unit. In this case, in the (d-1)th codec tree unit CTU_d-1, the available neighboring codec units of the 4th codec unit CU4d will be CU3d. Conversely, in the (d-2)th codec tree unit CTU_d-2, the available neighboring codec units of the 4th codec unit CU4d will be CU3d and CU7d. That is, even for codec units of the same type, the available neighboring codec units can differ depending on the processing order.

[0088] As described above, the multi-codec circuit 1140 can perform signal processing (e.g., inter-frame prediction or intra-frame prediction such as motion estimation, motion compensation, etc.) on the input image IMG_IN based on codec tree units or codec units. As an example, the multi-codec circuit 1140 can perform the aforementioned signal processing (e.g., inter-frame prediction or intra-frame prediction such as motion estimation, motion compensation, etc.) using the signal processing results of neighboring codec units (i.e., available neighboring codec units) of the currently being signal-processed codec unit (i.e., the current codec unit), where signal processing for neighboring codec units (i.e., available neighboring codec units) is completed. In this case, since the size or shape of the codec unit is defined individually by each of the multiple codecs, the size or shape of the codec units can differ from each other. Furthermore, the processing order for the codec units can be modified in various ways, and in this case, even for the same codec unit, the available neighboring codec units can be different. As described above, separate components or information are required to check the availability of neighboring codec units of the current codec unit. However, this increases the complexity or burden of the multi-codec circuitry 1140 because such separate components or information must be defined individually for each of the multiple codecs. Furthermore, when additional codecs are developed in addition to existing codecs, separate components are required to apply the appropriate size and shape of the codec unit for the additional codecs.

[0089] This disclosure provides a codec unit availability checking block for checking the availability of adjacent codec units across multiple codecs. As described in more detail below, the codec unit availability checking block can check the availability (or signal processing completion) of adjacent codec units of the current codec unit based on a codec unit bitmap. In this case, the availability of adjacent codec units can be checked or determined based on the position of the current codec unit, the size of the current codec unit, and the position of the current codec tree unit, without requiring separate or additional information such as the size, shape, depth, etc., of adjacent codec units. However, such additional information may optionally be used to confirm the availability of adjacent codec units. Furthermore, even with the addition of a new codec, the availability of adjacent codec units can be checked or determined without additional information or additional hardware.

[0090] Figure 7 This illustrates one or more embodiments. Figure 1 Block diagram of a multi-codec circuit. (Reference) Figure 1 and Figure 7 The multi-codec circuitry 1140 may include a motion estimation block 1141a, a motion compensation block 1141b, an intra-frame prediction block 1142, a mode selection block (SEL), a transform and quantization block 1143, an entropy coding block 1144, an entropy decoding block 1145, an inverse quantization and inverse transform block 1146, a deblocking filter 1147, and a memory interface block 1148. (Note: The last sentence appears to be incomplete and possibly refers to a reference block.) Figure 2 The components of the multi-codec circuit 1140 are described, and therefore their additional descriptions will be omitted to avoid redundancy.

[0091] like Figure 7 As shown, the multicoder / decoder circuitry 1140 may further include a codec unit availability check block 1149. The codec unit availability check block 1149 may be configured to check the availability of neighboring codec units of the codec unit currently being processed in the intra-frame prediction block 1142.

[0092] For example, the codec unit availability check block 1149 may include a codec unit bitmap. The codec unit bitmap may include information about the signal processing of codec units included in a codec tree unit. The codec unit bitmap may include multiple bits. Each of the multiple bits may correspond to a unit pixel or the smallest unit of a codec unit used by multiple codecs.

[0093] The codec unit availability check block 1149 can check the availability of adjacent codec units based on the codec unit bitmap. For example, in Figure 6AIn the a-th codec tree unit CTU_a, it is assumed that the 4a codec unit CU4a is the current codec unit. In this case, the codec unit availability check block 1149 can extract the reference bit corresponding to the 4a codec unit CU4a from the codec unit bitmap. The codec unit availability check block 1149 can determine that the 1a codec unit CU1a, the 3a codec unit CU3a, and the 5a codec unit CU5a are available based on the reference bit. In this case, the intra-frame prediction block 1142 can perform intra-frame prediction on the 4a codec unit CU4a using the processing results (i.e., encoding results or decoding results) of the 1a codec unit CU1a, the 3a codec unit CU3a, and the 5a codec unit CU5a.

[0094] Figure 8 It is shown Figure 7 A block diagram of the encoder / decoder unit availability check block. (See reference) Figure 7 and Figure 8 The codec unit availability check block 1149 may include a codec unit bitmap memory 1149a, a reference bit extractor 1149b, and a codec unit availability determiner 1149c.

[0095] The codec unit bitmap memory 1149a can be configured to store a codec unit bitmap BM. In one or more embodiments, the codec unit bitmap memory 1149a may include high-speed registers, but the scope of this disclosure is not limited thereto, and it may be implemented as a memory such as SRAM or DRAM.

[0096] A codec unit bitmap (BM) can correspond to a codec tree unit (CTU). Each bit in the BM can correspond to a unit pixel or the smallest unit of the codec unit (e.g., the smallest CTU size supported by the codec). For example, suppose the multi-codec circuit 1140 supports H.264, HEVC, AV1, VVC, and AVS3 codecs. In this case, the maximum size of the codec tree unit supported by the H.264, HEVC, AV1, VVC, and AVS3 codecs can be 128×128 pixels. Furthermore, the smallest unit size of the codec unit supported by the H.264, HEVC, AV1, VVC, and AVS3 codecs can be 4×4. In this case, the BM can consist of (128 / 4)×(128 / 4)=32×32 bits. In this case, each bit can correspond to the smallest unit of the codec unit.

[0097] When the multi-codec circuit 1140 begins signal processing for the first codec tree unit, the codec unit bitmap BM of the codec unit bitmap memory 1149a can be initialized. For example, all bits of the codec unit bitmap BM of the codec unit bitmap memory 1149a can be initialized by setting them to "0". Subsequently, the multi-codec circuit 1140 can perform signal processing on multiple codec units included in the first codec tree unit. When signal processing for each of the multiple codec units is completed, the corresponding bit in the codec unit bitmap of the codec unit bitmap memory 1149a can be set to "1".

[0098] Reference bit extractor 1149b can extract reference bits RB[6:0] from the codec unit bitmap BM based on the position CU_PS of the current codec unit and the size CU_SIZE of the current codec unit. For example, reference bit extractor 1149b can determine the position of the bit in the codec unit bitmap BM corresponding to the current codec unit based on the position CU_PS of the current codec unit and the size CU_SIZE of the current codec unit. Reference bit extractor 1149b can output some bits adjacent to the bit corresponding to the current codec unit as reference bits RB[6:0].

[0099] The codec unit availability determiner 1149c can be configured to determine the available codec unit CU_AVAIL among adjacent codec units based on reference bits RB[6:0] and the position CTU_PS of the codec tree unit. For example, the codec unit availability determiner 1149c can determine the codec unit corresponding to the bit that is "1" in the reference bits RB[6:0] as the available codec unit CU_AVAIL. Alternatively, when the reference bits RB[6:0] are not present (e.g., when the current codec unit is located at the boundary of the current codec tree unit CTU), the available codec unit can be determined based on the position CTU_PS of the codec tree unit. In one or more embodiments, the signal processing result of the codec unit determined to be the available codec unit can be used for the signal processing of the current codec unit (e.g., used in intra-frame prediction). The operation of the codec unit availability checking block 1149 according to one or more embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings.

[0100] Figure 9 It is shown Figure 7A flowchart of the operation of the multi-codec circuit is provided. In the following, for the purpose of briefly describing one or more embodiments of the present disclosure, the operation of the multi-codec circuit 1140 performing signal processing on a plurality of codec units included in a codec tree unit and checking the availability of adjacent codec units will be primarily described. However, the scope of the present disclosure is not limited thereto, and the multi-codec circuit 1140 may perform encoding or decoding operations corresponding to the plurality of codecs.

[0101] refer to Figures 7 to 9 In operation S1100, the multi-codec circuit 1140 can select the codec tree unit (CTU) to which signal processing is to be performed (hereinafter, for ease of description, it is referred to as the "current codec tree unit CTU"). For example, see reference Figure 3 The multi-codec circuit 1140 can divide the input image IMG_IN into multiple codec tree units. In one or more embodiments, the size of each of the multiple codec tree units can correspond to the size of the currently used codec among the multiple codecs. The multi-codec circuit 1140 can select the current codec tree unit among the multiple codec tree units.

[0102] During operation S1200, the multicoder / decoder circuit 1140 can clear the codec unit bitmap BM. For example, when signal processing is initiated for the current codec tree unit, the codec unit availability check block 1149 of the multicoder / decoder circuit 1140 can clear the codec unit bitmap memory 1149a. That is, all bits of the codec unit bitmap BM in the codec unit bitmap memory 1149a can be set to a reset value (e.g., zero "0").

[0103] In operation S1001, the variable "k" may be set to "0". In one or more embodiments, the variable "k" is intended to describe the repetition of signal processing for multiple codec units and should not be interpreted as having any other technical meaning.

[0104] In operation S1300, the multi-codec circuit 1140 may select the k-th codec unit as the current codec unit. For example, the multi-codec circuit 1140 may select the k-th codec unit as the current codec unit from a plurality of codec units included in the current codec tree unit. In one or more embodiments, the k-th codec unit may be determined according to a predetermined order or a z-scan order.

[0105] In operation S1400, the multi-codec circuit 1140 can extract reference bits from the codec unit bitmap BM based on the position and size of the current codec unit. For example, the reference bit extractor 1149b of the codec unit availability check block 1149 can determine the position of the current bit corresponding to the current codec unit in the codec unit bitmap BM based on the position and size of the current codec unit. The reference bit extractor 1149b can extract at least some of the adjacent bits of the current bit as reference bits.

[0106] In operation S1500, the multi-codec circuit 1140 can determine the available codec units among the adjacent codec units of the current codec unit based on reference bits. For example, the codec unit availability determiner 1149c of the codec unit availability check block 1149 can determine the codec unit corresponding to the reference bit that is set to "1" in the reference bits as the available codec unit.

[0107] In operation S1600, the multicodec circuit 1140 can use available adjacent codec units to perform signal processing (e.g., intra-frame prediction, etc.) on the current codec unit.

[0108] In operation S1700, the multicoder / decoder circuit 1140 can set the bit corresponding to the current codec unit among a plurality of bits included in the codec unit bitmap BM to "1".

[0109] In operation S1800, it can be determined whether the variable "k" is at its maximum value. For example, it can be determined whether signal processing for all codec units included in the current codec tree unit has been completed.

[0110] When the variable “k” is not at its maximum value (i.e., when there are unprocessed codec units among the multiple codec units included in the current codec tree unit), in operation S1002, the variable “k” is incremented by “1”, and the multicodec circuit 1140 can repeatedly perform the operations of S1300 to S1800.

[0111] When the variable "k" is at its maximum value (i.e., when all codec units included in the current codec tree unit are processed), the multicodec circuit 1140 can complete the signal processing for the current codec tree unit and can continue to process the signal for subsequent codec tree units.

[0112] Figure 10A , 10B 11A and 11B show Figure 8 The operation of the encoder / decoder unit availability check block.

[0113] For ease of description, the following text primarily describes signal processing for a codec tree unit (CTU). In this case, a CTU may have a size of 64×64 pixels. Assume the CTU bitmap BM comprises multiple bits, and each of these bits corresponds to the smallest unit of the codec unit (e.g., a size of 4×4 in pixels). In this case, for a 64×64 CTU, the CTU bitmap BM may comprise 16×16 bits.

[0114] Hereinafter, the notation {a, b} or [c, d] will be used to represent the position of a codec unit or bit. {a, b} can be a symbol indicating the coordinates of a pixel within a codec tree unit. That is, {a, b} can indicate the position of a pixel located in row a and column b of a codec tree unit. [c, d] can be a symbol indicating the coordinates of a bit within a codec unit bitmap BM. That is, [c, d] can indicate the position of a bit located in row c and column d of a codec unit bitmap BM.

[0115] For ease of description, Figures 10A to 11B In this context, it is assumed that the multi-codec circuit 1140 performs signal processing on the a-th codec tree unit CTU_a. In this case, the a-th codec tree unit CTU_a may have a size of 64×64 based on a unit pixel, and may be divided into codec units 0a to 15a, CU0a to CU15a, and may be processed sequentially according to the Z-scan order.

[0116] refer to Figure 7 , Figure 8 , Figure 10A and Figure 10B The multi-codec circuit 1140 can perform signal processing for codec unit 0a, codec unit 1a, codec unit 1a, codec unit 2a, and codec unit 3a. Therefore, at this point in time, the multi-codec circuit 1140 can perform signal processing on codec unit 4a. That is, codec unit 4a can be the current codec unit.

[0117] In this case, the codec unit bitmap BM can be as follows: Figure 10B As shown. For example, a codec unit bitmap (BM) can include 16×16 bits. The position of each bit in the codec unit bitmap (BM) can be represented as [c,d] as described above (where "c" is the row position in the codec unit bitmap (BM) and "d" is the column position in the codec unit bitmap (BM)). Each bit in the codec unit bitmap (BM) can correspond to a 4×4 minimum unit codec unit.

[0118] In the a-th codec tree unit CTU_a, the 0a-th codec unit CU0a can have a size of 16×16 pixels and a position of {0,0} pixels (i.e., the position of the first pixel of the 0a-th codec unit CU0a). In this case, in the codec unit bitmap BM, sixteen bits [0,0]-[0,3], [1,0]-[1,3], [2,0]-[2,3], and [3,0]-[3,3] can correspond to the 0a-th codec unit CU0a. As signal processing for the 0a-th codec unit CU0a is completed, the sixteen bits [0,0]-[0,3], [1,0]-[1,3], [2,0]-[2,3], and [3,0]-[3,3] can be set to "1" ( Figure 10B (The gray shadows in the image).

[0119] Subsequently, signal processing is performed sequentially for the first, second, and third codec units CU1a, CU2a, and CU3a, and as the signal processing for the first, second, and third codec units CU1a, CU2a, and CU3a is completed, the corresponding bits in the codec unit bitmap BM can be set to "1".

[0120] Subsequently, the multi-codec circuit 1140 can perform signal processing on the 4a codec unit CU4a. In this case, the multi-codec circuit 1140 can check the completion of encoding or decoding of codec units adjacent to the 4a codec unit CU4a for intra-frame prediction. In this case, reference bits RB0 to RB6 can be extracted from the codec unit bitmap BM based on the position and size of the 4a codec unit CU4a.

[0121] For example, such as Figure 10A and 10B As shown, the position or start position of the 4a codec unit CU4a can be {16, 24} for pixels in the a codec tree unit CTU_a, and [4, 6] for the codec unit bitmap BM. The 4a codec unit CU4a can have a size of 8×8 for pixels, and can correspond to 2×2 bits for the codec unit bitmap BM. In this case, some bits adjacent to the bits corresponding to the 4a codec unit CU4a in the codec unit bitmap BM can be extracted as reference bits RB0 to RB6.

[0122] For ease of description, in the codec unit bitmap BM, the bit corresponding to the current codec unit is called the current codec bit. For example, when codec unit 4a CU4a is the current codec unit, the four bits [4,6], [4,7], [5,6], and [5,7] can be called the current codec bits in the codec unit bitmap BM.

[0123] The reference bit extractor 1149b can extract some of the adjacent bits of the currently encoded / decoded bit as reference bits RB. As an example, multiple reference bits RB0 to RB6 can be extracted based on the following conditions.

[0124] In the bitmap BM of the encoding / decoding unit, the 0th reference bit RB0 can be a bit in the range of [X+S_h, Y-1], the first reference bit RB1 can be a bit in the range of [X+S_h-1, Y-1], the second reference bit RB2 can be a bit in the range of [X, Y-1], the third reference bit RB3 can be a bit in the range of [X-1, Y-1], the fourth reference bit RB4 can be a bit in the range of [X-1, Y], the fifth reference bit RB5 can be a bit in the range of [X-1, Y+S_w-1], and the sixth reference bit RB6 can be a bit in the range of [X-1, Y+S_w].

[0125] In this case, "X" can indicate the row position of the bit corresponding to the start position of the current codec unit, "Y" can indicate the column position of the bit corresponding to the start position of the current codec unit, S_h can indicate the height of the bit in the codec unit bitmap BM corresponding to the size of the current codec unit, and S_w can indicate the width of the bit in the codec unit bitmap BM corresponding to the size of the current codec unit.

[0126] Reference bit RB0 can be used to check the availability of the lower-left adjacent codec unit nCU_BL located to the lower left of the current codec unit. Reference bit RB1 or reference bit RB2 can be used to check the availability of the left-side adjacent codec unit nCU_L located to the left of the current codec unit. Reference bit RB3 can be used to check the availability of the upper-left adjacent codec unit nCU_UL located to the upper left of the current codec unit. Reference bit RB4 or reference bit RB5 can be used to check the availability of the upper-upper adjacent codec unit nCU_U located above the current codec unit. Reference bit RB6 can be used to check the availability of the upper-right adjacent codec unit nCU_UR located to the upper right of the current codec unit. In one or more embodiments, based on the current codec unit, the right side can correspond to the first direction DR1, the left side can correspond to the opposite direction of the first direction DR1, the lower side can correspond to the second direction DR2, and the upper side can correspond to the opposite direction of the second direction DR2.

[0127] exist Figure 10BIn the embodiment, when the 4a codec unit CU4a is the current codec unit, "X" will be "4", "Y" will be "6", S_h will be "2", and S_w will be "2". Therefore, the 0th reference bit RB0 will be bit [6,5], the first reference bit RB1 will be bit [5,5], the second reference bit RB2 will be bit [4,5], the third reference bit RB3 will be bit [3,5], the fourth reference bit RB4 will be bit [3,6], the fifth reference bit RB5 will be bit [3,7], and the sixth reference bit RB6 will be bit [3,8].

[0128] like Figure 10B As shown, the 0th reference bit RB0 and the 6th reference bit RB6 can be "0". In this case, the lower left adjacent codec unit nCU_BL (e.g., Cu5a) corresponding to the 0th reference bit RB0 and the upper right adjacent codec unit nCU_UR (e.g., Cu7a) corresponding to the 6th reference bit RB6 can be determined as unavailable (i.e., not yet processed).

[0129] like Figure 10B As shown, the first reference bit RB1, the second reference bit RB2, the third reference bit RB3, the fourth reference bit RB4, and the fifth reference bit RB5 will be "1". In this case, it can be determined that the left adjacent codec unit nCU_L (e.g., CU3a) corresponding to the first reference bit RB1 and the second reference bit RB2, the upper left adjacent codec unit nCU_UL (e.g., CU1a) corresponding to the third reference bit RB3, and the upper adjacent codec unit nCU_U (e.g., CU1a) corresponding to the fourth reference bit RB4 and the fifth reference bit RB5 are available (i.e., processed).

[0130] When the 4a codec unit CU4a is the current codec unit, the availability of the 1a codec unit CU1a and the 3a codec unit CU3a can be determined based on the above operations. Therefore, the multicodec circuit 1140 can perform signal processing (e.g., intra-frame prediction) on the 4a codec unit CU4a using the processing results for the 1a codec unit CU1a and the 3a codec unit CU3a. In one or more embodiments, in response to the completion of signal processing for the 4a codec unit CU4a, the current codec bits (e.g., [4,6], [4,7], [5,6], and [5,7]) in the codec unit bitmap BM can be set to "1".

[0131] Next, refer to Figure 7 , Figure 8 , Figure 11A and Figure 11BThe multi-codec circuit 1140 can perform signal processing on the 7a codec unit CU7a. That is, the 7a codec unit CU7a can be selected as the current codec unit. In this case, since codec units 0a to 6a codec units CU6a are in a processed state, the corresponding bits in the codec unit bitmap BM are set to "1", such as... Figure 11B As shown.

[0132] The position of the 7a codec unit CU7a can be {0, 32} in pixels, and [0, 8] in the codec unit bitmap BM. The size of the 7a codec unit CU7a can be 32×32 in pixels.

[0133] In this case, as described above, the 0th reference bit RB0 can be bit [8,7], the first reference bit RB1 can be bit [7,7], and the second reference bit RB2 can be bit [0,7]. Meanwhile, the 7a codec unit CU7a can be located at the boundary of the a-th codec tree unit CTU_a. For example, the starting position of the seventh codec unit CU7a in terms of pixels is {0,32}. In this case, the upper surface of the seventh codec unit CU7a, that is, the surface located in the direction opposite to the second direction DR2, is located on the boundary surface of the a-th codec tree unit CTU_a.

[0134] When the upper surface of the seventh codec unit CU7a is located on the boundary surface of the a-th codec tree unit CTU_a, the third reference bits RB3 to the sixth reference bits RB6 located outside the a-th codec tree unit CTU_a may not be extracted from the codec unit bitmap BM. In this case, the availability of adjacent codec units (e.g., nCU_UL, nCU_U, nCU_UR) located on the upper surface (i.e., the boundary surface) of the seventh codec unit CU7a can be checked or determined based on the position of the a-th codec tree unit CTU_a, which is the current codec tree unit.

[0135] For example, the upper adjacent codec unit adjacent to the upper surface of codec unit CU7a (7a) can be a codec unit included in another codec tree unit CTU_x. As an example, see reference... Figure 3The codec tree units located on the upper surface (the surface opposite to the second direction DR2) of the a-th codec tree unit CTU_a, which is the current codec tree unit, can be in a processed state. That is, all codec units adjacent to the upper surface of the 7a codec unit CU7a can be in a processed state (i.e., an available state). Therefore, based on the position of the a-th codec tree unit CTU_a, which is the current codec tree unit, it can be determined that all adjacent codec units (e.g., nCU_UL, nCU_U, and nCU_UR) located on the upper surface (i.e., the boundary surface) of the 7a codec unit CU7a are available.

[0136] Alternatively, when the a-th codec tree unit CTU_a is located in the 0th row (i.e., the first row) of the input image IMG_IN, there is no codec tree unit or codec unit adjacent to the upper surface of the a-th codec tree unit CTU_a. In this case, it can be determined that the adjacent codec units (e.g., nCU_UL, nCU_U, and nCU_UR) located at the upper surface (i.e., the boundary surface) of the 7a codec unit CU7a are absent or unavailable. Specifically, signal processing for the 7a codec unit CU7a can be performed without using the adjacent codec units (e.g., nCU_UL, nCU_U, and nCU_UR) located at the upper surface (i.e., the boundary surface) of the 7a codec unit CU7a.

[0137] As a result, Figure 11B As shown, when the 7th codec unit CU7a, located at the boundary of the a-th codec tree unit CTU_a, is the current codec unit, based on the first reference bit RB1 and the second reference bit RB2, the left-adjacent codec units (e.g., the 1st codec unit CU1a, the 4th codec unit CU4a, and the 6th codec unit CU6a) can be determined to be available (i.e., processed). Additionally, based on the position of the a-th codec tree unit CTU_a, all the upper adjacent codec units of the 7th codec unit CU7a (e.g., the codec units of CTU_x) can be determined to be available (i.e., processed).

[0138] The multi-codec circuit 1140 can perform signal processing (e.g., intra-frame prediction) on the 7a codec unit using available adjacent codec units. In one or more embodiments, in response to the completion of signal processing for the 7a codec unit CU_7a, the corresponding current codec bit (e.g., 64 bits from [0,8] to [8,15]) in the codec unit bitmap BM can be set to "1".

[0139] Figure 12A and Figure 12B It shows Figure 8 The operation of the encoder / decoder unit availability check block. Figures 10A to 11B In this embodiment, the multi-codec circuit 1140 performs partitioning of the codec units based on the HEVC codec. That is, in Figures 10A to 11B In some embodiments, the codec unit has a square shape. However, the scope of this disclosure is not limited thereto, and the multi-codec circuit 1140 can support various codecs, and the codec unit can have various sizes or shapes depending on the target codec. In this case, the codec unit availability check block 1149 of the multi-codec circuit 1140 can check the availability of adjacent codec units in the same manner or without additional hardware.

[0140] refer to Figure 7 , Figure 8 , Figure 12A and Figure 12B The multi-codec circuit 1140 can perform signal processing on the c-th codec tree unit CTU_c. As an example, see reference... Figure 6A The c-th codec tree unit CTU_c can be a codec tree unit in which codec unit partitioning based on the VVC codec has been performed. That is, some of the codec units CU0c to CU18c included in the c-th codec tree unit CTU_c can be squares, while the remaining codec units may not be squares. In this case, it can be similar to the reference... Figures 10A to 11B The described method checks the availability of adjacent codec units.

[0141] For example, suppose the 6cth codec unit CU6c is the current codec unit. In this case, signal processing is completed for codec units CU0c to CU5c from 0c to 5c. Therefore, as... Figure 12B As shown, the bits in the bitmap BM corresponding to the codec units 0c to 5c (CU0c to CU5c) (e.g., 64 bits from [0,0] to [7,7]) are set to "1".

[0142] like Figure 12B As shown, the current codec unit is the 6th codec unit CU6c, and the position of the 6th codec unit CU6c in terms of pixels can be {0,32}, and in terms of the codec unit bitmap BM, it can be [0,8]. The size of the 6th codec unit CU6c in terms of pixels can be 8×32, and in terms of the codec unit bitmap BM, it can be 2×8.

[0143] In this scenario, as described above, reference bit RB0 can be bit [8,7], reference bit RB1 can be bit [7,7], and reference bit RB2 can be bit [0,7]. Since the [8,7] bits of reference bit RB0 are "0", the lower left adjacent codec unit CU12c is determined to be unavailable (i.e., unprocessed). Since the [7,7] bits of reference bit RB1 and the [0,7] bits of reference bit RB2 are both "1", the left adjacent codec units CU0c, CU3c, and CU5c are determined to be available.

[0144] In one or more embodiments, the upper surface of the 6c codec unit CU6c may be adjacent to the boundary surface of the cth codec tree unit CTU_c. In this case, as described above with reference to Figures 11a and 11b, the availability of neighboring codec units on the upper surface of the 6c codec unit CU6c can be checked or determined based on the position of the cth codec tree unit CTU_c. For example, when signal processing for a codec tree unit CTU_x located on the upper surface of the cth codec tree unit CTU_c is completed, neighboring codec units on the upper surface of the 6c codec unit CU6c can be determined to be available.

[0145] Figure 13A , 13B 14A and 14B show Figure 8 The operation of the encoder / decoder unit availability check block. For ease of description, additional descriptions associated with the above components or operations will be omitted to avoid redundancy.

[0146] refer to Figure 7 , Figure 8 , Figure 13A and Figure 13B The multicoder circuit 1140 can perform signal processing on the (d-1)th codec tree unit CTU_d-1. For example, the (d-1)th codec tree unit CTU_d-1 can be divided into codec units CU0d to CU18d from 0d to 18d.

[0147] In one or more embodiments, it is assumed that the 4th codec unit CU4d is the current codec unit. In this case, as Figure 13A As shown, signal processing for the encoding / decoding unit can be performed in the order of CU0d→CU1d→CU2d→CU3d→CU4d.

[0148] In this case, such as Figure 13BAs shown, in the codec unit bitmap BM, the bits corresponding to codec units CU0d, CU1d, CU2d, and CU3d of the 0d, 1d, 2d, and 3d encoding / decoding units can be set to "1". The position of the 4d codec unit CU4d can be {0, 48} in terms of pixels, and [0, 12] in terms of the codec unit bitmap BM. The size of the 4d codec unit CU4d can be 8×16 in terms of pixels, and 2×4 in terms of the codec unit bitmap BM.

[0149] In this case, as described above, the 0th reference bit RB0 can be bits [4, 11], the first reference bit RB1 can be bits [3, 11], and the second reference bit RB2 can be bits [0, 11]. Therefore, since the bits [4, 11] of the 0th reference bit RB0 are "0", the lower-left adjacent codec unit (e.g., CU7d) is determined to be unavailable. Since the bits [3, 11] of the first reference bit RB1 and the bits [0, 11] of the second reference bit RB2 are "1", the left-side adjacent codec unit (e.g., CU3d) is determined to be available. In one or more embodiments, since the 4th codec unit CU4d is adjacent to the boundary of the (d-1)th codec tree unit CTU_d-1, the third reference bits RB3 to the sixth reference bits RB6 are not extracted, and the availability of adjacent codec units on the upper surface of the 4th codec unit CU4d can be determined based on the position of the (d-1)th codec tree unit CTU_d-1. This is similar to the reference... Figure 11A and Figure 11B The content described is omitted here, and therefore its additional description is omitted to avoid redundancy.

[0150] Next, refer to Figure 7 , Figure 8 , Figure 14A and Figure 14B The multi-codec circuit 1140 can perform signal processing on the (d-2)th codec tree unit CTU_d-2. For example, the (d-2)th codec tree unit CTU_d-2 can be divided into codec units CU0d to CU18d from 0d to 18d.

[0151] In one or more embodiments, the (d-1) and (d-2) codec tree units CTU_d-1 and CTU_d-2 can be partitioned using the same codec unit partitioning method. That is, the codec units included in the (d-1) and (d-2) codec tree units CTU_d-1 and CTU_d-2 can have the same size or the same shape. Conversely, the processing order for the codec units in the (d-1) and (d-2) codec tree units CTU_d-1 and CTU_d-2 can be different. For example, as... Figure 14AAs shown, for the (d-2)th codec tree unit CTU_d-2, signal processing for the codec unit can be performed in the order of CU0d→CU1d→CU2d→CU3d→CU7d→CU4d.

[0152] In this case, assuming that the 4th codec unit CU4d is the current codec unit, as described above. In this case, since signal processing has been completed for the 0th, 1st, 2nd, 3rd, and 7th codec units CU0d, CU1d, CU2d, CU3d, and CU7d, the bits in the codec unit bitmap BM corresponding to the 0th, 1st, 2nd, 3rd, and 7th codec units CU0d, CU1d, CU2d, CU3d, and CU7d are set to "1".

[0153] The position of the 4d codec unit CU4d can be {0, 48} in terms of pixels, and [0, 12] in terms of the codec unit bitmap BM. The size of the 4d codec unit CU4d can be 8×16 in terms of pixels, and 2×4 in terms of the codec unit bitmap BM. In this case, as described above, the 0th reference bit RB0 can be bit [4, 11], the first reference bit RB1 can be bit [3, 11], and the second reference bit RB2 can be bit [0, 11].

[0154] In this case, since bits [4,11] of the 0th reference bit RB0 are "1", the adjacent codec unit to the lower left (e.g., CU7d) is determined to be available, and since bits [3,11] of the first reference bit RB1 and bits [0,11] of the second reference bit RB2 are "1", the adjacent codec unit to the left (e.g., CU3d) is determined to be available. In one or more embodiments, since the 4th codec unit CU4d is adjacent to the boundary of the (d-2)th codec tree unit CTU_d-2, the third reference bits RB3 to the sixth reference bits RB6 are not extracted, and the availability of adjacent codec units on the upper surface of the 4th codec unit CU4d can be determined based on the position of the (d-2)th codec tree unit CTU_d-2. This is similar to the reference Figure 11A and Figure 11B The content described is omitted here, and therefore its additional description is omitted to avoid redundancy.

[0155] For reference Figures 13A to 14B Even if the processing order of the codec units changes, the multi-codec circuit 1140 according to the embodiments of this disclosure can easily check the availability of adjacent codec units without additional information or additional hardware.

[0156] Figure 15A and 15B It shows Figure 8 The operation of the encoder / decoder unit availability check block. For ease of description, additional descriptions associated with the above components or operations will be omitted to avoid redundancy.

[0157] refer to Figure 7 , Figure 8 , Figure 15A and Figure 15B The multi-codec circuit 1140 can perform signal processing on the c-th codec tree unit CTU_c. The c-th codec tree unit CTU_c can be divided into codec units 0c to 18c, CU0c to CU18c. The multi-codec circuit 1140 can sequentially perform signal processing on codec units 0c to 18c, CU0c to CU18c.

[0158] In this case, assume that codec unit 15c (CU15c) is the current codec unit. In this case, as... Figure 15B As shown, in the bitmap BM, the bits corresponding to codec units 0c to 14c, CU0c-CU14c, can be set to "1".

[0159] The position of the 15c codec unit CU15c can be {32, 32} in pixels and [8, 8] in the codec unit bitmap BM. The size of the 15c codec unit CU15c can be 8×32 in pixels and 2×8 in the codec unit bitmap BM.

[0160] In one or more embodiments, the reference bit extractor 1149b of the codec unit availability check block 1149 can extract the surrounding bits of the bit corresponding to the 15c codec unit CU15c in the codec unit bitmap BM as reference bits RM. For example, in the codec unit bitmap BM, the bit corresponding to the 15c codec unit CU15c can be [8,8] to [8,15] and [9,8] to [9,15]. In this case, the left-hand bits (e.g., [7,8] to [7,15]) of the bit corresponding to the 15c codec unit CU15c can be extracted as left-hand reference bits RB_L. The upper-left bits (e.g., [7,7]) of the bit corresponding to the 15c codec unit CU15c can be extracted as upper-left reference bits RB_UL. The upper bits (e.g., [8,7] and [9,7]) of the bit corresponding to the 15c codec unit CU15c can be extracted as upper-right reference bits RB_U. The upper right bit (e.g., [7,9]) corresponding to the bit of the 15c codec unit CU15c can be extracted as the upper right reference bit RB_UR.

[0161] Based on the left reference bit RB_L, the availability of the left adjacent codec unit (e.g., CU12c and CU14c) can be checked; based on the upper left reference bit RB_UL, the availability of the upper left adjacent codec unit (e.g., CU5c) can be checked; based on the upper reference bit RB_U, the availability of the upper adjacent codec unit (e.g., CU6c) can be checked; and based on the upper right reference bit RB_UR, the availability of the upper right adjacent codec unit (e.g., CU7c) can be checked. In one or more embodiments, since the lower side of the 15c codec unit CU15c is adjacent to the boundary of the c-th codec tree unit CTU_c, the availability of the adjacent codec unit below the 15c codec unit CU15c can be checked based on the position of the c-th codec tree unit CTU_c.

[0162] Figure 16A and 16B It shows Figure 8 The operation of the encoder / decoder unit availability check block. For ease of description, additional descriptions associated with the above components or operations will be omitted to avoid redundancy.

[0163] refer to Figure 7 , Figure 8 , Figure 16A and Figure 16B The multi-codec circuit 1140 can perform signal processing on the c-th codec tree unit CTU_c. The c-th codec tree unit CTU_c can be divided into codec units 0c to 18c, CU0c to CU18c. In one or more embodiments, the multi-codec circuit 1140 can perform signal processing on codec units 0c to 18c, CU0c-CU18c, non-sequentially.

[0164] For example, suppose codec unit 16c (CU16c) is the current codec unit. In this case, codec units 0c to 15c and 18c (CU0c to CU15c and CU18c) can be in a state where signal processing is completed. Therefore, in the codec unit bitmap BM, the bits corresponding to codec units 0c to 15c and 18c (CU0c to CU15c and CU18c) can be set to "1".

[0165] In this case, the reference bit extractor 1149b of the codec unit availability check block 1149 can extract all the surrounding bits of the bit corresponding to the 16c codec unit CU16c as reference bits RM from the codec unit bitmap BM. For example, the position of the 16c codec unit CU16c can be {32, 40} in terms of pixels, and [8, 10] in terms of the codec unit bitmap BM. The size of the 16c codec unit CU16c can be 16×32 in terms of pixels, and 4×8 in terms of the codec unit bitmap BM.

[0166] In the codec unit bitmap BM, the current codec bit and adjacent bits corresponding to the 16c codec unit CU16c can be extracted as reference bits RB. For example, based on the current codec bit, the upper left bit can be extracted as the upper left reference bit RB_UL, the upper bit can be extracted as the upper reference bit RB_U, the upper right bit can be extracted as the upper right reference bit RB_UR, the right bit can be extracted as the right reference bit RB_R, the lower right bit can be extracted as the lower right reference bit RB_BR, the lower bit can be extracted as the lower reference bit RB_B, the lower left bit can be extracted as the lower left reference bit RB_BL, and the left bit can be extracted as the left reference bit RB_L.

[0167] As an example, in the codec unit bitmap BM, the current codec bit corresponding to the 16c codec unit CU16c can be 16 bits of [8 to 11, 10 to 13] (that is, it can be represented as [8:11, 10:13], or alternatively, as [8,10] to [11,10], [8,11] to [11,11], [8,12] to [11,12] and [8,13] to [11,13]). In this case, the upper left reference bit RB_UL can be bit [7,9] in the codec unit bitmap BM, the upper reference bit RB_U can be bit [7,10 to 13] in the codec unit bitmap BM, the upper right reference bit RB_UR can be bit [7,14] in the codec unit bitmap BM, the right reference bit RB_R can be bit [8 to 11,14] in the codec unit bitmap BM, the lower right reference bit RB_BR can be bit [12,14] in the codec unit bitmap BM, the lower reference bit RB_B can be bit [12,10 to 13] in the codec unit bitmap BM, the lower left reference bit RB_BL can be bit [12,9] in the codec unit bitmap BM, and the left reference bit RB_L can be bit [8 to 11,9] in the codec unit bitmap BM.

[0168] like Figure 16BAs shown, since the remaining reference bits RB_UL, RB_U, RB_UR, RB_R, RB_BR, RB_BL, and RB_L (excluding the lower reference bit RB_B) are all "1", the adjacent codec units (e.g., CU6c, CU7c, CU11c, CU18c, and CU15c) corresponding to the remaining reference bits RB_UL, RB_U, RB_UR, RB_R, RB_BR, RB_BL, and RB_L (excluding the lower reference bit RB_B) can be determined to be available. Therefore, the multicodec circuit 1140 can use the available adjacent codec units (e.g., CU6c, CU7c, CU11c, CU18c, and CU15c) to perform signal processing (e.g., intra-frame prediction) on the 16c codec unit CU16c.

[0169] As described above, according to one or more embodiments of this disclosure, the multi-codec circuit 1140 can be configured to check the availability of adjacent codec units using the same method, even if the size, shape, or processing order of the codec units supported by various codecs differs, or if the codec unit partitioning method differs. Therefore, since no separate hardware configuration or additional information is required to check the availability of codec units for each of the multiple codecs, the complexity of the multi-codec circuit 1140 can be reduced. Furthermore, even if additional codecs besides existing codecs are developed, the availability of adjacent codec units can be checked without additional information or hardware, thus improving the scalability of the multi-codec circuit 1140.

[0170] In the above embodiments, each bit of the codec unit bitmap BM corresponds to the smallest unit of the codec unit. However, the scope of this disclosure is not limited thereto. For example, each bit of the codec unit bitmap BM may correspond to a single pixel. In this case, the multiple bits of the codec unit bitmap BM may have a size of M×N, and the size of M×N may correspond to the maximum size of the pixel basis of the codec tree unit.

[0171] Figure 17 It is used to describe the use of by Figure 1 The multi-codec circuit processes the input image's codec tree unit and a diagram of the codec unit. (See diagram for example.) Figure 17 As illustrated, the input image IMG_IN can have a size of 1024×768. In this case, the input image IMG_IN can be divided into multiple codec tree units (CTUs). As an example, each of the multiple codec tree units (CTUs) can have a size of 64×64. In this case, for the input image IMG_IN, the multiple codec tree units (CTUs) can be arranged in a 16×12 configuration.

[0172] As described above, when the current codec unit is located at the boundary of a codec tree unit, the availability of adjacent codec units corresponding to the boundary can be checked based on the position of the codec tree unit. In this case, when the codec tree unit is located at the boundary of the input image IMG_IN, it can be determined that the adjacent codec units corresponding to the boundary do not exist or are unavailable.

[0173] In one or more embodiments, the input image IMG_IN can be divided into multiple tiles. For example, the input image IMG_IN can be divided into tile 0 (TILE0) and tile 1 (TILE1). Tile 0 (TILE0) may include the upper region of the input image IMG_IN, and tile 1 (TILE1) may include the lower region of the input image IMG_IN. As an example, in multiple codec tree units (CTUs) of 16×12 form, the codec tree units located in rows 0 to 7 can be included in tile 0 (TILE0), and the codec tree units located in rows 8 to 15 can be included in tile 1 (TILE1).

[0174] In one or more embodiments, the multi-codec circuitry 1140 can perform signal processing on the 0th and first blocks TILE0 and TILE1 in parallel or independently. In this case, the codec tree unit located in row 0 and the codec tree unit located in row 8 can be processed in parallel. In this case, when checking the availability of adjacent codec units, the boundaries of the 0th and first blocks TILE0 and TILE1 can be considered to be the same as the boundaries of the input image IMG_IN.

[0175] For example, the a1 codec tree unit CTU_a1 is not located on the boundary of the input image IMG_IN. However, as mentioned above, due to the parallel processing of the 0th and first patches TILE0 and TILE1, when processing the signal for the a1 codec tree unit CTU_a1, the codec tree units located on the upper surface of the a1 codec tree unit CTU_a1 may not have been processed yet. Therefore, the codec tree units or codec units located on the upper surface of the a1 codec tree unit CTU_a1 may be determined to be absent or unavailable.

[0176] In one or more embodiments, the codec unit availability check block 1149 of the multi-codec circuit 1140 may further include an additional codec unit bitmap for codec tree units. The additional codec unit bitmap may include multiple bits indicating whether multiple codec tree units included in the input image IMG_IN have been processed. The codec unit availability check block 1149 may determine the availability of codec units included in codec tree units adjacent to the current codec tree unit based on the additional codec unit bitmap. In one or more embodiments, the additional codec unit bitmap may include multiple bits, and each of the multiple bits may correspond to the smallest unit of the codec tree unit supported by each of the multiple codecs.

[0177] Figure 18 This is a block diagram schematically illustrating a multi-codec circuit according to one or more embodiments of the present disclosure. Reference Figure 18 This document briefly describes the decoding operation of a multi-codec circuit 2000 according to one or more embodiments of the present disclosure. However, the scope of this disclosure is not limited thereto, and the multi-codec circuit 2000 may also include additional hardware for performing encoding or decoding corresponding to each of the plurality of codecs.

[0178] refer to Figure 18 The multicoder / decoder circuit 2000 may include a preprocessing block 2100, a frame buffer 2200, a codec unit processing block 2300, a codec unit availability check block 2400, and a filter 2500.

[0179] Preprocessing block 2100 can perform preprocessing operations on the bitstream BST. For example, the bitstream BST can be compressed data obtained by encoding the input image IMG_IN. Preprocessing block 2100 can perform entropy decoding on the bitstream BST to generate a decoded image. In one or more embodiments, preprocessing block 2100 can store the decoded image in frame buffer 2200.

[0180] The codec unit processing block 2300 can perform various signal processing on the image stored in the frame buffer 2200 based on the codec unit. In one or more embodiments, the codec unit processing block 2300 can be configured to perform intra-frame prediction or inter-frame prediction on a codec unit basis.

[0181] The codec unit availability check block 2400 can be configured to check the availability of adjacent codec units with respect to the codec unit being processed by the codec unit processing block 2300. In one or more embodiments, the codec unit availability check block 2400 may be a reference... Figures 7 to 17 The described encoding / decoding unit availability check block 1149 or can be based on reference Figures 7 to 17The described operation method checks the availability of adjacent codec units. Based on the check result of the codec unit availability check block 2400 (i.e., information about available adjacent codec units), the codec unit processing block 2300 can perform various signal processing on the codec units.

[0182] Filter 2500 may include a deblocking filter and a sample adaptive offset (SAO), which are configured to reduce discontinuities at block boundaries and attenuate ring artifacts and sample distortion. The filtered information may be stored in frame buffer 2200 and may be used as a reference image.

[0183] As described above, according to one or more embodiments of this disclosure, a multi-codec circuit can be configured to support multiple codecs. The multi-codec circuit can be configured to check the availability of adjacent codec units using the same method, even if the sizes, shapes, or processing orders of the codec units supported by various codecs differ, or if the codec unit partitioning methods differ. Therefore, since no separate hardware configuration or additional information is required to check the availability of codec units for each of the multiple codecs, the complexity of the multi-codec circuit 1140 can be reduced. Furthermore, even if additional codecs besides existing codecs are developed, the availability of adjacent codec units can be checked without additional information or hardware, thus improving the scalability of the multi-codec circuit.

[0184] According to one or more embodiments of this disclosure, a multi-codec circuit can perform signal processing (e.g., intra-frame prediction, inter-frame prediction, etc.) on a unit-by-unit basis. In this case, the multi-codec circuit can use a bitmap of the unit to check the availability of adjacent units. Even if the size, shape, or processing order of the units changes for each of the various codecs, the availability of adjacent units can be checked based on the same configuration or the same method. Furthermore, even if a new codec is developed in addition to existing codecs, the availability of adjacent units can be checked without additional information or hardware. Therefore, the complexity of the multi-codec circuit can be reduced. Thus, a multi-codec circuit with reduced cost and improved performance, and its operation method, are provided.

[0185] The above description describes detailed embodiments for carrying out this disclosure. Embodiments that are simply modified or easily modified in design may be included in this disclosure and one or more of the above embodiments. Furthermore, techniques that are easily modified and implemented using the above embodiments may be included in this disclosure. Therefore, the scope of this disclosure should not be limited to the above embodiments and should be defined not only by the claims described later but also by claims equivalent to those of this disclosure.

Claims

1. A method of operating a multicodec circuit configured to support multiple codecs, the method comprising: Select the current CTU from multiple codec tree units (CTUs) partitioned from the input image; Select the current CU from the multiple codec units (CUs) divided from the current CTU; The availability of adjacent CUs of the current CU is determined based on the bitmap of the encoding / decoding unit; Perform signal processing on the current CU using at least one of the neighboring CUs that are determined to be available; as well as Based on the completion of signal processing for the current CU, update the encoding / decoding bit in the encoding / decoding unit bitmap corresponding to the current CU; The encoding / decoding unit bitmap includes multiple bits, each bit indicating whether signal processing has been performed on the corresponding CU among the multiple CUs. Each of the plurality of bits corresponds to the minimum CU supported by each of the plurality of codecs.

2. The method according to claim 1, wherein, The plurality of bits in the bitmap of the encoding / decoding unit has a size of N×M, where each of N and M is a natural number; and The size of N×M corresponds to the maximum CTU size supported by each of the plurality of codecs.

3. The method according to claim 1, in, At least two bits in the bitmap of the encoding / decoding unit are allocated to represent the encoding / decoding bit corresponding to the current CU.

4. The method according to claim 1, wherein, Determining the availability of the adjacent CUs includes: Based on the current CU's position and size, extract multiple reference bits from the encoding / decoding unit bitmap; and The availability of the adjacent CUs is determined based on the plurality of reference bits.

5. The method according to claim 4, wherein, In the codec unit bitmap, the plurality of reference bits include at least 7 bits adjacent to the current codec bit corresponding to the current CU.

6. The method according to claim 4, wherein, Determining the availability of the adjacent CUs based on the plurality of reference bits includes: The adjacent CUs corresponding to the reference bits indicating the completion of signal processing among the plurality of reference bits are determined as available adjacent CUs.

7. The method according to claim 4, wherein, Determining the availability of the adjacent CUs based on the codec unit bitmap further includes: Based on the fact that the current CU and the current CTU are adjacent to each other at the boundary, the adjacent CU at the boundary is determined to be available based on the position of the current CTU.

8. The method according to claim 1, further comprising: Based on the current CTU being selected, the plurality of bits in the codec unit bitmap are cleared to reset values.

9. The method according to claim 1, wherein, The signal processing includes inter-frame prediction or intra-frame prediction.

10. The method according to claim 1, wherein, The multiple codecs include at least two of the following: High-level video codec H.264, High-efficiency video codec HEVC, AOMedia Video 1 AV1, Universal Video Codec VVC, or Audio Video Codec Standard AVS3.

11. A method of operating a multicodec circuit configured to support multiple codecs, the method comprising: The availability of the first adjacent CU of the first CU among the multiple codec units (CUs) divided from the first codec tree unit (CTU) is determined based on the codec unit bitmap; Based on the fact that the first adjacent CU is determined to be available for the first signal processing, the first signal processing is performed on the first CU using the first adjacent CU; Based on the completion of the first signal processing, the first current encoding / decoding bit in the bitmap of the encoding / decoding unit corresponding to the first CU is set to a first value; The availability of the second adjacent CU of the second CU among multiple CUs is determined based on the bitmap of the encoding and decoding unit; Based on the fact that the second adjacent CU is determined to be available for the second signal processing, the second signal processing is performed on the second CU using the second adjacent CU; and Based on the completion of the second signal processing, the second current encoding / decoding bit in the bitmap of the encoding / decoding unit corresponding to the second CU is set to the first value. The codec unit bitmap includes multiple bits indicating whether signal processing has been performed on the plurality of CUs, and Each of the plurality of bits corresponds to the minimum CU supported by each of the plurality of codecs.

12. The method of claim 11, wherein determining the availability of the first adjacent CU comprises: Extract the bit adjacent to the first current codec bit corresponding to the first CU from the bitmap of the codec unit as the first reference bit; and The availability of the first adjacent CU is determined based on the first reference bit.

13. The method of claim 12, wherein determining the availability of the second adjacent CU comprises: Extract the bit adjacent to the second current codec bit corresponding to the second CU from the bitmap of the codec unit as the second reference bit; and The availability of the second adjacent CU is determined based on the second reference bit, and The second reference bit includes at least one of the first current codec bits that is set to the first value.

14. The method according to claim 13, wherein, Each of the first reference bit and the second reference bit includes at least 7 bits of the plurality of bits in the encoding / decoding unit bitmap.

15. The method according to claim 11, wherein, The plurality of bits in the bitmap of the encoding / decoding unit has a size of N×M, where each of N and M is a natural number; and The size of N×M corresponds to the maximum CTU size supported by each of the plurality of codecs.

16. The method of claim 11, further comprising: The availability of the third adjacent CU, which is adjacent to the third CU among the multiple CUs divided from the second CTU, is determined based on the bitmap of the encoding / decoding unit. Based on the fact that the third adjacent CU is determined to be available for the third signal processing, the third signal processing is performed on the third CU using the available third adjacent CU; and Set the third current encoding / decoding bit in the bitmap corresponding to the third CU to the first value.

17. The method according to claim 16, wherein, A first CTU is delineated from the first input image based on the first codec among the plurality of codecs; and The second CTU is partitioned from the second input image based on a second codec that is different from the first codec among the plurality of codecs.

18. The method according to claim 16, wherein, The size of the first CTU is different from the size of the second CTU.

19. The method of claim 16, wherein, The first CU, the second CU, and the third CU are different in size, shape, and position from one another.

20. A method of operating a multicodec circuit configured to support multiple codecs, the method comprising: Clear the codec unit bitmap to its reset value; Based on the bitmap of the codec unit, the availability of the first adjacent CU of the first codec unit CU in the first codec tree unit CTU is determined; Based on the fact that the first adjacent CU is determined to be available for the first signal processing, the first signal processing is performed on the first CU using the first adjacent CU; Set the first current encoding / decoding bit in the bitmap of the encoding / decoding unit corresponding to the first CU to a first value; Clear the bitmap of the encoding / decoding unit to the reset value; The availability of the second adjacent CU in the second CU of the second CTU is determined based on the bitmap of the encoding / decoding unit; Based on the fact that the second adjacent CU is determined to be available for the second signal processing, the second signal processing is performed on the second CU using the second adjacent CU; and Set the second current encoding / decoding bit in the bitmap of the encoding / decoding unit corresponding to the second CU to the first value; The size of the first CTU is different from the size of the second CTU. The codec unit bitmap includes multiple bits indicating whether signal processing has been performed on a plurality of CUs, including the first CU, the second CU, the first adjacent CU, and the second adjacent CU. Each of the plurality of bits corresponds to the minimum CU supported by each of the plurality of codecs.

Images