Method for encoding and decoding image on basis of intra prediction for color block

Abstract

In order to solve above-described technical problems, an image decoding method according to an embodiment of the present invention may comprise the steps of: acquiring intra prediction mode information of a current color block from a bit string; identifying illuminance blocks at multiple positions corresponding to the current color block, wherein the multiple positions include at least one of a center position, an upper left position, an upper right position, a lower left position, and a lower right position of the current color block; acquiring an intra prediction mode from at least one of the identified illuminance blocks and including same in an intra prediction mode candidate list for the current color block; acquiring an intra prediction mode from at least one color block spatially adjacent to the current color block and including same in the candidate list; determining an intra prediction mode of the current color block on the basis of the configured candidate list and the acquired intra prediction mode information; and generating a prediction block for the current color block by using the determined intra prediction mode.

Description

Image encoding and decoding method based on intra-frame prediction for color blocks

[0001] The present invention relates to the field of encoding and decoding of digital video, and to a method for encoding and decoding digital video, a method for recording such data, and components, devices, and systems for realizing such a method.

[0002] The present invention may correspond to a technical field identical to at least one of the digital video compression technology standards known by standard names such as MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG, a technical field for improving the inherent efficiency of the standard, or a technical field for improving or replacing the standard.

[0003] Digital video encoding and decoding are widely utilized in various digital video applications. For example, devices such as video recording equipment and camcorders used for video recording activities—including digital television broadcasting, video transmission via communication networks, video calls, video conversations, and video chats, recording and provision of video content using optical media such as VCDs (video compact discs), DVDs (digital versatile discs), and Blu-rays, all procedures for the production, editing, collection, and distribution of video content, and video recording for personal, commercial, industrial, and security purposes—are all dependent on video encoding and decoding technology.

[0004] Accordingly, embodiments that can be referred to as digital video encoders and decoders may constitute a part of a wide range of devices related to the creation, recording, and provision of digital video, including digital television, digital broadcasting systems, wireless broadcasting systems, computers in the form of notebooks / desktops / tablets, e-book readers, digital cameras, digital recording devices, digital multimedia playback devices, video game devices / terminals / consoles, mobile phones equipped with multimedia playback functions (including smartphones), equipment for video conferencing, and other devices.

[0005] Digital video encoders and decoders as described above can be implemented by digital video compression standards that are understood by and widely used by people skilled in the art. The digital video compression standards may include at least one of the compression standards known by standard names such as MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

[0006] Video encoders and decoders can be implemented to encode or decode digital video information more efficiently while complying with the above specifications, or by improving or modifying them. Attempts to modify the above specifications may also lead to the development of new specifications. Among well-known examples is the so-called enhanced compression model (ECM), which is an attempt to improve and replace the conventional H.266 / VVC specifications, currently being developed by the Joint Video Experts Team (JVET), a joint international standardization group of ISO, IEC, and ITU-T.

[0007] It is known that digital video requires a significant amount of information to depict its content in an uncompressed state. Therefore, recording or transmitting this information in its original form can be inefficient. Consequently, digital video is compressed using various methods prior to recording or transmission. These compression methods include lossy encoding and lossless encoding. Lossy encoding achieves high compression performance by sacrificing some image quality, while lossless encoding sacrifices some compression performance to prevent image degradation. Regardless of the encoding method, there is a need to implement technologies that achieve high compression ratios while minimizing the sacrifice of image quality to meet the demand for using high-quality digital video within the limits of limited memory recording capacity and communication transmission bandwidth.

[0008] As described above, the encoding process for compression requires various operations. For example, a series of processes including spatial segmentation of the digital video, segmentation and / or processing in color channels, removal of spatial redundancy, removal of temporal redundancy, tracking of motion vectors within the video, encoding of differential images, quantization, coefficient scan, run-length coding, entropy coding, and loop filtering may be required. These encoding operations generally consume computing resources and require a certain amount of time to complete. Similarly, the decoding operation for the above encoding operations also requires a certain amount of computing resources and a certain amount of time. The main objective of video encoding and decoding technology is to ensure that the aforementioned resource consumption and time requirements do not hinder the production, recording, distribution, and viewing of digital video.

[0009] Accordingly, the present invention provides a new technology capable of contributing to at least one of the following technical problems in the field of video encoding and decoding as described above: improvement of encoding efficiency, improvement of decoding efficiency, improvement of video quality, reduction of computational load, reduction of software size, reduction of hardware size, and other improvements in performance related to encoding and decoding.

[0010] The present invention provides a method for constructing a candidate list for an intra prediction mode for intra-screen prediction of a chroma block.

[0011] A video decoding method according to an embodiment of the present invention for solving the aforementioned technical problem may include: a step of obtaining in-frame prediction mode information of a current color block from a bit sequence; a step of identifying illumination blocks at a plurality of locations corresponding to the current color block, wherein the plurality of locations include at least one of the center location, upper-left location, upper-right location, lower-left location, and lower-right location of the current color block; a step of obtaining an in-frame prediction mode from at least one of the identified illumination blocks and including it in a candidate list of in-frame prediction modes for the current color block; a step of obtaining an in-frame prediction mode from at least one color block spatially adjacent to the current color block and including it in the candidate list; a step of determining an in-frame prediction mode of the current color block based on the configured candidate list and the obtained in-frame prediction mode information; and a step of generating a prediction block for the current color block using the determined in-frame prediction mode.

[0012] The step of identifying illumination blocks at the plurality of locations mentioned above may be characterized by being performed when the image is divided into a dual tree structure.

[0013] Acquiring an in-screen prediction mode from the above illumination blocks may be characterized by applying priority in the order of center position, top-left position, top-right position, bottom-left position, and bottom-right position, and checking the next position if the illumination block of the previous position is not available.

[0014] The in-screen prediction mode of the illuminance block included in the above candidate list may be characterized by being limited to a maximum of five different in-screen prediction modes.

[0015] The above method may further include the step of obtaining an in-screen prediction mode from non-adjacent color blocks that are not spatially adjacent to the current color block and including it in the candidate list.

[0016] The step of obtaining an in-screen prediction mode from the above non-adjacent color blocks may be characterized by preferentially selecting the most frequently used in-screen prediction mode and applying a weight inversely proportional to the distance from the current color block.

[0017] The above method may further include the step of including a recently used in-screen prediction mode stored in a buffer of a FIFO (First In First Out) structure in the candidate list.

[0018] The above adjacent color blocks may include at least one of the positions left A1, left A0, upper B1, upper B0, and upper-left B2, and may be characterized by checking the availability of each position to obtain only the in-screen prediction mode of the available block.

[0019] Constructing the above candidate list may be characterized by excluding modes that overlap with already included modes, and limiting the maximum number of the above candidate list to one of 8, 16, or 32.

[0020] The prediction mode information within the screen above may be parsed as an index for the candidate list, and may be characterized by decoding the first bin of the index as a context model and the remaining bins as a bypass mode.

[0021] A video encoding method according to an embodiment of the present invention for solving the above-described technical problem may include the steps of: identifying illumination blocks at a plurality of locations corresponding to a current color block; wherein the plurality of locations include at least one of the center location, upper-left location, upper-right location, lower-left location, and lower-right location of the current color block; obtaining an in-frame prediction mode from at least one of the identified illumination blocks and including it in a candidate list of in-frame prediction modes for the current color block; obtaining an in-frame prediction mode from at least one color block spatially adjacent to the current color block and including it in the candidate list; selecting an in-frame prediction mode for the current color block from the configured candidate list; generating a prediction block for the current color block using the selected in-frame prediction mode; and encoding information representing the selected in-frame prediction mode into a bit sequence.

[0022] The step of identifying illumination blocks at the plurality of locations mentioned above may be characterized by being performed when the image is divided into a dual tree structure.

[0023] The above method may further include the step of obtaining an in-screen prediction mode from surrounding color blocks that are not spatially adjacent to the current color block and including it in the candidate list.

[0024] The step of selecting a prediction mode within the above screen may be characterized by selecting an optimal mode from the candidate list through Rate-Distortion Optimization.

[0025] The above method may further include the step of reordering the configured candidate list based on the Sum of Absolute Transformed Differences (SATD) value.

[0026] An image decoding device according to an embodiment of the present invention for solving the aforementioned technical problem comprises: a parser configured to obtain in-frame prediction mode information of a current color block from a bit sequence; a candidate list constructor configured to identify illumination blocks at a plurality of locations corresponding to the current color block, obtain an in-frame prediction mode from at least one of the identified illumination blocks and include it in an in-frame prediction mode candidate list, and obtain an in-frame prediction mode from at least one color block spatially adjacent to the current color block and include it in the candidate list; a mode determination unit configured to determine an in-frame prediction mode of the current color block based on the constructed candidate list and the obtained in-frame prediction mode information; and an in-frame prediction unit configured to generate a prediction block for the current color block using the determined in-frame prediction mode, wherein the plurality of locations may include at least one of the center location, top-left location, top-right location, bottom-left location, and bottom-right location of the current color block.

[0027] The above candidate list configuration unit may be characterized by identifying illumination blocks at the plurality of locations when the image is divided into a dual tree structure.

[0028] The above candidate list configuration unit may be further configured to acquire an in-screen prediction mode from surrounding color blocks that are not spatially adjacent to the current color block and include it in the candidate list.

[0029] The above device may further include a FIFO buffer unit that stores a recently used in-screen prediction mode, and the candidate list configuration unit may be characterized by obtaining an in-screen prediction mode from the FIFO buffer unit and including it in the candidate list.

[0030] The above candidate list configuration unit may be characterized by being configured to exclude modes that overlap with already included modes and to limit the maximum number of the above candidate list.

[0031] The present invention constructs a candidate list to predict the most suitable in-frame prediction mode for a target color block. After identifying the in-frame prediction mode for one or more (corresponding) luminance (luma) blocks at the same location as the target color block, the area is slightly expanded to identify the in-frame prediction mode for one or more adjacent color blocks of the target color block. Subsequently, the area is expanded further to identify the in-frame prediction mode for one or more non-adjacent color blocks of the target color block, thereby improving compression performance.

[0032] FIG. 1 is a conceptual diagram of a video communication system according to an embodiment of the present invention,

[0033] FIG. 2 is a conceptual diagram of the arrangement of an encoder and a decoder in a real-time video streaming environment according to an embodiment of the present invention.

[0034] FIG. 3 is a conceptual diagram of a functional unit of a video decoder according to an embodiment of the present invention,

[0035] FIG. 4 is a conceptual diagram of a functional unit of a video encoder according to an embodiment of the present invention,

[0036] FIG. 5 is a conceptual diagram of a frame type according to an embodiment of the present invention,

[0037] FIG. 6 is a conceptual diagram showing the structure of a video encoder according to another embodiment of the present invention,

[0038] FIG. 7 is a conceptual diagram illustrating a directional prediction mode of VVC according to an embodiment of the present invention.

[0039] FIG. 8 is a conceptual diagram showing the concepts of a single tree and a dual tree according to an embodiment of the present invention.

[0040] FIG. 9 is a conceptual diagram showing a position within an illuminance block corresponding to a color block in one embodiment of the present invention,

[0041] FIG. 10 is an exemplary diagram of spatially adjacent candidate locations in one embodiment of the present invention, and

[0042] FIG. 11 is an example of spatially non-adjacent candidate locations in one embodiment of the present invention.

[0043] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.

[0044] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of multiple related described items or any one of the multiple related described items, and is non-exclusive unless otherwise indicated. When items are listed in this application, they are merely illustrative descriptions intended to facilitate the explanation of the spirit of the present invention and possible methods of implementation, and are therefore not intended to limit the scope of the embodiments of the present invention.

[0045] In this specification, "A or B" may mean "only A," "only B," or "both A and B." Alternatively, in this specification, "A or B" may be interpreted as "A and / or B." For example, in this specification, "A, B or C" may mean "only A," "only B," "only C," or "any combination of A, B and C."

[0046] A slash ( / ) or a comma used in this specification may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B or C."

[0047] In this specification, "at least one of A and B" may mean "only A," "only B," or "both A and B." Additionally, in this specification, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as synonymous with "at least one of A and B."

[0048] Additionally, in this specification, "at least one of A, B and C" may mean "only A," "only B," "only C," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0049] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0050] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0051] Unless otherwise defined, all terms used herein, including technical or scientific terms, are used with the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0052] In describing the invention in this application, embodiments may be described or illustrated in terms of the described functions or unit blocks that perform the functions. The blocks may be expressed in this application as one or more devices, units, modules, parts, etc. The blocks may be implemented in hardware by a method of implementing one or more logic gates, integrated circuits, processors, controllers, memory, electronic components, or information processing hardware, which are not limited thereto. Alternatively, the blocks may be implemented in software by a method of implementing application software, operating system software, firmware, or information processing software, which are not limited thereto. A single block may be implemented by being separated into multiple blocks that perform the same function, or conversely, a single block may be implemented to perform the functions of multiple blocks simultaneously. The blocks may also be implemented by being physically separated or combined according to any criteria. The blocks may be implemented to operate in an environment where their physical locations are not specified and they are spaced apart from each other by a communication network, the Internet, a cloud service, or a communication method not limited thereto. Since all of the above-mentioned methods of implementation fall within the scope of various embodiments that a person skilled in the art familiar with the field of information and communication technology can adopt to realize the same technical concept, any detailed methods of implementation should be interpreted as being included within the scope of the technical concept of the invention in this application.

[0053] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, to facilitate overall understanding, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted. Furthermore, it is assumed that multiple embodiments are not mutually exclusive and that some embodiments may be combined with one or more other embodiments to form new embodiments.

[0054]

[0055] digital video codecs

[0056] FIG. 1 is a conceptual diagram of a video communication system according to an embodiment of the present invention. The video communication system (100) may be configured to include at least two terminals (110, 120) connected to each other through a network (105).

[0057] In one embodiment of the present invention, FIG. 1 may represent a block diagram for configuring a unidirectional video communication network. A first terminal (110) among the terminals may encode the video data in order to transmit (111) the video data through a network (105). A second terminal (120) among the terminals may be configured to receive (121) the encoded video data through a network, decode it, and display it.

[0058] In another embodiment of the present invention, FIG. 1 may represent a block diagram for configuring a bidirectional video communication network. For the bidirectional video communication, each terminal (110, 120) may be configured to encode video data acquired by itself for video transmission (112, 122) to each other terminal passing through the network. Each terminal may also receive (113, 123) video data transmitted through the network by another terminal, decode it, and be configured to display the decoded video data.

[0059] Each of the terminals (110, 120) shown in FIG. 1 may be exemplified as a device such as a server computer, a personal computer, a portable computer, and a smartphone, depending on the embodiment, but is not limited thereto. The present invention is applicable to any environment for establishing a unidirectional or bidirectional video communication network, and the network (105) should be understood to be formed by any means for carrying encoded video data between the terminals (110, 120).

[0060] In one embodiment of the present invention, the network (105) may refer to a wired or wireless communication network. In this case, depending on the embodiment, the network may be configured to communicate information using any communication standard, and the communication standard may include packet-based communication. The term "packet communication" may be understood to mean, for example, packets known as TCP or UDP.

[0061] However, in another embodiment of the present invention, the network (105) may be understood to include a process of information transmission using a recording medium. In this case, the configuration of the network is not limited to communication media only, but should be understood to include a process of temporarily storing and physically transporting information on a hard disk, solid-state disk (SSD), flash memory, CD (compact disc), DVD (digital versatile disc), Blu-ray disc, and other mechanical, electronic, or optical recording media.

[0062] Any other means of information communication or transport applied may be considered to fall within the scope of embodiments of the present invention as long as it is structured to support decoding by transmitting video data in an encoded state. Accordingly, in addition to the examples listed above, all means of information communication or transport known in the prior art or newly provided may fall within the scope of application of the present invention.

[0063] FIG. 2 is a conceptual diagram of the arrangement of encoders and decoders in a real-time video streaming environment according to an embodiment of the present invention. The streaming system (200) exemplified by FIG. 2 can be seen as applicable to a video data communication network including, for example, digital broadcasting, video telephone, and video conferencing. However, it should be seen that a technical structure identical or similar to the streaming system can be equally applied even when information transport via a recording medium is involved, as described above.

[0064] According to one embodiment of the present invention, the streaming system may include a video source (210) that generates a video stream. The video source may include a digital video acquisition means (212) that acquires uncompressed raw video, which may be composed of, for example, a digital camera or other equipment. The raw video stream (215) may have a massive capacity and may therefore be compressed by a video encoder (217) coupled to or connected to the video source.

[0065] The above encoder (217) may be composed of means including hardware, software, or a combination of both, configured to implement an image encoding method and / or a method of implementing the same according to an embodiment of the present invention.

[0066] By passing through the encoder (217), an encoded bitstream (219) with a reduced capacity compared to the original video stream can be output. The bitstream (219) can be provided in real-time via communication by a relay device, for example, which may be referred to as a streaming server (220), and / or can be stored in a recording medium (225) of the streaming server (220) for subsequent use.

[0067] The streaming system (200) may include at least one streaming client (230, 240) that connects to the streaming server (220) to receive the encoded bit sequence (229) in real time or to acquire it subsequently. The streaming client may include a video decoder (232) that acquires the encoded bit sequence (229) (which may also be considered as a copy of the bit sequence (219) received by the streaming server), decodes the bit sequence (229), and outputs the resulting video data as video data in a form that can be displayed on a display (235) or other visual, auditory, or other sensory display means.

[0068] As mentioned above, the functions for encoding and decoding video data are collectively referred to as the coder-and-decoder system, or video codec.

[0069] FIG. 3 is a conceptual diagram of a functional unit of a video decoder according to an embodiment of the present invention. As shown in FIG. 3, a receiver (310) can receive at least one encoded video data to be decoded by a decoder (305). In an embodiment of the present invention, the encoded video data may be independent for each reception, and the decoding procedure of each independent video data may be independent from the decoding procedure of other video data. The encoded video data may be received by the receiver (310) through a hardware or software connection (315) to a device that stores the data, and as described above, the device that stores the data may be a type of streaming server located on the opposite side of a communication network or may refer to a physical recording medium, but is not limited thereto.

[0070] The receiving unit (310) can receive the encoded video data along with other accompanying data, such as encoded audio data or other auxiliary data, and each of the data can be separated from the video data and provided to an appropriate processing unit (312) other than the video decoder.

[0071] When the video data is received through a communication network, a buffer memory (320) may be combined between the receiver (310) and the decoder (305) to minimize delays and interruptions caused by the network environment. The buffer memory (320) may refer to a computer-readable recording medium that temporarily stores the received video data and reliably supplies it to a parser (330) corresponding to the input terminal of the decoder (305). However, the buffer memory may be unnecessary in environments where the bandwidth of the communication network is sufficient, where video data is being read from a recording medium located at a local location that is not physically separated, or where the possibility of communication delay is not predicted.

[0072] The video decoder (305) may include the parser (330) as its input to interpret the encoded video data. The parser separates (parses) a number of pieces of information stored in the form of bit sequences in the encoded video data according to a predetermined rule, and, if necessary, performs the function of entropy decoding (335) the entropy-coded video data, thereby reconstructing the symbols (338) which are segments of video encoding information. The symbols (338) may include all information for controlling the operation of the decoder (305), and / or may further include information for controlling a device that can operate in conjunction with the decoder (305), such as a display device. Control information for controlling the above-mentioned display device may include information in a format referred to as supplementary enhancement information (SEI) or video usability information (VUI).

[0073] As described above, the parser (330) may be configured to perform entropy decoding (335) of the encoded video data. The method of entropy encoding of the encoded video data may vary depending on the standard of the encoding, and decoding may be performed accordingly. Representative examples of the entropy encoding standard may include variable length coding, Huffman coding, and arithmetic coding, and each of the encoding methods may be context-adaptive or context-sensitive depending on the standard, or may be based on principles widely known to a person skilled in the art.

[0074] The parser (330) may be configured to extract at least one picture from the encoded video data. The definition of the picture may vary depending on the specification of the encoding, and depending on the specification, one or more of the examples listed below may simultaneously overlap. The picture may be grouped, defined, and / or divided into encoding / decoding units such as, for example, group of pictures (GOPs), pictures / frames, tiles, slices, macroblocks, blocks, subblocks, transform units (TUs), and prediction units (PUs).

[0075] The parser (330) may be configured to extract encoding information, such as transform coefficients, quantization parameters (QPs), and / or motion vectors, from the encoded video data. The parser (330) may be configured to perform entropy decoding (335) and parsing operations on the video data received from the buffer memory, and to selectively decode symbols (338) representing the encoding information. Additionally, the parser (330) may be configured to selectively supply specific symbols (338) to specific decoding function units within the decoder (305), such as inverse quantization and inverse transform units (340), intra prediction units (350), inter prediction units (355), or loop filter units (360). The control of such information supply may be determined by the information permutation contained in the encoded video and may vary depending on the encoding standard; as such, it is not limited to the scope of the embodiments of the present invention and is not described in detail in this conceptual diagram.

[0076] The above decoder (305) may be composed of a plurality of conceptual functional units that receive and process the encoding information from the parser (330). It is obvious that these conceptual functional units may be combined with one another or further subdivided according to implementation needs. For example, they may be further separated for ease of implementation, or integrated into one for operational efficiency. In any case, each functional unit may be configured to perform close interaction with one another. However, despite the possibility of such integration or separation, the decoding procedure of video data applied as an embodiment of the present invention will be described as a combination of conceptual functional units as described below.

[0077] The above decoder may include an inverse quantization and inverse transform unit (340). The inverse quantization and inverse transform unit (340) may be configured to receive encoding information from the parser (330), including a method to be used for numerical transformation, a block size, quantization coefficients for recovering quantized information, and separation information of a quantization matrix representing the quantization coefficients in a simplified manner, and may be configured to output block values ​​(341) that can be input to an aggregator (370) as a result of processing the encoding information.

[0078] In one embodiment of the present invention, the output values ​​of the inverse quantization and inverse transformation unit (340) may include in-frame predicted encoded block values. The in-frame predicted block value may mean a value that can be decoded using prediction information within the picture currently being decoded, such as the current frame, without using prediction information from a previously decoded picture, such as a previous frame.

[0079] The prediction information within the current picture may be provided by the in-picture prediction unit (350). According to an embodiment of the present invention, the in-picture prediction unit (350) generates a block value of the same form as the block being decoded as prediction information using picture information of a spatially adjacent area derived from a picture that is currently being decoded and has been partially decoded. The picture information may be provided (381) from a buffer for the current picture, so-called line buffer (380). According to an embodiment, the merging unit (370) may be configured to merge the prediction information (351) generated by the in-picture prediction unit (350) with the block values ​​(341) provided by the inverse quantization and inverse transformation unit (340).

[0080] In another embodiment, the output values ​​of the inverse quantization and inverse transformation unit (340) are inter-frame prediction encoded block values, and in some cases, may include block values ​​for which motion compensation has been performed. In this case, the inter-frame prediction unit (355) may extract and use sample information (386) used for motion-based prediction from the reference picture buffer (385). The information (356) derived by performing motion compensation on the sample information based on the symbols (338) included in the block value as the output value may be configured to be merged by the merging unit (370) with the block values ​​(341) provided by the inverse quantization and inverse transformation unit (340). In this case, the block values ​​(341) may be referred to as so-called different or residual values.

[0081] The position information in memory used by the inter-frame prediction unit (355) to extract the sample information from the reference picture may be determined by a motion vector provided to the inter-frame prediction unit (355), which is composed of, for example, a combination of X, Y, and other symbols (338) for representing specific points of the reference picture. The inter-frame prediction unit (355) may also include a function to interpolate and use the sample values ​​when a so-called 'subsampling' possible motion vector is provided, and may further include a function to predict and reinforce the value of the motion vector.

[0082] The output values ​​(371) of the merging unit (370) are provided to the loop filter unit (360) and can be processed by various loop filtering methods. The loop filter unit (360) may be configured to receive not only the block unit output (371) of the merging unit (370) but also the symbol (338) provided by the parser (330) to control its operation. The output of the loop filter unit (360) may be output to an external display means, such as the display device, through an output connection (390), but may be stored (361) in a line buffer (380) for use in prediction to interpret subsequent in-screen or inter-screen encoded block values, and may also be stored in a reference picture buffer (385) via this.

[0083] Specific pictures, such as frames, can be used as reference pictures for performing predictive decoding during a subsequent decoding process once their decoding is complete. A picture (or frame) can be accumulated step by step in a line buffer (380) and decoding can proceed, and when a frame is decoded, the contents of the line buffer (380) are transferred (383) to the reference picture buffer (385), and a new line buffer (380) can be allocated for decoding a new frame.

[0084] The above video decoder (305) may be configured to perform a decoding operation according to a predetermined video compression technology that may be documented by various international standard specifications or commercial specifications. The specifications may include, for example, H.264, H.265, H.266, etc., which are international standard recommendations defined by the International Telecommunication Union Standardization Committee (ITU-T). A person skilled in the art will understand that each of these recommendations is equivalent to an international standard jointly defined by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). The encoded video data may comply with a specific bitstream syntax defined by the specifications, as defined by the profile and level specified in the video compression specification document and standard document, and specifically within such document, and as required. In addition, the complexity of the encoded video data may be limited to a certain level for compliance with the profile and level. For example, any profile or level may be configured to limit the maximum picture size, the maximum decoding rate, and the maximum reference picture size. These limitations may also, in some embodiments, be further limited through metadata signals regarding the management of the HRD buffer included in the hypothetical reference decoder (HRD) and the encoded video data.

[0085] According to one embodiment of the present invention, the receiver (310) may receive additional redundant data along with the encoded video. The additional data may be considered as part of the encoded video data. The additional data may include information that can be used by the decoder (305) to properly decode the data or to more accurately reconstruct an image that is close to the image before encoding. The additional data may be provided in the form, for example, time, space, or layers for signal-to-noise ratio (SNR) enhancement, redundant slices, redundant pictures, and forward error correction codes.

[0086] FIG. 4 is a conceptual diagram of a functional unit of a video encoder according to an embodiment of the present invention. The encoder (405) may be configured to receive original video information (402) from a video source (401) and perform encoding.

[0087] The original video information (402) may have any suitable bit depth, for example, 8 bits, 10 bits, 12 bits, etc. Additionally, the original video information (402) may have any suitable color space, for example, R / G / B, Y / U / V, Y / Cb / Cr, etc. Additionally, the original video information (402) may have any suitable sampling structure corresponding to the color space, for example, in the form of Y / Cb / Cr 4:2:0, Y / Cb / Cr 4:4:4. The original video information (402) having such a predetermined format may be provided to the encoder in the form of a digital video stream.

[0088] In a unidirectional video communication network, the original video information (402) may be obtained from a recording medium that stores a pre-prepared original video. In a bidirectional video communication network, the original video information (402) may be obtained from a video acquisition device, such as a camera, that generates at least one video transmission stream included in the bidirectional video communication.

[0089] Video data containing the original video information (402) may be composed of a plurality of pictures configured to simulate motion by playing them in chronological order. The pictures may also be expressed as concepts such as frames in addition to pictures. The pictures may include one or more samples depending on the type of sampling structure, color space, etc. used. A person skilled in the art will understand that the terms "sample" and "pixel" in digital images are closely related. The operation of the encoder will be explained below with a focus on such samples.

[0090] According to one embodiment of the present invention, an encoder (405) may be configured to encode and compress pictures (and / or information in which they are grouped or divided) constituting the original video information (402) into the form of encoded video information in real time (or according to other temporal requirements as required by the method of implementation).

[0091] In the above encoder (405), the control unit (450) may be a functional unit configured to control an appropriate encoding speed. The control unit (450) may be configured to control other functional units and to be functionally coupled to the functional units described below. The parameters set by the control unit (450) may include parameters related to bitrate control, such as picture skip, quantizer, and variable values ​​for applying image quality optimization techniques, and may also include values ​​such as picture size, the structure of a group of pictures (GOP), and the maximum search range of motion vectors. A person skilled in the art will be able to understand the various other functions that the control unit (450) may have, and such other functions may be added or removed according to the design of the video encoder optimized for the individual system design.

[0092] According to an embodiment of the present invention, the encoder (405) may be configured to operate in a structure such as a "coding loop" that is well known to a person skilled in the art. To explain it in a simplified manner for example, the coding loop may consist of an internal encoder (so-called "source coder") (410) responsible for receiving a picture to be encoded and generating symbols based on at least one reference picture that has been previously encoded, and a local decoder (420) configured to be connected to the internal encoder. The local decoder (420) may be configured to perform an operation to reproduce sample data to be generated by a decoder (490) located at an actual remote location that receives video information encoded from the encoder (405) by receiving the output of the internal encoder (410).

[0093] Video data composed of sample data reconstructed by the internal decoder (420) can be configured to be input into the reference picture buffer of the encoder (405). As described above, since the internal decoder (420) is implemented to reproduce the result output by the encoder (405) to be decoded at a remote decoder, the video data recorded in the reference picture buffer can also be identical in bit unit to the information of the reference picture buffer held by the remote decoder. That is, the prediction function unit that may be included in the encoder (405) can read values ​​identical to the sample values ​​of the previous frame that the decoder will refer to during the decoding process from the reference picture buffer of the encoder (405).

[0094] As described above, the principle of achieving a match between the encoder (405) and the decoder (490) in reference picture buffers by means of an internal decoder (420) on the side of the encoder (405) is widely known to a person skilled in the art, and the method of responding to an environment where such an environment is not guaranteed (e.g., loss of information due to communication failure) may also follow what is known to a person skilled in the art.

[0095] An example of the operation method of the internal decoder (420) described above has been explained in detail with reference to FIG. 3. The decoder of FIG. 3 can be considered as the decoder (490) of the "remote location" described above. The internal decoder (420) may be implemented excluding lossless encoding and decoding sections such as the parser (330) or entropy decoding (335), because the internal encoder (405) is implemented simply to reproduce the operation of the decoder located at the remote location, so it is acceptable to decode the symbols immediately without requiring a process of compressing and restoring the symbols. Therefore, it is acceptable for the functional parts preceding the parser and entropy decoder shown in FIG. 3 not to be provided or at least to be implemented only partially.

[0096] As described above, according to a preferred embodiment of the present invention, any decoder function part present in the decoder (excluding the parser and entropy decoder) may naturally exist as substantially the same function part in the corresponding encoder (405).

[0097] The operation of the encoding function unit that may be included in the encoder (405) can be considered as the inverse operation of the decoder function unit. Therefore, the embodiment can generally be explained by performing the operation of the decoder function unit in reverse. For example, a quantization and transform function unit corresponding to the inverse quantization and inverse transform unit may be provided, and an inter-frame prediction encoding unit corresponding to the inter-frame prediction unit may be provided. In addition to this, some additional explanations will be added.

[0098] The internal encoder (410) may be configured to perform encoding for input picture information, e.g., an input frame, by a predictive encoding method executed by a predictive encoding unit (440) that operates by referencing at least one picture (or frame) encoded in a temporally earlier order from video data designated as at least one reference picture information, e.g., a reference frame, from a reference picture buffer (430). In this case, the encoder (405) may be configured to encode a differential between blocks of samples constituting the input picture and blocks of samples constituting the reference picture.

[0099] The internal decoder (420) can decode video data that can be designated as the reference picture from the symbols generated by the internal encoder (410). As described above, since this video data is identical to the decoding operation performed by the remote decoder, the video data used as the reference picture may be provided to the encoder (405) in a form that has undergone lossy compression and has some damage, and this operation may be intended to match the operation with the decoder.

[0100] The prediction encoding unit (440) may be configured to perform a prediction search operation within the encoder (405). The prediction search operation may refer to an operation corresponding to the inter-frame prediction or intra-frame prediction described in the description of the decoder. For picture information that is input and scheduled to be newly encoded, the prediction unit may access the reference picture buffer (430) to retrieve information in order to obtain information such as a motion vector, a block shape, metadata that may include the same, and a sample block to be actually referenced, which is information indicating a point of a reference picture that can function as prediction reference information suitable for the new picture information. The prediction encoding unit (440) may operate according to the so-called "sample block by pixel block" standard to obtain appropriate prediction reference information. According to one embodiment of the present invention, at least one prediction reference information pointing to at least one reference picture information stored in the reference picture buffer (430) may be designated for the input picture, as determined based on the search results obtained by the prediction encoding unit (440).

[0101] In one embodiment of the present invention, the control unit (450) may be configured to manage the overall encoding operation of the internal encoder (410), including the setting of parameters used to encode video data.

[0102] The outputs of all the aforementioned functional units may be subject to entropy coding (460) for final output. The entropy coding (460) may include various entropy coding techniques such as variable length coding, Huffman coding, and arithmetic coding as described above for the symbols generated by the various functional units, and each of the coding methods may be context-adaptive or context-sensitive methods depending on the standard, or may be based on principles widely known to a person skilled in the art. Such entropy coding (460) can typically achieve lossless compression and may be configured to convert at least one symbol generated by the functional units into encoded video data.

[0103] The control unit (450) may apply a type of encoding in which a specific picture is encoded during the encoding interval to each picture (or frame) in controlling the operation of the encoder (405). Depending on the type, the way the picture is encoded may be affected. Depending on the embodiment, the type may include being classified into the following "frame types."

[0104] FIG. 5 is a conceptual diagram of a frame type according to an embodiment of the present invention. The following description will be explained together with reference to FIG. 5.

[0105] An intra ("I") picture (510) may refer to a picture that can be encoded and decoded using only its own information without referencing other picture information within the video data through in-frame predictive encoding. The "I" picture may be designated by names such as key frame, independent / instantaneous decoder refresh (IDR) frame, and clean random-access (CRA) frame according to the video encoding standard. The "I" picture designated by such various names may have various modifications and application methods as permitted by each standard and may differ partially from one another. In addition to those listed above, various application methods for implementing the "I" picture may be based on various methods that are already known to a person skilled in the art or may be newly provided.

[0106] A prediction ("P") picture (520) may mean a picture that can be encoded and decoded through in-frame or inter-frame prediction based on at least one prediction information and / or motion vector pointing to at least one reference picture to predict sample values ​​of a block constituting the picture. The "P" picture may be configured to reference only one reference frame according to the video encoding standard, or configured to reference one or more reference frames. In the case of referencing one or more reference frames, sample information and / or associated metadata derived from multiple reference pictures may be used for the reconstruction of a single block. However, in common cases, a picture designated as a "P" picture may be understood as a picture that performs a reference limited to a temporally preceding picture.

[0107] A bidirectional prediction ("B") picture (530) may mean a picture that can be encoded and decoded through in-frame or inter-frame prediction based on at least one prediction information and / or motion vector pointing to at least two reference pictures to predict sample values ​​of blocks constituting the picture. In common cases, the picture designated as the "B" picture is distinguished from the picture designated as the "P" picture and may be understood as a picture performing the reference, not limited to a picture that precedes in time.

[0108] Video data is spatially divided into multiple sample blocks during the encoding and decoding process, and encoding can proceed in blocks. The block units include, for example, sizes such as 4x4, 8x8, 4x8, or 16x16 in units of horizontal / vertical pixels, as is widely known, but are not limited thereto. The blocks may be encoded by a predictive encoding method by referencing any other (already encoded) blocks, as allowed and / or restricted by the type specified for each picture in which the blocks are included. For example, blocks of the "I" picture (510) may not use a predictive encoding method, or may be encoded by referencing blocks that have already been encoded within the same part of the picture. That is, only the so-called intra-frame prediction method may be used. In contrast, for the "P" picture (520), at least one reference picture encoded in a previous time unit may be referenced, and thus, inter-frame prediction may also be used in encoding along with intra-frame prediction. In the case of the "B" picture (530), reference can be performed even among reference pictures that are encoded earlier in the encoding order but follow later in terms of time unit. However, it is widely known that there may be blocks encoded without relying on predictive encoding within the "P" picture or the "B" picture.

[0109] The above video encoder (405) may be configured to perform encoding operations according to a predetermined video compression technology that may be documented by various international standard specifications or commercial specifications. Examples of the specifications may include all those described in the decoder.

[0110] According to one embodiment of the present invention, a transmitting unit (470) may buffer the encoded video data generated by the entropy encoding in order to provide / transmit the video data (ultimately to a decoder (490) at a remote location) through a hardware or software connection (495) to a device storing the encoded video data. According to an embodiment, when providing / transmitting the encoded video data from the video encoder (405), the transmitting unit (470) may receive and merge other data accompanying the encoded video data, such as encoded audio data or other auxiliary data, from a separate source (480).

[0111] According to one embodiment of the present invention, the transmitting unit (470) may be configured to transmit additional data along with the encoded video. The additional data may be considered as part of the encoded video data. The additional data may include information that can be used by a decoder to properly decode the data, or to more accurately reconstruct an image that is close to the image before encoding. Examples of the additional data may include all the examples previously shown in relation to the receiving unit (310) of the decoder.

[0112] The present invention can be implemented by digital video compression standards that are understood by and widely used by people skilled in the art, as described above. The digital video compression standards may include at least one of the compression standards known by standard names such as MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AV1, QuickTime, VP-9, VP-10, and Motion JPEG.

[0113] FIG. 6 is a conceptual diagram showing the structure of a video encoder according to another embodiment of the present invention. What is shown in FIG. 6 may be the approximate structure of a video encoder widely known by standard codes such as ITU-T H.266 and ISO / IEC 23090-3, and by the designation MPEG-I Part 3 or the common name versatile video coding (VVC).

[0114] According to FIG. 6, a video encoder (605) may be configured to receive original video data (601) that is not compressed or encoded as input and output an encoded bit sequence (602). When the video data (601) is encoded within a frame, it may be supplied directly to a luminance signal mapping unit (610a), or supplied to a luminance signal mapping unit (610b) via an inter-frame prediction unit (620) that includes motion vector extraction. When encoded within a frame, the mapped luminance signal may be supplied to an output merger (606) by selecting (608) at least one of the following: an intra-frame prediction encoded signal via an intra-frame prediction unit (625), or an inter-frame prediction encoded signal output from the luminance signal mapping unit (610b) via an inter-frame prediction unit (620). The output of the above output merger can be applied to a chroma scaling unit (615). (The operation of the above illuminance signal mapping unit (610) and the operation of the above chroma scaling unit (615) are collectively referred to as the luma mapping / chroma scaling (LMCS) process.) The reduced chroma signal can be provided to a transform unit (630), and the transform unit (630) can perform an adaptive color transform, particularly on the chroma signal. The coefficients derived as a result of the transformation are applied to a quantization unit (640) and quantized. This performs lossy compression, and the result of the lossy compression can be output as a bit sequence (602) after passing through a multi-hypothesis context-adaptive arithmetic coding unit (650), which is a lossless compression method.

[0115] Meanwhile, the result of the lossy compression above can enter the decoding procedure by undergoing inverse quantization (645), inverse transform (635), and luminance signal expansion (617) processes to generate a coding loop. The result of the luminance signal expansion above can be supplied to an internal merger (607) along with the result of selecting (608) at least one of the previously generated intra-frame prediction coded signal or inter-frame prediction coded signal. The result of the internal merger above can undergo inverse luminance signal mapping (617) and then undergo processing such as a deblocking filter (660), a sample adaptive offset (SAO) (670), and an adaptive loop filter (ALF) to reproduce the image quality improvement process in the decoder. The result of reproducing the operation in the decoder as described above is applied to the reference picture buffer (690) and can be recycled for prediction encoding by the inter-frame prediction unit (620).

[0116] The present invention may also be utilized by or combined with an enhanced compression model (ECM), which is an implementation of a next-generation video codec being developed by the Joint Video Experts Team (JVET), an international standardization expert organization. The enhanced compression model may include an enhanced intra-frame prediction coding method, an enhanced inter-frame prediction coding method, an enhanced transform and transform factor coding method, an enhanced adaptive loop filtering method, a bilateral filtering method, a new sample adaptive offset (SAO) method for image quality improvement, an extended entropy coding method, and an enhanced gradual decoding refresh (GDR) technique.

[0117]

[0118] In-screen prediction technology

[0119] Versatile Video Coding (VVC), an international standard for video compression, can provide a total of 67 intra-prediction modes, including a planar mode, a DC mode, and 65 angular prediction modes, as an intra-prediction coding method. Fig. 7 is a conceptual diagram illustrating the angular prediction modes of VVC according to an embodiment of the present invention. Referring to Fig. 7, modes 0 and 1 are non-angular modes without directionality, such as the planar mode and the DC mode, respectively, while modes 2 through 66 represent 65 angular prediction modes. The angular prediction modes can be configured to adaptively respond to edge and texture directions within the image by providing prediction directions of various angles.

[0120] A single chroma block is encoded in one of 67 intra-frame prediction modes, but it is not necessary to support all 67 intra-frame prediction modes for the chroma block every time. This is because, in order to increase the encoding efficiency and reduce the complexity of the chroma block, a method is adopted to provide only a few highly probable modes as a candidate list.

[0121] The intra-modes supported for a single color block to be encoded are shown in Table 1. One mode is selected from a total of five candidates through evaluation. The selected intra-mode can transmit the corresponding index value by binarizing it.

[0122] The 5 candidates (based on index) may include the following.

[0123] - Index 0: Planar mode 0.

[0124] - Index 1: 50 vertical mode.

[0125] - Index 2: Horizontal mode 18.

[0126] - Index 3: DC Mode 1.

[0127] - Index 4: DM (direct mode).

[0128] The above DM (Direct Mode) may refer to a method of using the in-screen prediction mode of an illuminance (luma) block that is spatially co-located with the color block. By utilizing the high correlation between the illuminance component and the color component, the above DM can contribute to improving the prediction efficiency of the color block. If the DM matches one of the modes of planar, DC, vertical, or horizontal, directional mode 66 is added, allowing for a total of five candidates to be formed.

[0129] Here, DM may refer to the in-frame prediction mode for the luminance (luma) block at the same location as the color block. For example, if the in-frame prediction mode of the luminance block is mode 10, the DM of the corresponding color block becomes 10, and the five candidates are composed of (mode number) 0, 50, 18, 1, and 10 in index order. As another example, if the in-frame prediction mode of the luminance block is mode 50 (vertical), the DM of the corresponding color block becomes 50, and the five candidates are composed of (mode number) 0, 66, 18, 1, and 50 in index order.

[0130] intra_chroma_pred_modelumaIntraPredMode050181X ( 0≤X≤66 )0660000150665050502181866181831116614050181X

[0131] DM mode is an in-frame prediction mode that utilizes the high correlation between the illuminance block and the color block. Table 2 is a table showing the binarization results for the mode index of the color block according to an embodiment of the present invention. Referring to Table 2, in the case of DM mode (index 4), the binary value is represented as 1 bit ("0"), while for the remaining 4 modes (indexes 0 to 3), the binary value can be assigned as 3 bits ("100" - "111"). Through this, it can be seen that DM mode is used more frequently than other modes in the case of the color block. The binarization process can produce a compression effect by allocating a small number of bits to frequently occurring signals and a relatively long number of bits to infrequently occurring signals. That is, since DM mode has the highest probability of being statistically selected, it is assigned the shortest codeword.

[0132] Value of intra_chroma_pred_modeBin string010011012110311140

[0133] In the VVC standard technology, to improve the performance of in-frame prediction, a function can be provided to set the position and size of CUs by separately distinguishing the block partitions of the illuminance plane and the color plane. This is referred to as a dual tree (DUAL_TREE), and the block partition information for the illuminance plane can be referred to as DUAL_TREE_LUMA, while the block partition information for the color plane can be referred to as DUAL_TREE_CHROMA. On the other hand, the case where the existing block partition methods for the illuminance plane and the color plane are set commonly can be referred to as a single tree (SINGLE_TREE).

[0134] FIG. 8 is a conceptual diagram illustrating the concepts of a single tree and a dual tree according to an embodiment of the present invention. Referring to FIG. 8, at least one of a dual tree or a single tree can be selected only in an intra-slice (I-slice). In conventional standard specifications, the dual tree structure was limited to application in intra-slices, but in the latest improved specifications, it can be extended to allow dual tree configuration in inter-slices, i.e., P-slices or B-slices. In addition, the dual tree can be applied to various color sampling structures such as YUV 4:2:2 or YUV 4:4:4 formats in addition to the YUV 4:2:0 format.

[0135] Since the split positions and shapes of the color block and the illuminance block in a single tree are identical, the DM mode of the color block can be set by taking the in-frame prediction mode used in the illuminance block at the (corresponding) same position. In this case, since exactly one illuminance block corresponds 1:1 to one color block, the definition of DM is clear and unambiguous.

[0136] However, in the case of a double tree, the block division shape of the illuminance block and the block division shape of the color block are different, so the in-frame prediction mode value of the illuminance block cannot be directly used as the DM (in-frame prediction mode) value of the color block. This is because there may be multiple illuminance blocks corresponding to a single color block, rather than just one. In other words, in a double tree structure, a 1:N matching relationship may occur between the color block and the illuminance block, and in this case, a problem may arise where it is unclear which illuminance block's in-frame prediction mode to select as the DM.

[0137]

[0138] Composition of the present invention

[0139] In conventional video encoding / decoding standards, methods such as allowing only 5 out of 67 intra-frame prediction modes for the encoding target block and selecting the final mode out of the 5 were used, but the present invention allows more than 5 intra-frame prediction modes for the target color block to improve compression performance for the color block.

[0140] The present invention may include a method for constructing a candidate list of in-screen prediction modes for in-screen prediction of a color block. The method for constructing the candidate list may adopt a hierarchical approach of sequentially collecting in-screen prediction mode information from blocks expected to have the highest correlation with the target color block.

[0141] In addition, the present invention may also include a method for setting a DM mode that includes both cases where the slice is divided into a double tree and cases where it is divided into a single tree. In particular, the invention may provide a method for effectively defining and utilizing DM even in 1:N block matching situations occurring in a double tree structure. (In this case, N may mean a natural number greater than or equal to 1.) This can be achieved by selectively utilizing one or more DM candidates by considering the positional characteristics of each illuminance block when there is one or more illuminance blocks corresponding to a single color block.

[0142] It should be understood that the present invention is not limited to cases where the dual tree function is supported in the case of intra-slice (I-slice), but can also be applied in cases where dual tree settings are allowed to enhance compression performance in inter-slice (P-slice, B-slice). For example, the method of the present invention can be effectively applied to color blocks encoded by intra-slice within an inter-slice prediction encoding slice. Furthermore, the present invention can be applied to YUV 422 or YUV 444 formats in addition to YUV 420 when dual trees are used.

[0143] In the present invention, in a plurality of embodiments for constructing a candidate list for an in-frame prediction mode for in-frame prediction of color blocks, the candidate list can be organized into several groups. Each of these groups is designed to utilize different spatial correlation characteristics, enabling adaptive response to various image characteristics. Each group is mentioned as a technical component of the present invention, and some non-limited embodiments possible through the combination of said components are listed.

[0144]

[0145] 1st composition

[0146] The first configuration of the present invention may mean obtaining in-screen prediction mode information from illumination blocks at corresponding locations that extend the DM. Specifically, the DM of the present invention may include in-screen prediction modes of one or more illumination blocks. According to a preferred embodiment of the present invention, it may be configured to selectively utilize up to five illumination block modes, and depending on the configuration method of the embodiment, the number of illumination block modes may be determined as any number without being bound by an upper limit.

[0147] In the case of a single tree, there is one illuminance block at a position corresponding to a color block, but in the case of a double tree, the block division shape of the illuminance block and the block division shape of the color block are different, so there may be multiple illuminance blocks at a position corresponding to a color block. In such a 1:N matching situation, the present invention is not limited to a single DM value, but can improve prediction accuracy by utilizing one or more DM candidates derived from multiple positions.

[0148] FIG. 9 is a conceptual diagram showing the location within an illuminance block corresponding to a color block in an embodiment of the present invention. Referring to FIG. 9, the left side (910) can be interpreted as an example representing an illuminance area, and the right side (920) as a color area. FIG. 9 shows an example of multiple illuminance blocks at corresponding locations within a single color block in the case of a double tree. Referring to FIG. 9, the hatched portion (921) in the right color area (920) may represent a single color block to be encoded, and the hatched portion (911) in the left illuminance area (910) may represent at least one illuminance block positionally corresponding to the color block (921). For example, the illuminance blocks (911) corresponding to the left color block (921) in the color area may amount to a total of 10 (911-1 to 10).

[0149] The locations of the gray target color blocks and the corresponding illuminance blocks can be represented as the top-left (TL), top-right (TR), center (CR or CT), bottom-left (BL), and / or bottom-right (BR) as shown in the figure. The above five locations were selected as strategic locations that can effectively represent the spatial range of the color blocks, and the in-screen prediction mode of the illuminance block at each location can be seen as reflecting the texture characteristics of the corresponding area.

[0150] In the first configuration above, the in-frame prediction mode values ​​used in the illuminance blocks at the corresponding CT, TL, TR, BL, and / or BR positions can be retrieved and used to construct a candidate list of target color blocks. Specifically, one or more, up to five, in-frame prediction modes from the five positions can be optionally included in the candidate list. In this case, the selection order can be configured to follow the priority order of CT→TL→TR→BL→BR, and a sequential fallback method can be applied in which the TL position is checked if the illuminance block at the CT position is not available, and the TR position is checked if the TL position is not available. This reflects the tendency for the central position to have the highest statistical correlation.

[0151] Additionally, the in-frame prediction mode value used in the illuminance block at the CT position in the first configuration above can be set as DM and used when configuring the candidate list of target color blocks. This provides a clear definition in a dual-tree environment while maintaining compatibility with the DM concept in existing standards. The candidate list of target color blocks can be configured by taking the in-frame prediction mode value used in the illuminance block at the corresponding TL, TR, BL, and / or BR positions other than DM. These extended DM candidates can contribute to capturing local texture changes that a conventional single DM might miss.

[0152] According to one embodiment of the present invention, if the in-frame prediction method used in the illuminance block corresponding to the color block is Intra Block Copy (IBC) or Intra Template Matching (IntraTMP), the block vector (BV) value used in the corresponding illuminance block can be referenced and used to construct a candidate list of the target color block. For example, if the in-frame prediction method used in the illuminance block corresponding to the color block is IBC or IntraTMP, the in-frame prediction mode value for the corresponding illuminance block may not exist separately.

[0153] In addition, the block vector value used in the illumination block at the CT position in the first configuration above can be set as a direct block vector (direct BV; DBV) and used when constructing a candidate list of target color blocks. Additionally, the candidate list of target color blocks can be constructed by referencing the BV value used in the illumination block at the corresponding TL, TR, BL, and / or BR positions other than the DBV. The extended DBV candidates can contribute to capturing local texture changes that a single DBV based on the prior art might miss during the encoding process.

[0154] In the first configuration above, if the in-frame prediction method used in the illuminance block corresponding to the color block uses a general in-frame mode, the in-frame prediction mode value used in the illuminance block can be retrieved and used when constructing the candidate list of the target color block. On the other hand, if the in-frame prediction method used in the illuminance block corresponding to the color block is IBC or IntraTMP, the BV (block vector) value used in the corresponding illuminance block can be retrieved and used when constructing the candidate list of the target color block.

[0155] The positions of the illuminance blocks at the TL, TR, CT, BL, and / or BR positions in FIG. 9 are as shown in (1) to (5) of Equation 1 below. For example, CT may refer to an illuminance block containing a pixel located at a position moved by half the width of the current encoding target block along the horizontal axis and half the height of the current encoding target block along the vertical axis from the current encoding target block position. The above position calculation is performed with integer precision, and rounding operations may be applied as needed. The position, width, and / or height information for the current encoding target block may be based on the illuminance blocks. For convenience, the present invention selects the positions based on illuminance blocks, but color blocks are also possible, and scaling operations may be added to the mapping between illuminance blocks and color blocks as needed. For example, in the case of the YUV 4:2:0 format, 2:1 scaling may be applied, and in the case of the YUV 4:2:2 format, 2:1 scaling may be applied only in the horizontal direction. Specifically, in the case of the YUV 4:2:0 format, 2:1 scaling is applied in both the horizontal and vertical directions, in the case of the YUV 4:2:2 format, 2:1 scaling is applied only in the horizontal direction, and in the case of the YUV 4:4:4 format, scaling may not be required.

[0156]

[0157] In the above mathematical formula 1, IntraPredModeY[x][y] represents in-frame prediction information for the illuminance block at position (x, y), (xCb, yCb) represents the position of the block to be encoded (based on illuminance), and (nCbW, nCbH) may represent the width and height of the block to be encoded (based on illuminance).

[0158]

[0159] Second composition

[0160] The second configuration of the present invention may mean obtaining in-screen prediction mode information from adjacent surrounding color blocks.

[0161] FIG. 10 is an example of spatially adjacent candidate locations in one embodiment of the present invention. FIG. 10 can define spatially adjacent surrounding locations of a color block to be encoded. A1 and A0 may be located to the left of the center target block, B1 and B0 may be located above it, and B2 may be located to the upper left. These locations were selected to be consistent with spatially predicted candidate locations generally used in HEVC and VVC standards, thereby maintaining compatibility with existing hardware implementations.

[0162] In the second configuration above, the in-screen prediction mode values ​​used by color blocks located at adjacent positions A1, A0, B1, B0, and / or B2 can be retrieved and used to construct a candidate list for the target color block. The collection of modes from the adjacent blocks is performed through an availability check, and blocks located at slice boundaries, tile boundaries, or CTU boundaries may be treated as unavailable. Additionally, blocks that have not yet been encoded or decoded may also be treated as unavailable.

[0163] The above availability check may be performed based on conditions such as whether the location is outside the current picture boundary, whether the location crosses the slice boundary or tile boundary, whether the block at the location has not yet been encoded / decoded, or whether the block at the location has been encoded in the in-frame prediction mode. If any of the above conditions are not satisfied, the location is treated as unavailable, and the system may be configured to check the location of the next priority.

[0164]

[0165] Third composition

[0166] A third configuration of the present invention may mean obtaining in-screen prediction mode information used in multiple non-adjacent surrounding color blocks.

[0167] FIG. 11 is an example of a spatially non-adjacent candidate location in an embodiment of the present invention. FIG. 11 illustrates an example of defining a spatially non-adjacent surrounding location (1120) of a color block (1110) to be encoded. It shows the locations of a plurality of surrounding color blocks based on a target color block (1110) located in the center, but the present embodiment is not limited thereto and may include a variety of color blocks located around the target color block that are not adjacent to it. The non-adjacent blocks are generally located at a distance of at least one block away from the current block, and the maximum search range may be limited considering the complexity of implementation.

[0168] In the third configuration above, information on the most frequently used in-frame prediction mode among the in-frame prediction modes used in multiple color blocks in the surrounding non-adjacent area can be utilized. That is, the encoded in-frame prediction modes of the color blocks whose locations are specified in FIG. 11 can be sorted based on frequency, and the in-frame prediction modes that appear most frequently can be brought in order and used to construct a candidate list for the target color block. The frequency calculation can be performed using a simple counting method or a weighted counting method, and if modes with the same frequency exist, the order can be determined according to a predefined priority.

[0169] In addition, by considering the distance between the target block and surrounding blocks, higher weights are assigned to the in-frame prediction modes of blocks at close range and relatively lower weights are assigned to the in-frame prediction modes of blocks at distant range. These are then sorted based on frequency considering (distance) weights, and the in-frame prediction modes with the highest scores are retrieved and used to construct a candidate list for the target color block. Additionally, weights considering block size may be applied, configured so that larger block sizes are assigned higher weights. For example, the weight can be calculated in the form weight = (block_size / reference_block_size) × (1 / distance). The above distance-based weights can be calculated based on Manhattan distance or Euclidean distance, and a weighting function inversely proportional to distance may be applied.

[0170]

[0171] 4th component

[0172] The fourth configuration of the present invention may mean retrieving recently used in-screen prediction mode information from surrounding color blocks.

[0173] Intra-prediction mode information of color blocks generated during the encoding / decoding process can be stored and input in a memory buffer space with a FIFO (first input, first output) structure (hereinafter, intra-prediction mode memory). The intra-prediction mode memory is a form of history-based prediction and utilizes the statistical characteristic that recently used modes are likely to be used again by utilizing the principle of temporal locality.

[0174] In the above fourth configuration, the in-screen prediction mode value entered in the most recent order can be retrieved and used to construct a candidate list of target color blocks. The size of the above FIFO buffer can be determined by considering the trade-off between implementation complexity and performance, and generally, it can have between 4 and 16 entries. A variation of the LRU (Least Recently Used) method can also be applied, in which when a new mode is added to the buffer, the same mode already existing in the buffer is removed and moved to the latest position.

[0175] The prediction mode memory within the screen described above can be reset when starting a single slice of encoding. Additionally, the memory can be reset when starting a row-by-row encoding of a single CTU (coding tree unit) according to the raster scan order. Resetting at the CTU row level provides compatibility with parallelization techniques such as Wavefront Parallel Processing (WPP) and also has the effect of limiting memory usage.

[0176] The present invention may be designed to operate independently of a Multiple Reference Line (MRL), and may also be combined with an MRL to provide extended prediction performance as needed. When operating independently of an MRL, the configuration of the in-frame prediction mode candidate list of the present invention applies only to a single reference line; when combined with an MRL, an independent candidate list may be configured for each reference line, or an integrated candidate list may be configured for all reference lines.

[0177]

[0178] 5th composition

[0179] The fifth configuration of the present invention may mean retrieving prediction mode information within the default screen.

[0180] One or more in-screen prediction mode values ​​from Planar Mode, DC Mode, Vertical Mode, Horizontal Mode, and Diagonal Mode (No. 66) can be retrieved and used to construct a candidate list for the target color block. These default modes are the most statistically used modes and function as fallback options in cases where sufficient candidates cannot be obtained by other prediction methods.

[0181] The present invention is not limited to the above five in-frame prediction modes and may include a predetermined number of basic in-frame prediction modes. For example, diagonal modes at a 45-degree angle (modes near 34 and 50) may be additionally included to effectively predict diagonal edge patterns that frequently appear in natural images.

[0182] Additionally, the prediction mode within the basic screen may refer to directional modes such as -1, +1, -2, +2, etc., based on the first directional mode entered in the existing candidate list. For example, if 33 is entered as the first directional mode in the candidate list of the target color block, it can be used when constructing the candidate list in the order of mode 32, mode 34, mode 31, mode 35, etc. This derived mode method enables response to fine texture changes by providing predictions of angles that are similar to the already selected mode but slightly different. The range of the above derived modes can be extended up to ±4, and a cyclic boundary condition can be applied so that if the mode number falls outside the valid range, it can be mapped appropriately.

[0183]

[0184] Sixth component

[0185] The sixth configuration of the present invention may refer to a configuration applicable to color blocks encoded by inter-frame prediction within an inter-frame slice. Specifically, it may refer to a configuration that identifies a corresponding color block at the same location (collocated) as the color block to be encoded in a collocated picture, and operates by referring to the intra-frame prediction mode information of said color block.

[0186] The inter-frame encoding slice may include information of the same-location reference picture as a high-level bit sequence syntax (HLS). In this case, the HLS may refer, for example, to a slice header (SH) syntax. In this case, one embodiment of the present invention may include a configuration that references mode information when a color block located at the same position as the encoding target block within the same-location reference picture set in the inter-frame slice has intra-frame prediction mode information.

[0187] Additionally, one embodiment of the present invention may include a method for obtaining an in-frame prediction mode of an illuminance block at the same location within a reference picture at the same location corresponding to a block to be encoded. In this case, if an illuminance block at the same location set in an inter-frame slice has in-frame prediction information, the method may include a configuration for referencing the corresponding mode information. In this case, scaling of the block's location may be required depending on the encoding color space of the image. For example, 2:1 scaling may be applied in the case of a YUV 4:2:0 format, and 2:1 scaling may be applied only in the horizontal direction in the case of a YUV 4:2:2 format.

[0188]

[0189] Combination of configurations for carrying out the present invention

[0190] In the present invention, in order to construct a candidate list by predicting the most suitable in-screen prediction mode for a target color block, the in-screen prediction mode for one or more (corresponding) illuminance blocks at the same location as the target color block is checked, and then the area is slightly expanded to check the in-screen prediction mode for one or more adjacent color blocks of the target color block. Subsequently, the area is further expanded to check the in-screen prediction mode for one or more non-adjacent color blocks of the target color block.

[0191] The above hierarchical approach is based on the statistical characteristic that spatial correlation is generally inversely proportional to distance. In other words, information located closer to the current block is more likely to provide higher prediction accuracy. Therefore, the present invention may adopt a strategy of gradually expanding the search area, starting from the illuminance block mode expected to have the highest correlation.

[0192] The following describes various embodiments of the present invention that constitute a list of in-screen prediction mode candidates for actual target color blocks. The following embodiments are not intended to limit the implementable forms of the present invention and are provided only for illustrative purposes. Each embodiment may be optimized for different image characteristics or implementation complexity requirements, and the encoder may select the most suitable configuration based on the Rate-Distortion Optimization criteria.

[0193] In all embodiments, when configuring the candidate list to avoid duplicate information, in-screen prediction modes already entered into the candidate list may not be added. The duplicate removal is performed by comparing each mode with the existing list before adding it to the candidate list, which is intended to efficiently utilize the limited candidate list size and ensure diversity. In all embodiments, the candidate list may be configured using one or more configurations among the aforementioned first to fifth configuration groups.

[0194]

[0195] First embodiment

[0196] In this embodiment, a list of intra-frame prediction mode candidates for the color block to be encoded can be configured into five groups. Additionally, as another example, block vector values ​​including intra-frame prediction modes can also be configured as a candidate list. This is the most comprehensive configuration, seeking maximum prediction accuracy by utilizing all available prediction sources.

[0197] By the first configuration above, in-screen prediction mode information can be obtained from illuminance blocks at corresponding locations that extend DM. Modes can be collected from up to five illuminance block locations (CT, TL, TR, BL, BR) to maximize the utilization of illuminance-color correlations. Alternatively, by the first configuration above, block vector information can be obtained and added from illuminance blocks at corresponding locations obtained by extending DBV. For example, block vectors can be collected from up to five illuminance block locations (CT, TL, TR, BL, BR) to maximize the utilization of illuminance-color correlations.

[0198] By the second configuration above, in-screen prediction mode information can be obtained from adjacent surrounding color blocks. Spatial correlations can be utilized from up to 5 adjacent blocks at locations A1, A0, B1, B0, and B2.

[0199] By the above third configuration, information on in-screen prediction modes used multiple times in non-adjacent surrounding color blocks can be obtained. By reflecting statistical characteristics over a wider area, global texture characteristics that deviate from local patterns can be captured.

[0200] By the above fourth configuration, information on the in-screen prediction mode recently used from surrounding color blocks can be retrieved. Temporal correlation can be utilized through the FIFO history buffer.

[0201] By means of the above fifth configuration, prediction mode information within the basic screen can be retrieved. It can serve as a safety mechanism in cases where other prediction methods fail to provide sufficient candidates.

[0202] This embodiment is suitable for high-definition applications where compression efficiency is prioritized, and although the implementation complexity may be relatively high, it can provide the most comprehensive prediction performance.

[0203]

[0204] Second embodiment

[0205] In this embodiment, a list of intra-frame prediction mode candidates for the color block to be encoded can be configured into four groups. By excluding the history-based prediction (fourth configuration) from the first embodiment, the complexity of memory management can be reduced.

[0206] By the first configuration above, in-screen prediction mode information can be obtained from illumination blocks at corresponding locations by extending the DM. Alternatively, by the first configuration above, block vector information can be obtained and added from illumination blocks at corresponding locations obtained by extending the DBV.

[0207] By the above second configuration, in-screen prediction mode information can be obtained from adjacent surrounding color blocks.

[0208] By the above third configuration, screen prediction mode information used multiple times in non-adjacent surrounding color blocks can be obtained.

[0209] By the above fifth configuration, prediction mode information within the basic screen can be retrieved.

[0210] This embodiment is a balanced configuration that is simple to implement as it does not require FIFO buffer management, while still ensuring sufficient spatial prediction performance.

[0211]

[0212] Third embodiment

[0213] In this embodiment, a list of intra-screen prediction mode candidates for color blocks to be encoded can be configured into four groups. In the first embodiment, the search range can be limited by excluding non-adjacent block search (third configuration) to reduce computational complexity.

[0214] By the first configuration above, in-screen prediction mode information can be obtained from illumination blocks at corresponding locations by extending the DM. Alternatively, by the first configuration above, block vector information can be obtained and added from illumination blocks at corresponding locations obtained by extending the DBV.

[0215] By the above second configuration, in-screen prediction mode information can be obtained from adjacent surrounding color blocks.

[0216] By the above fourth configuration, recently used in-screen prediction mode information can be retrieved from surrounding color blocks.

[0217] By the above fifth configuration, prediction mode information within the basic screen can be retrieved.

[0218] This embodiment is a configuration that eliminates memory access overhead caused by non-adjacent block search while maintaining temporal prediction through a history buffer.

[0219]

[0220] Fourth embodiment

[0221] In this embodiment, a list of intra-frame prediction mode candidates for the color block to be encoded can be configured into three groups. This embodiment can perform effective prediction with minimal complexity.

[0222] By the first configuration above, in-screen prediction mode information can be obtained from illumination blocks at corresponding locations by extending the DM. Alternatively, by the first configuration above, block vector information can be obtained and added from illumination blocks at corresponding locations obtained by extending the DBV.

[0223] By the above second configuration, in-screen prediction mode information can be obtained from adjacent surrounding color blocks.

[0224] By the above fifth configuration, prediction mode information within the basic screen can be retrieved. However, the DC mode may be included in the basic mode group as a non-highest priority, so that other prediction methods are considered preferentially.

[0225] This embodiment excludes Chroma DIMD (Decoder-side Intra Mode Derivation) in consideration of decoder complexity while maintaining compatibility with existing encoding / decoding processing methods, in order to eliminate the additional computational complexity required by the mode derivation process at the decoder stage. In addition, it is a practical configuration that improves prediction accuracy by adjusting the priority of the DC mode. It may be particularly suitable for applications or hardware implementations requiring real-time processing.

[0226]

[0227] Fifth embodiment

[0228] In this embodiment, a list of intra-screen prediction mode candidates for a color block to be encoded can be configured into three groups. In the configuration of this embodiment, the order of the three groups may be changed when configuring the candidate list.

[0229] According to the first configuration above, intra-screen prediction mode information can be obtained from an illuminance block at a location corresponding to the DM. At this time, if the illuminance block at the location corresponding to the DM has block vector information, DBV information can be obtained from the illuminance block. Alternatively, according to the first configuration above, intra-prediction mode information can be obtained from illuminance blocks at corresponding locations obtained by expanding and excluding the DM, or according to the first configuration above, block vector information can be obtained from illuminance blocks at corresponding locations obtained by expanding and excluding the DBV.

[0230] By the above second configuration, in-screen prediction mode information can be obtained from adjacent surrounding color blocks.

[0231] By the above fifth configuration, prediction mode information within the basic screen can be retrieved. Here, planar, horizontal, vertical, and / or DC modes can be included in the basic mode group and used.

[0232] To explain the present embodiment more specifically, in the present embodiment, when forming the three groups based on the combination of each embodiment configuration, at least some of the components described below may be included. For example, it may include a direct mode (DM) or a direct block vector (DBV) from an illuminance block corresponding to the central position of the color block to be encoded, or include basic in-frame prediction modes including flat, horizontal, vertical, and DC, or include in-frame prediction modes from illuminance blocks corresponding to the top-left (TL), top-right (TR), bottom-left (BL), and bottom-right (BR) positions of the color block to be encoded, or include in-frame prediction modes from surrounding color blocks adjacent to the color block to be encoded, or include block vector information from illuminance blocks corresponding to the top-left (TL), top-right (TR), bottom-left (BL), and bottom-right (BR) positions of the color block to be encoded. The type of component selected among the components described above and the order in the candidate list may be changed, and the information obtained from each component may be included sequentially in the candidate list after removing duplicates.

[0233]

[0234] Common features of the examples

[0235] In any method of carrying out the present invention, that is, for all embodiments described above and their ordinary variations, the following examples of methods of carrying out may be applied.

[0236] In all embodiments of the present invention, when the current slice is an inter-frame encoding slice, the sixth configuration may be added to configure an intra-frame prediction mode candidate list for the color block to be encoded. At this time, the temporal_intra_mode_enabled_flag value may be added to the bit sequence so that when the flag value is "on", the candidate list addition by the sixth configuration is performed. The flag value may be a high-level syntax (HLS) value and may be information belonging to one or more of SPS (sequence parameter set), PPS (picture parameter set), PH (picture header), and SH (slice header).

[0237] In all embodiments of the present invention, when the ibc_enabled_flag value is "on", it is possible to determine whether the in-frame prediction method used in the illuminance block corresponding to the color block is IBC or IntraTMP. That is, when the ibc_enabled_flag value is "on", block vector information including the DBV used in the illuminance block corresponding to the color block can be retrieved. Here, the flag value may be information belonging to one or more of the sequence parameter set (SPS), picture parameter set (PPS), picture header (PH), and slice header (SH) as a high-level syntax (HLS) value.

[0238] In all embodiments of the present invention, when configuring a candidate list by dividing into groups, a maximum number k is limited for each group, so that when the maximum number k of a group is filled, in-screen prediction mode information is no longer added to the list from that group. The maximum number per group may be adaptively set considering the statistical contribution and implementation complexity of each group, and may be composed of a fixed value or an adaptive value.

[0239] For example, if the maximum number in the first configuration above is limited to three, the system can be configured to input up to three in-screen prediction modes of the illumination block in sequence into the list and no further in-screen prediction modes of the illumination block are checked (i.e., they may not be input into the list). This can have the effect of considering only CT, TL, and TR positions while ignoring BL and BR positions, thereby limiting the number of memory accesses.

[0240] For example, if the maximum number in the second configuration above is limited to two, it can be configured so that up to two in-screen prediction modes of color blocks are entered into the list in sequence and no further in-screen prediction modes of color blocks are checked (i.e., they may not be entered into the list). This minimizes search complexity by considering only the blocks at the closest locations, such as A1 and B1.

[0241] In addition, in all embodiments of the present invention, the candidate list can be configured by limiting the maximum number of the entire candidate list. The maximum number is generally set to a power of 2 value, such as 8, 16, or 32, to increase the efficiency of index signaling. In this case, the maximum number of the candidate list is a high-level syntax (HLS) value and may be information belonging to one or more of SPS (sequence parameter set), PPS (picture parameter set), PH (picture header), and SH (slice header). When set at the SPS level, a consistent setting can be applied to the entire sequence, and when set at the SH level, adaptive adjustment is possible per slice. If the maximum number is exceeded during the process of configuring the candidate list, the list configuration process may be configured to be terminated.

[0242] Additionally, a reordering process can be optionally performed on the configured candidate list. The reordering can be performed based on RD cost estimation, SATD (Sum of Absolute Transformed Differences) calculation, TM (template matching) calculation, or a statistical probability model, and the entropy encoding efficiency can be improved by rearranging so that the most likely mode has a low index.

[0243] As an example of an embodiment of the present invention, a candidate list may be constructed without limiting the maximum number of all candidate lists, and then the reordering process may be performed, and the final candidate list may be constructed by limiting the maximum number (f) in order of lowest error value. In this case, if at least one block among the illuminance blocks at the position corresponding to the color block is encoded in IBC or IntraTMP, the maximum number (f) may be determined to be 7, and if all illuminance blocks at the position corresponding to the color block are encoded in the in-frame prediction mode, the maximum number (f) may be determined to be 6. As another example, a candidate list may be constructed by limiting the maximum number (g) of all candidate lists, and then the reordering process may be performed, and the final candidate list may be constructed by limiting the maximum number (f) in order of lowest error value. In this case, the maximum number f may be less than or equal to the maximum number g.

[0244] In addition, in all embodiments of the present invention, a candidate list for a color block to be encoded is constructed, and when a single in-frame prediction mode is selected through candidate evaluation, information can be transmitted using the candidate index (chroma_mode_idx) value of the selected in-frame prediction mode. The index may be entropied using a truncated unary code, a fixed-length code, or a context-adaptive binary arithmetic coding (CABAC) method, and the optimal encoding method may be selected based on the size of the candidate list.

[0245] According to an embodiment, when entropy-coding the index of a configured candidate list, the encoding method may be adaptively selected based on the size of the candidate list. A truncated unary code may be used when there are 8 or fewer candidates, a truncated binary code when there are 9 to 16 candidates, and a fixed-length code when there are 17 or more candidates. Additionally, a context model may be applied to the first bin, and the remaining bins may be encoded in bypass mode to improve throughput.

[0246] The present invention provides a method for constructing a candidate list to predict the most suitable in-frame prediction mode for a target color block. After identifying the in-frame prediction mode for one or more (corresponding) illumination blocks at the same location as the target color block, the area can be slightly expanded to identify the in-frame prediction mode for one or more adjacent color blocks of the target color block. Then, by adopting a method of expanding the area further to identify the in-frame prediction mode for one or more non-adjacent color blocks of the target color block, compression performance can be improved.

[0247]

[0248] Encoder and decoder

[0249] It is evident that the encoding method according to the present invention can be applied identically in an encoder and a decoder. First, the encoding method according to the present invention can be used as one of the methods for generating sample values ​​of a prediction block to perform in-frame and / or inter-frame prediction in an encoder. When residual signal information is encoded by the difference value with respect to such a prediction block, the decoder can generate sample values ​​of the same prediction block using a corresponding and / or symmetric method, and obtain decoded samples by combining the difference value with such a prediction block. Additionally, as illustrated in FIG. 4 through an internal decoder (420) and a coding loop including it, this decoding process can be implemented identically within the encoder to predict the state of the decoder.

[0250] The encoding method according to the present invention described above can be implemented through an encoder as a device. The encoder as a device may be implemented by maintaining the conventional encoder structure exemplified in FIGS. 1 to 6 or by applying a certain change therefrom, but the form of implementation is not necessarily limited to that exemplified, and any form of encoder structure capable of functioning as a video encoder should be considered an encoder established by the present invention as long as it embodies the technical concept of the present invention.

[0251] In addition, the decoding method of the encoded result according to the present invention described above can be implemented through a decoder as a device. The decoder as a device may be implemented in a form that maintains the conventional decoder structure exemplified in FIGS. 1 to 6 or applies a certain change therefrom, but the form of implementation is not necessarily limited to that exemplified, and any form of decoder structure capable of functioning as a video decoder should be considered a decoder established by the present invention as long as it embodies the technical concept of the present invention.

[0252] A person skilled in the art will readily understand that a bit sequence encoded by the method and apparatus described above can be decoded by applying a method symmetric and / or in reverse order to the encoding method. In one embodiment, when reading information for decoding from the encoded bit sequence, at least one variable-length coded phrase included in the encoded bit sequence may be interpreted, and in one embodiment, the variable-length coding may be performed by an entropy coding method. The technical details and application methods of implementing such a decoding procedure will be readily understood from the encoding procedure described above.

[0253] The encoder and / or decoder described herein may correspond to a device comprising a processor and memory, preferably implemented as a computing device. The processor that may be included in the encoder and / or decoder described herein may mean one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions.

[0254] Even if the above processor is expressed in the singular for ease of understanding, a person skilled in the art will understand that the above processor may include a plurality of processing elements and / or a plurality of types of processing elements. For example, an apparatus according to one embodiment of the present invention may include a plurality of processors or one processor and one controller as the processor. In addition, the processor may be implemented by various processing configurations, such as a parallel processor or a multi-core processor.

[0255] The processor may be configured to execute an operating system (OS) and one or more software executed on the operating system. Additionally, the processor may access, store, manipulate, process, and generate data in response to the execution of the software.

[0256] The software may include a computer program, code, instructions, or a combination of one or more of these, and may be configured to control the processor to operate as desired and to issue instructions to the processor independently or collectively. The software may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processor or to provide instructions or data to the processor. The software may be distributed over networked computer systems and may be stored or executed in a distributed manner.

[0257] The software described above may also be implemented in the form of program instructions that can be executed through various computer means and may be recorded or stored in the memory. The memory may be a computer-readable recording medium, and program instructions, data files, data structures, etc., may be recorded on the computer-readable recording medium alone or in combination. The program instructions stored in the memory may be based on a command system specifically designed and configured for embodiments of the present invention, or may follow a command system known and available to those skilled in the art of computer software, such as Assembly, C, C++, Java, Python, etc. It should be understood that the command system and the program instructions derived therefrom include not only machine code such as that generated by a compiler, but also high-level language code that can be executed by a device and / or processor according to an embodiment of the present invention using an interpreter, etc.

[0258] A computer-readable recording medium constituting an device according to an embodiment of the present invention, including the memory described herein, may include a temporary or volatile recording medium that is maintained only while the processor is operating, such as a processor cache, RAM, or flash memory; or may include a relatively non-volatile or long-term recording medium such as a magnetic media such as a hard disk, floppy disk, and magnetic tape; an optical recording medium such as a CD-ROM or DVD; a magneto-optical media such as a floptical disk; or a solid-state memory; or may include a read-only recording medium such as a ROM placed on hardware; furthermore, the hardware itself, configured to perform operations equivalent to a series of program instructions by a hard-wired structure by circuit wiring, may also be considered as having each step for performing the operation implementing the embodiment of the present invention recorded by the connection and arrangement of the hardware components, so the method of connection and arrangement is equivalent to the memory. It is obvious to an ordinary technician that it can be seen.

[0259] The embodiments described above with respect to the processor and the memory are not mutually exclusive and may be selected or combined as needed. For example, a hardware device may be configured to operate as a module composed of one or more of the software to perform the operation of an embodiment of the present invention, and vice versa. As another example, in this specification, all or part of the operation assigned to a certain functional unit may be implemented by one or more of the software stored in a device according to an embodiment of the present invention (preferably in a recording medium falling within the category of the memory) and configured to be executed by the processor, in which case such a functional unit may be referred to as a functional unit "included" in the processor.

[0260]

[0261]

[0262] Although the present invention has been described above with reference to the drawings and embodiments, as previously stated, the scope of protection of the present invention is not limited by the drawings or embodiments presented above, and those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the invention as described in the claims of the present invention.

Claims

1. In a video decoding method, A step of obtaining in-screen prediction mode information of the current color block from a bit sequence; A step of identifying illumination blocks at a plurality of positions corresponding to the current color block, wherein the plurality of positions include at least one of the center position, upper-left position, upper-right position, lower-left position, and lower-right position of the current color block; A step of obtaining an in-screen prediction mode from at least one of the identified illuminance blocks and including it in a list of in-screen prediction mode candidates for the current color block; A step of obtaining an in-screen prediction mode from at least one color block spatially adjacent to the current color block and including it in the candidate list; A step of determining an in-screen prediction mode of the current color block based on the above-configured candidate list and the above-acquired in-screen prediction mode information; and A video decoding method comprising the step of generating a prediction block for the current color block using the above-determined in-screen prediction mode.

2. In Paragraph 1, The step of identifying illumination blocks at the plurality of locations above is, An image decoding method characterized by being performed when the above image is divided into a dual tree structure.

3. In Paragraph 2, Acquiring an in-screen prediction mode from the above illumination blocks is, An image decoding method characterized by applying priority in the order of center position, top-left position, top-right position, bottom-left position, and bottom-right position, and checking the next position when the illumination block of the previous position is not available.

4. In Paragraph 3, The in-screen prediction mode of the illuminance block included in the above candidate list is, A video decoding method characterized by being limited to a maximum of five different intra-frame prediction modes.

5. In Paragraph 1, A video decoding method further comprising the step of obtaining an in-frame prediction mode from non-adjacent color blocks that are not spatially adjacent to the current color block and including it in the candidate list.

6. In Paragraph 5, The step of obtaining an in-screen prediction mode from the above non-adjacent color blocks is: An image decoding method characterized by prioritizing the most frequently used in-frame prediction mode and applying a weight inversely proportional to the distance from the current color block.

7. In Paragraph 1, A video decoding method further comprising the step of including a recently used in-frame prediction mode stored in a buffer of a FIFO (First In First Out) structure in the candidate list.

8. In Paragraph 1, The above adjacent color blocks are, A video decoding method characterized by including at least one of the positions left A1, left A0, upper B1, upper B0, and upper-left B2, and checking the availability of each position to obtain only the in-frame prediction mode of the available block.

9. In Paragraph 1, Constituting the above candidate list is, A video decoding method characterized by excluding modes that overlap with already included modes and limiting the maximum number of candidate lists to one of 8, 16, or 32.

10. In Paragraph 1, The prediction mode information within the screen above is, An image decoding method characterized by parsing as an index for the above-mentioned candidate list, decoding the first bin of the index as a context model and the remaining bins as a bypass mode.

11. In a video encoding method, A step of identifying illumination blocks at a plurality of positions corresponding to a current color block, wherein the plurality of positions include at least one of the center position, upper-left position, upper-right position, lower-left position, and lower-right position of the current color block; A step of obtaining an in-screen prediction mode from at least one of the identified illuminance blocks and including it in a list of in-screen prediction mode candidates for the current color block; A step of obtaining an in-screen prediction mode from at least one color block spatially adjacent to the current color block and including it in the candidate list; A step of selecting an in-screen prediction mode for the current color block from the above-configured candidate list; A step of generating a prediction block for the current color block using the prediction mode within the selected screen; and A video encoding method comprising the step of encoding information representing a prediction mode within the selected screen into a bit sequence.

12. In Paragraph 11, The step of identifying illumination blocks at the plurality of locations above is, An image encoding method characterized by being performed when the above image is divided into a double tree structure.

13. In Paragraph 11, A video encoding method further comprising the step of obtaining an in-frame prediction mode from surrounding color blocks that are not spatially adjacent to the current color block and including it in the candidate list.

14. In Paragraph 11, The step of selecting a prediction mode within the above screen is, An image encoding method characterized by selecting an optimal mode from the above candidate list through rate-distortion optimization.

15. In Paragraph 11, A video encoding method further comprising the step of reordering the above-described candidate list based on the Sum of Absolute Transformed Differences (SATD) value.

16. In a video decoding device, A parser configured to obtain in-screen prediction mode information of the current color block from a bit sequence; A candidate list configuration unit configured to identify illumination blocks at multiple locations corresponding to the current color block, obtain an in-screen prediction mode from at least one of the identified illumination blocks and include it in an in-screen prediction mode candidate list, and obtain an in-screen prediction mode from at least one color block spatially adjacent to the current color block and include it in the candidate list; A mode determination unit configured to determine the in-screen prediction mode of the current color block based on the above-configured candidate list and the above-acquired in-screen prediction mode information; and Including an in-screen prediction unit configured to generate a prediction block for the current color block using the in-screen prediction mode determined above; An image decoding device comprising at least one of the above plurality of positions, the center position, the top-left position, the top-right position, the bottom-left position, and the bottom-right position of the current color block.

17. In Paragraph 16, The above candidate list composition section is, An image decoding device characterized by identifying illumination blocks at multiple locations when the above image is divided into a double tree structure.

18. In Paragraph 16, The above candidate list composition section is, An image decoding device characterized by being further configured to acquire an in-frame prediction mode from surrounding color blocks that are not spatially adjacent to the current color block and include it in the candidate list.

19. In Paragraph 16, It further includes a FIFO buffer section that stores the recently used in-screen prediction mode, and An image decoding device characterized in that the above candidate list configuration unit obtains an in-frame prediction mode from the above FIFO buffer unit and includes it in the above candidate list.

20. In Paragraph 16, The above candidate list composition section is, An image decoding device characterized by being configured to limit the maximum number of candidate lists while excluding modes that overlap with already included modes.

Images