Method and apparatus for video encoding and decoding using improved intra prediction structure

HK40137494APending Publication Date: 2026-09-11JIAWEN GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK62026126835
Authority / Receiving Office
HK · HK
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-10-24
Filing Date
2026-07-29
Publication Date
2026-09-11

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
  • Figure 00000000_0001_ABST
    Figure 00000000_0001_ABST
Patent Text Reader

Abstract

The present invention provides a video encoding and decoding method and apparatus using an improved intra prediction structure. A template matching-based decoding method performed by a decoder according to an embodiment of the present invention may comprise: a step of acquiring information on a template, which is a set of samples adjacent to a current block, from a restoration region of a current image; executing a template matching step of searching for a prediction template corresponding to the template in a search region included in the restoration region; selecting a prediction block adjacent to the prediction template; and performing intra prediction decoding of the current block on the basis of the prediction block, in which the template matching is performed by using a partial sample within the template or a part of the search region.
Need to check novelty before this filing date? Find Prior Art

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202480047701.2 (22) Application Date 2024.06.28 (30) Priority Data 10-2023-0094036 2023.07.19 KR 10-2023-0142997 2023.10.24 KR (85) PCT International Application Entering National Phase Date 2026.01.16 (86) PCT International Application Application Data PCT / KR2024 / 009075 2024.06.28 (87) PCT International Application Publication Data WO2025 / 018639 KO 2025.01.23 (71) Applicant: Jiawen Group Co., Ltd. Address: South Korea Applicant: Sejong University Industry-Academia Collaboration Group (72) Inventors: Yoo Chang-woo, Lee Jin-young (74) Patent Agency: Beijing Pinyuan Patent Agency Co., Ltd. 11332 Patent Attorneys: Lü Lin, Tian Ying-ai (51) Int.Cl. H04N 19 / 105 (2006.01) H04N 19 / 132 (2006.01) H04N 19 / 593 (2006.01) H04N 19 / 176 (2006.01) H04N 19 / 70 (2006.01) H04N 19 / 186 (2006.01) (54) Invention Title: A video coding and decoding method and apparatus using an improved intra-frame prediction structure (57) Abstract: This invention provides a video coding and decoding method and apparatus using an improved intra-frame prediction structure. A template-matching-based decoding method performed by a decoder according to an embodiment of the present invention may include: obtaining information about a template, which is a set of samples adjacent to the current block, from a restored region of the current image; performing a template matching step within a search region contained in the restored region to search for a prediction template corresponding to the template; selecting a prediction block adjacent to the prediction template; and performing intra-frame prediction decoding of the current block based on the prediction block, wherein the template matching is performed by using a portion of the samples within the template or by using a portion of the search region.Claims 3 pages, Description 18 pages, Drawings 12 pages, CN 121533011 A 2026.02.13 CN 1 21 53 30 11 A 1. A template-matching-based decoding method performed by a decoder, characterized in that it includes: obtaining information about a template as a set of samples adjacent to the current block from a restored region of the current image; performing a template matching step within a search region contained in the restored region to search for a predicted template corresponding to the template; selecting a predicted block adjacent to the predicted template; and performing intra-frame prediction decoding of the current block based on the predicted block, wherein the template matching is performed by using a portion of samples within the template or by using a portion of the search region. 2. The method according to claim 1, characterized in that the step of performing the template matching includes: downsampling the template; and searching for the predicted template location by comparing the samples of the downsampled template with the samples of the search region. 3. The method according to claim 2, wherein the downsampling step comprises: adjusting at least one of the execution of downsampling and the scaling factor for the upper sample of the template based on the horizontal size of the current block. 4. The method according to claim 2, wherein the downsampling step comprises: adjusting at least one of the execution of downsampling and the scaling factor for the left sample of the template based on the vertical size of the current block. 5. The method according to any one of claims 3 or 4, wherein the downsampling step is performed based on downsampling symbol information derived from the bitstream. 6. The method according to claim 1, wherein the search region is defined based on a prediction vector from the current block. 7. The method according to claim 6, wherein the prediction vector is generated based on at least one reference prediction vector information of the surrounding blocks of the current block. 8. The method according to claim 7, wherein the search region, centered on the point indicated by the prediction vector, has a first sample width in the horizontal direction and a second sample width in the vertical direction. 9. The method according to claim 8, wherein at least one of the first sample width and the second sample width is derived from the bitstream. 10. The method according to claim 6, wherein the prediction vector is selected by an index from at least one of the block vector candidates of the surrounding blocks of the current block. (Claims 1 / 3 page 2 CN 121533011 A) 11. The method according to claim 10, wherein the index is derived from the bitstream.12. The method of claim 6, wherein the prediction vector is generated based on differential vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the reference prediction information. 13. The method of claim 6, further comprising: comparing the current block sample with samples of a search region contained in the restored region of the image to search for the best prediction block position closest to the current block sample, wherein the prediction vector is generated based on differential vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the best prediction vector information indicating the best prediction block position. 14. The method of any one of claims 12 or 13, wherein the differential vector information is derived from a bitstream. 15. The method of claim 14, wherein the differential vector information is specified by an index indicating any one of at least two preset values. 16. The method of claim 6, further comprising: storing the prediction vector through the decoder for use in a subsequent decoding process. 17. The method of claim 6, further comprising: generating the prediction vectors for at least one lower-level decoding unit; designating one of the at least one prediction vectors as a representative prediction vector for an upper-level decoding unit including the lower-level decoding unit; and storing the representative prediction vectors by the decoder. 18. The method of claim 1, further comprising: generating a prediction vector based on at least one reference prediction vector corresponding to a block at the same position in the luminance signal region, with reference to the chrominance signal region, when performing template matching in a chrominance signal region. 19. A decoder apparatus, comprising: a template matching unit for obtaining information about a template as a set of samples adjacent to the current block from a restored region of a current image, and performing template matching within a search region contained in the restored region to search for a prediction template corresponding to the template; and a prediction decoding unit for selecting a prediction block adjacent to the prediction template and performing intra-frame prediction decoding of the current block based on the prediction block, wherein the template matching is performed by using a portion of samples within the template or a portion of the search region.20. An encoder apparatus, characterized in that it comprises: a template matching unit, configured to obtain information about a template as a set of samples adjacent to the current block from an encoded region of a current image, and to perform template matching for searching a predicted template corresponding to the template within a search region contained in the encoded region; and a predictive coding unit, configured to select a predicted block adjacent to the predicted template, and to perform intra-frame predictive coding of the current block based on the predicted block, wherein the template matching is performed by using a portion of the samples within the template or by using a portion of the search region. Claims 3 / 3 page 4 CN 121533011 A A Video Encoding and Decoding Method and Apparatus Utilizing an Improved Intra-Frame Prediction Structure Technical Field

[0001] This invention relates to the field of encoding and decoding of digital video, and more specifically, to methods for encoding and decoding digital video, methods for recording such data, and components, apparatus, and systems for implementing such methods.

[0002] This invention may be in the same technical field as at least one known digital video compression technology standard (e.g., standard names include MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AVI, QuickTime, VP-9, VP-10, and Motion JPEG), or may be used to improve the inherent efficiency of such standards, or may be used to improve or replace such standards. Background Art

[0003] Digital video encoding and decoding are widely used in various digital video application fields. For example, digital television broadcasting, video transmission via communication networks, video calls, video conversations, video chats, video content recorded and played using optical discs (including VCDs (video compact discs) / DVDs (digital versatile discs) / Blu-Ray discs), the entire process of video content production, editing, collection and distribution, and video shooting devices and cameras used in video shooting and recording activities (including personal, commercial, industrial and security monitoring purposes) all rely on video encoding and decoding technologies.

[0004] Therefore, physical devices that can be called digital video encoders and decoders can constitute part of a wide range of devices related to the generation, recording, and playback of digital video, including digital television, digital broadcasting systems, wireless broadcasting systems, laptops, desktop computers, tablets, e-book readers, digital cameras, digital video recording devices, digital multimedia playback devices, video game devices / terminals / control consoles, mobile phones (including smartphones) with multimedia playback capabilities, devices for video conferencing, and others.

[0005] The aforementioned digital video encoders and decoders can be implemented using digital video compression standards widely used and understood by those skilled in the art. These digital video compression standards may include at least one known compression standard, such as those named MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AVI, QuickTime, VP-9, VP-10, and Motion JPEG.

[0006] Video encoders and decoders can more efficiently encode or decode digital video information by conforming to the aforementioned standards, or by improving or modifying the aforementioned standards. Attempts to modify the aforementioned standards may also lead to new standards. In a well-known example, there exists an enhanced compression model (ECM), developed by the Joint Video Experts Team (JVET), a joint international standardization organization comprised of ISO, IEC, and ITU-T, aimed at improving and replacing the existing H.266 / VVC standard.

[0007] The encoding process for compressing digital video requires various operations, such as: spatial segmentation of digital video, segmentation and / or processing of color channels, removal of spatial redundancy, removal of temporal redundancy, tracking of motion vectors within the video, encoding of differential images, quantization, coefficient scan, run-length coding, entropy coding, and a series of processes including loop filtering. These encoding operations typically consume computational resources and require a certain amount of time to complete. Similarly, the decoding operations for the aforementioned encoding operations also require certain computational resources and a certain amount of time. The primary goal of video encoding and decoding technologies is to ensure that the aforementioned resource and time consumption does not affect the production, recording, distribution, and viewing of digital video.Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] The technical problem of the present invention is to provide a new technology that can play a role in at least one of the following aspects: improving encoding efficiency, improving decoding efficiency, improving video quality, reducing computational load, reducing software size, reducing hardware size, and improving other performance related to encoding and decoding.

[0010] Solution to the Problem

[0011] To solve the above technical problem, a template matching-based decoding method performed by a decoder according to an embodiment of the present invention may include: obtaining information about a template as a set of samples adjacent to the current block from a restored region of the current image; performing a template matching search for a prediction template corresponding to the template within a search region contained in the restored region; selecting a prediction block adjacent to the prediction template; and performing intra-frame prediction decoding of the current block based on the prediction block, wherein the template matching is performed by using a portion of the samples within the template or by using a portion of the search region.

[0012] The steps of performing the above template matching may include: downsampling the template; and searching for the predicted template position by comparing the sample of the downsampled template with the sample of the search region.

[0013] The downsampling step may include: adjusting at least one of the downsampling execution and scaling factor for the upper sample of the template based on the horizontal size of the current block.

[0014] The downsampling step may include: adjusting at least one of the downsampling execution and scaling factor for the left sample of the template based on the vertical size of the current block.

[0015] The downsampling step may be performed based on downsampling symbol information derived from the bitstream.

[0016] The search region may be defined based on the prediction vector from the current block.

[0017] The prediction vector may be generated based on at least one reference prediction vector information of the surrounding blocks of the current block.

[0018] The search region can have a first sample width in the horizontal direction and a second sample width in the vertical direction, centered on the point indicated by the prediction vector.

[0019] At least one of the first sample width and the second sample width can be derived from the bit stream.

[0020] The prediction vector can be selected from at least one of the block vector candidates of the surrounding blocks of the current block by means of an index.

[0021] The index can be derived from the bit stream.

[0022] The prediction vector can be generated based on differential vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the reference prediction information.

[0023] The method may further include: comparing the current block sample with a sample of the search area contained in the restored area of ​​the image to search for the best prediction block position closest to the current block sample, wherein the prediction vector can be generated based on differential vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the best prediction vector information indicating the best prediction block position. Specification 2 / 18 page 6 CN 121533011 A

[0024] The differential vector information can be derived from the bitstream.

[0025] The differential vector information can be specified by an index indicating any one of at least two preset values.

[0026] The method may further include: storing the prediction vector through the decoder for use in a subsequent decoding process.

[0027] The above method may further include: generating the prediction vectors for at least one lower-level decoding unit; designating one of the at least one prediction vectors as a representative prediction vector for an upper-level decoding unit containing the lower-level decoding unit; and storing the representative prediction vectors in the decoder.

[0028] The above method may further include: when performing the template matching in the chroma signal region, generating a prediction vector based on at least one reference prediction vector corresponding to a block at the same position in the luminance signal region with the chroma signal region as a reference.

[0029] To solve the above-mentioned technical problems, a decoder apparatus according to an embodiment of the present invention may include: a template matching unit, configured to obtain information about a template, which is a set of samples adjacent to the current block, from a restored region of the current image, and perform template matching to search for a prediction template corresponding to the template within a search region included in the restored region; and a prediction decoding unit, configured to select a prediction block adjacent to the prediction template, and perform intra-frame prediction decoding of the current block based on the prediction block, wherein the template matching is performed by using a portion of the samples within the template or by using a portion of the search region.

[0030] To solve the above-mentioned technical problems, a template matching-based encoding method performed by an encoder according to an embodiment of the present invention may include: a step of obtaining information about a template, which is a set of samples adjacent to the current block, from an encoded region of the current image; a step of performing a template matching search for a prediction template corresponding to the template within a search region contained in the encoded region; a step of selecting a prediction block adjacent to the prediction template; and a step of performing intra-frame prediction encoding of the current block based on the prediction block, wherein the template matching is performed by using a portion of the samples within the template or by using a portion of the search region.

[0031] To solve the above-mentioned technical problems, an encoder device according to an embodiment of the present invention may include: a template matching unit, configured to obtain information about a template as a set of samples adjacent to the current block from the encoded region of the current image, and perform template matching to search for a prediction template corresponding to the template within a search region contained in the encoded region; and a prediction coding unit, configured to select a prediction block adjacent to the prediction template, and perform intra-frame prediction coding of the current block based on the prediction block, wherein the template matching is performed by using some samples within the template or by using a portion of the search region.

[0032] Effects of the Invention

[0033] According to the present invention, by reducing the complexity of template matching for performing intra-frame prediction in the video encoder and decoder, efficient and high-speed video encoding and decoding can be achieved. Brief Description of the Drawings

[0034] FIG1 is a conceptual diagram of a video communication system according to an embodiment of the present invention.

[0035] FIG2 is a conceptual diagram of the arrangement of an encoder and decoder in a real-time video streaming environment according to an embodiment of the present invention.

[0036] FIG3 is a conceptual diagram of the functional units of a video decoder according to an embodiment of the present invention.

[0037] FIG4 is a conceptual diagram of the functional units of a video encoder according to an embodiment of the present invention.

[0038] FIG5 is a conceptual diagram of a frame type according to an embodiment of the present invention. Specification 3 / 18 pages 7 CN 121533011 A

[0039] FIG6 is a conceptual diagram showing the structure of a video encoder according to another embodiment of the present invention.

[0040] FIG7 is a conceptual diagram showing a template matching prediction method according to an embodiment.

[0041] FIG8 is a conceptual diagram concerning the storage and use of block vectors in the template matching prediction method.

[0042] FIG9 is an example diagram concerning template downsampling according to an embodiment of the present invention.

[0043] FIG10 is an example diagram concerning template downsampling based on block size according to an embodiment of the present invention.

[0044] FIG11 is an example diagram of the search range setting of block vectors according to an embodiment of the present invention.

[0045] FIG12 is a conceptual diagram of a block vector determination method based on a predicted vector according to an embodiment of the present invention. Detailed Description

[0046] The present invention can be modified in various ways and has various embodiments, and specific embodiments are illustrated in the accompanying drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, but should be understood as including all modifications, equivalents and even substitutions that fall within the scope of the ideas and techniques of the present invention.

[0047] Although terms such as “first” and “second” may be used to describe various components, the above components should not be limited by the above terms. The purpose of using the above terms is only to distinguish one component from another component. For example, without departing from the protection scope of the present invention, a first component may be named a second component, and similarly, a second component may be named a first component. The term “and / or” includes a combination of multiple related described items or one of multiple related described items, and is not exclusive unless otherwise stated. The items listed in this application are merely exemplary descriptions intended to facilitate understanding of the concepts and possible implementations of the present invention, and are therefore not intended to limit the scope of the embodiments of the present invention.

[0048] The term “A or B” in this specification can mean “A only”, “B only”, or “A and B”. In other words, “A or B” in this specification can also be interpreted as “A and / or B”. For example, “A, B or C” in this specification can mean “A only”, “B only”, “C only”, or “any combination of A, B and C”.

[0049] The forward slash ( / ) or comma used in this specification can mean “and / or”. For example, “A / B” can mean “A and / or B”. Accordingly, “A / B” can mean “A only”, “B only”, or “A and B”. For example, “A, B, C” can mean “A, B or C”.

[0050] The phrase “at least one of A and B” in this specification may mean “only A”, “only B”, or “A and B”. Furthermore, the expressions “at least one of A or B” or “at least one of A and / or B” in this specification may be interpreted equivalently as “at least one of A and B”.

[0051] Additionally, in this specification, "at least one of A, B and C" can mean "only A," "only B," "only C," or "any combination of all A, B, and C." Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C."

[0052] When a component is described as "connected" or "accessed" to another component, it should be understood that it may be directly connected to or accessed to another component, or there may be other components in between. Conversely, when a component is described as "directly connected" or "directly accessed" to another component, it should be understood that there are no other components in between.

[0053] The terminology used in this application is merely for illustrating specific embodiments and is not intended to limit the invention. Unless explicitly stated otherwise in the context of the article, singular expressions include plural expressions. In this application, terms such as "comprising" or "having" should be understood as indicating that the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification are present, rather than precluding the existence or additional possibility of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0054] Unless otherwise specified, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms that are commonly used and are identical to those defined in dictionaries have the same meaning as in the context of the related art, and do not have an ideal or excessive meaning in this application unless explicitly defined.

[0055] When describing the invention in this application, embodiments may be described or illustrated in the form of unit blocks that perform the described functions or functions. In this application, such blocks may represent one or more devices, units, modules, parts, etc. Such blocks may be implemented in hardware by one or more logic gates, integrated circuits, processors, controllers, memories, electronic components, or information processing hardware implementation methods including but not limited to these. Alternatively, the aforementioned blocks can be implemented in software using application software, operating system software, firmware, or information processing software methods including but not limited to these. A block can be divided into multiple blocks that perform the same function, and conversely, the function of implementing multiple blocks simultaneously can also be executed by a single block. The aforementioned blocks can also be implemented as physically separate or combined according to any standard.The above-mentioned blocks can also be implemented to operate in an environment where the physical location of the communication method is not specified and they are isolated from each other through a communication network, the Internet, cloud services or other communication methods including but not limited to these. All of the above implementation methods fall within the scope of various embodiments that can be adopted by those skilled in the art of information and communication to achieve the same technical idea. Therefore, any specific implementation method should be interpreted as falling within the scope of the inventive technical idea of ​​this application.

[0056] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. For ease of overall understanding of the present invention, the same components in the drawings will be represented by the same reference numerals, and repeated descriptions of the same components will be omitted. In addition, multiple embodiments are not mutually exclusive, and, provided that some embodiments can be combined with one or more other embodiments to form new embodiments.

[0057] Digital video codec

[0058] FIG1 is a conceptual diagram of a video communication system according to an embodiment of the present invention. The above-mentioned video communication system 100 can be configured to include at least two terminals 110, 120 interconnected through a network 105.

[0059] In one embodiment of the present invention, FIG1 can represent a block diagram for configuring a unidirectional video communication network. In order to send video data 111 via network 105, the first terminal 110 of the above-mentioned terminals can encode the video data. The second terminal 120 of the above-mentioned terminals can be configured to receive the encoded video data 121 via the network, and then decode and display it.

[0060] In another embodiment of the present invention, FIG1 can be a block diagram for configuring a two-way video communication network. In order to realize the two-way video communication, each of the above-mentioned terminals 110 and 120 can be configured to encode the video data they acquire, so as to transmit the video 112 and 122 to other terminals via the network. Each terminal can also be configured to receive video data sent by other terminals via the network 113 and 123 for decoding, and display the decoded video data.

[0061] The above-mentioned terminals 110 and 120 shown in FIG1 can be, for example, server computers, personal computers, portable computers and smartphones, etc., according to the embodiments, but are not limited thereto. This invention is applicable to any environment for constructing one-way or two-way video communication networks. It is understood that the network 105 can be configured to transmit encoded video data between the terminals 110 and 120.

[0062] In one embodiment of the invention, the network 105 may represent a wired or wireless communication network. In this case, according to the embodiment, the network can be configured to communicate information using any communication standard, including packet-based communication. Packet communication can be understood to include, for example, packets referred to as TCP or UDP.

[0063] However, in another embodiment of the present invention, the network 105 described above may also be understood to include a process of transmitting information using a recording medium. In this case, the configuration of the network described above is not limited to communication media, but should be understood to include a process of temporary storage and physical transmission in hard disks, solid-state drives (SSDs), flash memory, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs, and other mechanical, electronic, or optical recording media.

[0064] Furthermore, regardless of the information communication or transmission means used, as long as it supports the transmission and decoding of video data in an encoded state, it can be considered to fall within the scope of the embodiments of the present invention. Therefore, except for some of the examples listed above, all known or future information communication or transmission means can fall within the scope of the present invention.

[0065] FIG2 is a conceptual diagram of the arrangement of encoders and decoders in a real-time video streaming environment according to an embodiment of the present invention. The streaming media system 200 shown in FIG2 can be applied to video data communication networks, for example, including digital broadcasting, video telephony, and video conferencing. However, even when information transmission is involved via a recording medium, the same or similar technical structures as described above can be applied to the streaming media system.

[0066] According to an embodiment of the present invention, the streaming media system may include a video source 210 that generates the video stream. The video source may include a digital video acquisition unit 212 for acquiring uncompressed raw video. For example, the digital video acquisition unit may be configured as a digital camera or other device. The raw video stream 215 may have a large capacity and can therefore be compressed by combining it with or connected to the video encoder 217 of the video source.

[0067] The encoder 217 may be configured as a unit including hardware, software, or a combination of both, for implementing a video encoding method and / or its implementation method according to an embodiment of the present invention.

[0068] Through the encoder 217, an encoded bitstream 219 with a reduced capacity compared to the raw video stream can be output. The bitstream 219 may be provided in real time for communication via a relay device (e.g., streaming media server 220) and / or may be stored on the recording medium 225 of the streaming media server 220 for subsequent use.

[0069] The above-mentioned streaming media system 200 may include at least one streaming media client 230, 240 that accesses the above-mentioned streaming media server 220 to receive in real time or acquire the above-mentioned encoded bitstream 229 later.The aforementioned streaming media client may include a video decoder 232, which acquires the encoded bitstream 229 (which can also be considered a copy of the bitstream 219 received by the aforementioned streaming media server), decodes the bitstream 229, and outputs the resulting video data as video data that can be displayed on a display 235 or other visual, auditory, or other sensory display means.

[0070] As described above, the encoding and decoding functions of video data are collectively referred to as a coder-and-decoder system, i.e., a video codec.

[0071] FIG3 is a conceptual diagram of the functional unit of a video decoder according to an embodiment of the present invention. As shown in FIG3, the receiving unit 310 may receive at least one encoded video data to be decoded by the decoder 305. In one embodiment of the present invention, each received encoded video data may be independent, and the decoding process of each independent video data may be independent of the decoding process of other video data. The encoded video data may be received to the receiving unit 310 through a hardware or software connection 315 between the receiving unit and the storage device. As described above, the storage device may refer to a streaming media server located at the other end of a communication network, or it may be a physical storage medium, but is not limited thereto.

[0072] The receiving unit 310 can receive the encoded video data and other accompanying data, such as encoded audio data or other auxiliary data, and each of these data can be separated from the video data and provided to other appropriate processing units 312 other than the video decoder.

[0073] When the video data is received through a communication network, a buffer memory 320 can be connected between the receiving unit 310 and the decoder 305 to minimize latency and interruptions caused by the network environment. The buffer memory 320 (see page 6 / 18 of specification, CN 121533011 A) can be a computer-readable recording medium used to temporarily store the received video data and stably provide it to the parser 330 corresponding to the input of the decoder 305. However, if the bandwidth of the communication network is sufficient, or the video data is read from a local and non-physically isolated recording medium, or if no communication latency is expected in other environments, the buffer memory may not be necessary.

[0074] The video decoder 305 may include the parser 330 as its input terminal for parsing the encoded video data.The parser described above can separate (parsing) various information stored in bitstream form in the encoded video data according to predetermined rules, and perform entropy decoding 335 on the entropy-coded video data when needed, thereby performing the function of reconstructing symbols 338 as segments of video encoded information. The symbols 338 may include any information for controlling the operation of the decoder 305, and / or may further include information for controlling devices (such as display devices) attached to the decoder 305 that perform the operation. The control information for controlling the display device may include information in the format of supplementary enhancement information (SEI) or video usability information (VUI).

[0075] As described above, the parser 330 can be configured to perform entropy decoding 335 on the encoded video data. The entropy coding method of the encoded video data may vary depending on the encoding standard, and the decoding operation should be performed accordingly. Representative examples of the aforementioned entropy coding standards may include: variable length coding, Huffman coding, and arithmetic coding. The various coding methods described above may be context-adaptive or context-sensitive methods according to the standard, and may also be based on principles well known to those skilled in the art.

[0076] The parser 330 described above may be configured to extract at least one picture from the encoded video data described above. The definition of the picture may vary depending on the aforementioned coding standard. According to the standard, one of the following examples may apply, or multiple examples may be applied simultaneously. For example, the picture may be defined as the following units: group of pictures (GOPs), pictures / frames, tiles, slices, macroblocks, blocks, subblocks, transform units (TUs), prediction units (PUs), etc.

[0077] The parser 330 described above can be configured to extract encoded information from the encoded video data, such as transform coefficients, quantization parameters (QPs), and / or motion vectors. The parser 330 can be configured to perform entropy decoding 335 and parsing operations on the video data received from the buffer memory, and selectively decode symbols 338 representing the encoded information. Furthermore, the parser 330 can also be configured to selectively provide specific symbols 338 to specific decoding function units within the decoder 305, such as inverse quantization and inverse transform units 340, intra prediction units 350, inter prediction units 355, or loop filter units 360. The control of such information provision can be determined by the arrangement of information contained in the encoded video, and may vary depending on the encoding standard, and is not limited to the scope of the embodiments of the present invention, and is not described in detail in this conceptual diagram.

[0078] The decoder 305 described above may be configured with multiple conceptual functional units for receiving and processing the encoded information from the parser 330 described above. It should be understood that these conceptual functional units may be combined or further subdivided according to implementation needs. For example, they may be further separated for ease of implementation; they may be integrated together to improve operating efficiency. In any case, each functional unit may be configured to interact closely with each other. Despite the possibility of such integration or separation, the following description will still be based on the combination of conceptual functional units to clarify the decoding process of video data applied as an embodiment of the present invention. Specification 7 / 18 pages 11 CN 121533011 A

[0079] The decoder described above may include an inverse quantization and inverse transform unit 340. The aforementioned inverse quantization and inverse transform unit 340 can be configured to receive encoded information from the aforementioned parser 330, the encoded information including: a method for numerical transformation, block size, quantization coefficients for recovering quantized information, and distinguishing information for simplifying and representing the quantization coefficients, etc., and can be configured to output block values ​​341 that can be input to the aggregator 370 as a result of processing the aforementioned encoded information.

[0080] In one embodiment of the present invention, the output value of the aforementioned inverse quantization and inverse transform unit 340 may include intra-frame predicted encoded block values.The aforementioned intra-frame prediction block value refers to a value obtained by decoding using the prediction information of the currently being decoded image (e.g., within the current frame) without using previously decoded images (e.g., prediction information from the previous frame).

[0081] The prediction information in the current image can be provided by the intra-frame prediction unit 350. According to an embodiment of the present invention, the intra-frame prediction unit 350 uses image information of spatially adjacent regions extracted from the currently being decoded and partially decoded image to generate block values ​​with the same block form as the being decoded as prediction information. The aforementioned image information can be provided 381 by the current image buffer (i.e., the so-called line buffer 380). According to an embodiment, the aforementioned merging unit 370 can be configured to merge the prediction information 351 generated by the aforementioned intra-frame prediction unit 350 with the block values ​​341 provided by the aforementioned inverse quantization and inverse transform unit 340.

[0082] In another embodiment, the output value of the aforementioned inverse quantization and inverse transform unit 340 can be the inter-frame prediction encoded block value, and in some cases, the aforementioned output value can include block values ​​after motion compensation. In this case, the inter-frame prediction unit 355 can extract and use sample information 386 for motion-based prediction from the reference picture buffer 385. The information 356 obtained by performing motion compensation on the sample information based on the symbols 338 contained in the block value as the output value can be configured to be merged by the merging unit 370 with the block value 341 provided by the inverse quantization and inverse transform unit 340. In this case, the block value 341 can be referred to as a differential value or a residual value.

[0083] The position information in the memory, used by the inter-frame prediction unit 355 to extract the sample information from the reference image, can be determined, for example, by a combination of X, Y and other symbols 338 used to indicate a specific position in the reference image and by a motion vector transmitted to the inter-frame prediction unit 355. The inter-frame prediction unit 355 may also include the following functions: when providing motion vectors that support "subsampling", it can process the sample values ​​by using interpolation; and it can predict and enhance the motion vector values.

[0084] The output value 371 of the merging unit 370 may be provided to the loop filter unit 360 and processed by various loop filtering methods. The loop filter unit 360 may also be configured to receive not only the block unit output 371 of the merging unit 370, but also the symbols 338 from the parser 330 to control its operation.The output of the aforementioned loop filtering unit 360 can be transmitted to an external display device, such as the aforementioned display device, via the output connection 390. However, for the purpose of predicting and parsing the values ​​of subsequent intra-frame or inter-frame coded blocks, it can be stored 361 in the line buffer 380, and then stored in the reference image buffer 385.

[0085] After a specific image (e.g., a frame) is decoded, it can be used as a reference image for predictive decoding in subsequent decoding processes. An image (e.g., a frame) can be gradually accumulated into the line buffer 380 and decoded. When the decoding of a frame is completed, the contents of the aforementioned line buffer 380 can be transferred 383 to the aforementioned reference image buffer 385, and a new line buffer 380 is allocated for decoding new frames.

[0086] The aforementioned video decoder 305 can be configured to perform decoding operations according to a predetermined video compression technology, which can be documented based on various international standards or commercial standards as described in the specification page 8 / 18, 12 CN 121533011 A. The aforementioned standards may include, for example, international standard recommendations such as H.264, H.265, and H.266 defined by the International Telecommunication Union Standardization Sector (ITU-T). Those skilled in the art should understand that each of the aforementioned recommendations is equivalent to an international standard jointly developed by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). The encoded video data may conform to a specific bitstream syntax defined by the relevant standard, which is explicitly specified in the video compression specification document and standard document, and is specifically defined and constrained by the profile and level specified therein. To ensure compliance with the aforementioned profile and level, the complexity of the encoded video data also needs to be limited to a certain level. For example, certain profiles or levels may configure limitations such as maximum image size, maximum decoding rate, and maximum reference image size. In some embodiments, the aforementioned limitations may be further restricted by the hypothetical reference decoder (HRD) and the aforementioned HRD buffer management metadata signals included in the encoded video data.

[0087] According to an embodiment of the present invention, the receiving unit 310 may receive additional redundant data along with the encoded video. The aforementioned additional data can be considered as part of the aforementioned encoded video data. The aforementioned additional data may include information for use by the aforementioned decoder 305, which is used to correctly decode the data or more accurately reconstruct an image close to the unencoded image. The aforementioned additional data may be provided in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, and forward error correction codes.

[0088] FIG4 is a conceptual diagram of the functional units of a video encoder according to an embodiment of the present invention.The encoder 405 described above can be configured to receive raw video information 402 from video source 401 and encode it.

[0089] The raw video information 402 described above can have any suitable bit depth, such as 8 bits, 10 bits, 12 bits, etc. In addition, the raw video information 402 described above can have any suitable color space, such as R / G / B, Y / U / V, Y / Cb / Cr, etc. Furthermore, the raw video information 402 described above can have any suitable sampling structure corresponding to the above color space, such as Y / Cb / Cr 4:2:0, Y / Cb / Cr 4:4:4, etc. The raw video information 402 with such a predetermined format can be provided to the encoder described above in the form of a digital video stream.

[0090] In a one-way video communication network, the raw video information 402 described above can be obtained from a recording medium storing a pre-prepared video original. In a two-way video communication network, the aforementioned original video information 402 can be acquired from an image acquisition device (e.g., a camera) that generates at least one video transmission stream included in the two-way video communication.

[0091] The video data containing the aforementioned original video information 402 can be configured to play multiple images in chronological order to simulate motion. The aforementioned images can be represented not only as pictures, but also as concepts such as frames. Depending on the sampling structure and color space used, the aforementioned images may include more than one sample. Those skilled in the art should understand that the aforementioned sample is a term closely related to pixels in digital images. The working principle of the encoder will be explained below in relation to such samples.

[0092] According to an embodiment of the present invention, the encoder 405 can be configured to encode and compress the images (and / or their grouped or segmented information) constituting the aforementioned original video information 402 in real time (or according to other time requirements required for the implementation method) into the form of encoded video information.

[0093] In the aforementioned encoder 405, the control unit 450 can be a functional unit for appropriately controlling the encoding speed. As described below, the control unit 450 can be configured to control other functional units and to combine functions with the functional units described below. The parameters set by the control unit 450 may include parameters related to bitrate control, such as image skipping, quantizer, and variable values ​​for applying image quality optimization techniques. In addition, it may also include values ​​such as image size, group of pictures (GOP) structure, and maximum search range of motion vectors.Those skilled in the art should understand that the control unit 450 may have various other functions, and these other functions can be added or removed according to the video encoder design optimized for individual system design.

[0094] According to an embodiment of the present invention, the encoder 405 may be configured to operate in a structure such as a "coding loop" well known to those skilled in the art. For simplicity, exemplarily, the encoding loop may consist of an internal encoder (also called a "source encoder") 410 and an internal decoder 420, wherein the internal encoder is responsible for receiving the image to be encoded and generating a symbol based on at least one previously encoded reference image, and the internal decoder is configured to be connected to the internal encoder. The internal decoder 420 may be configured to, by receiving the output of the internal encoder 410, perform a reproduction action of the sample data generated after the encoded video information received by the remote decoder 490, which is actually located remotely, from the encoder 405.

[0095] The video data composed of sample data reconstructed by the internal decoder 420 described above can be configured to be input into the reference image buffer of the encoder 405 described above. As described above, since the internal decoder 420 is implemented to reproduce the output of the encoder 405 and the result of decoding by a remote decoder, the video data recorded in the reference image buffer can be identical to the information in the reference image buffer of the remote decoder in bit units. That is, the prediction function unit included in the encoder 405 can read the same value from the reference image buffer of the encoder 405 as the previous frame sample value that the subsequent decoder will reference during the decoding process.

[0096] As described above, the principle of achieving consistency of the reference image buffer between the encoder 405 and the decoder 490 through the internal decoder 420 at the encoder 405 end is well known to those skilled in the art. In addition, those skilled in the art are also well aware of how to deal with the situation where such an environment cannot be guaranteed (e.g., information loss due to communication failure, etc.).

[0097] An embodiment of the operation method of the internal decoder 420 described above has been described in detail with reference to FIG3. The decoder in Figure 3 above can be considered as the "remote" decoder 490 mentioned above. The internal decoder 420 may not include lossless encoding and decoding parts, such as the parser 330 or entropy decoder 335. This is because the implementation of the internal encoder 405 is merely to reproduce the operation of the remote decoder, thus it can directly decode symbols without the need for symbol compression and decompression. Therefore, as shown in Figure 3, the parser and entropy decoder may not possess the previous functional units, or at least partially implement the previous functional units.

[0098] As described above, according to a preferred embodiment of the present invention, any decoder functional unit (excluding the parser and entropy decoder) present in the decoder can naturally exist as substantially the same functional unit in the corresponding encoder 405.

[0099] The operation of the encoding functional unit that may be included in the encoder 405 can be regarded as the inverse process of the decoder functional unit. Therefore, this embodiment can generally be explained by reversing the operation of the decoder functional unit. For example, quantization and transform functional units corresponding to inverse quantization and inverse transform units can be provided, as well as inter-frame prediction coding units corresponding to inter-frame prediction units can be provided. In addition, some additional explanations are provided below.

[0100] The internal encoder 410 described above can be configured to perform encoding for input image information (e.g., input frame) according to the prediction coding method performed by the prediction coding unit 440 operating through the reference image buffer 430, for at least one reference image information, such as a video data designated as a reference frame that has at least one previously encoded image (or frame) in time sequence. In this case, the encoder 405 can be configured to encode the difference between the sample blocks constituting the input image and the sample blocks constituting the reference image.

[0101] The internal decoder 420 can decode video data that can be designated as the reference image from the symbols generated by the internal encoder 410. As described above, since this video data is the same as the decoding action performed by the remote decoder, the video data used as the reference image can be provided to the encoder 405 in a lossy compressed and somewhat corrupted form, which is intended to ensure consistency with the decoder's action. Specification 10 / 18 pages 14 CN 121533011 A

[0102] The prediction coding unit 440 can be configured to perform a prediction search action within the encoder 405. The prediction search action can represent the inter-frame prediction or intra-frame prediction action described in the description of the decoder. For image information that has been input and is planned to be newly encoded, in order to obtain the motion vectors, block shapes, and metadata that may include these, as well as the actual reference sample blocks, of the reference image's positional information that can provide suitable prediction reference information for the new image information, as well as the sample blocks that are actually referenced, the prediction unit can access the reference image buffer 430 and retrieve the information. The prediction encoding unit 440 can operate based on a so-called "sample block by pixel block" benchmark to obtain suitable prediction reference information.According to an embodiment of the present invention, as can be determined based on the search results obtained by the aforementioned predictive coding unit 440, at least one predictive reference information can be specified for the aforementioned input image to point to at least one reference image information stored in the aforementioned reference image buffer 430.

[0103] In an embodiment of the present invention, the aforementioned control unit 450 can be configured to include setting parameters for video data encoding to manage the overall encoding operation of the internal encoder 410.

[0104] The outputs of all the aforementioned functional units can be processed by entropy coding 460 to achieve the final output. For the symbols generated by the aforementioned functional units, the aforementioned entropy coding 460 uses various entropy coding techniques described above (which may include, for example, variable length coding, Huffman coding, and arithmetic coding) for encoding. Depending on different standards, the aforementioned various encoding methods can be context-adaptive or context-sensitive methods, in addition to being implemented based on principles well known to those skilled in the art. Such entropy coding 460 can generally achieve lossless compression, thus, at least one symbol generated by the aforementioned functional units can be converted into encoded video data.

[0105] During the operation of the encoder 405, the control unit 450 can apply a specific encoding type to each image (or frame) within the encoding interval. The encoding method of the image is also affected by the type. According to an embodiment, the type may include the following distinguished "frame type".

[0106] FIG5 is a conceptual diagram of a frame type according to an embodiment of the present invention. The following description will refer to FIG5.

[0107] An intra ("I") image 510 may represent an image that is encoded or decoded using only its own information without the need to predict and encode other picture information in the reference video data. According to the video coding standard, the "I" image may represent a key frame, an independent / instantaneous decoder refresh (IDR) frame, a clean random-access (CRA) frame, etc. As mentioned above, the "I" images referred to by various names may have various variations and applications within the limits allowed by various standards, and there may be some differences between them. In addition to the methods listed above, various application methods for realizing the "I" image can be methods known to those skilled in the art or newly provided.

[0108] The prediction (P) image 520 may represent an image used to predict sample values ​​of blocks constituting the above-described image. It may be at least one prediction information specifying at least one reference image, and / or an image encoded or decoded based on motion vectors and via intra-frame or inter-frame prediction. According to video coding standards, the above-described "P" image may be configured to reference only one reference frame, or to reference more than one reference frame. When referencing more than one reference frame, sample information and / or related metadata derived from multiple reference images may be used to reconstruct a single block. However, generally, an image designated as a "P" image can be understood as limited to referencing images that are temporally earlier.

[0109] The bidirectional prediction (B) image 530 may represent at least one prediction information of at least two or more reference images specified for predicting sample values ​​of blocks constituting the above-described image, and / or an image encoded or decoded based on motion vectors and via intra-frame or inter-frame prediction. Generally, an image designated as a "B" image, distinct from an image designated as a "P" image, can be understood as an image that performs the reference, and is not limited to temporally earlier images. Instruction manual, pages 11 / 18, CN 121533011 A

[0110] During the encoding and decoding process, video data can be spatially divided according to multiple sample blocks and encoded according to the block units described above. As is well known, the block units described above may include, for example, sizes such as 4x4, 8x8, 4x8 or 16x16 in units of horizontal / vertical pixels, but are not limited thereto. The blocks described above may be encoded by referencing any other (already encoded) blocks and by a predictive coding method, depending on the respective images to which they belong and according to the specified type for allowance and / or restriction. For example, the blocks of “I” image 510 may not use a predictive coding method, or may be encoded by referring to already encoded blocks in the same local image. That is, only the so-called intra-frame prediction method may be used. In contrast, “P” image 520 may further refer to at least one reference image encoded in a previous time unit, and therefore may also be encoded using inter-frame prediction together with intra-frame prediction. “B” image 530 may also refer to a reference image that was previously encoded in the encoding order but was executed later in the time unit. However, it is well known that even within a "P" image or a "B" image, there may be blocks that are encoded without relying on predictive coding.

[0111] The video encoder 405 described above can be configured to perform encoding actions according to predetermined video compression techniques documented according to various international or commercial standards. Examples of the above standards may include all the standards described in the decoder described above.

[0112] According to one embodiment of the present invention, the transmitting unit 470 may buffer the encoded video data generated by the entropy encoding described above, so as to provide / send the video data (ultimately transmitted to the remote decoder 490) to the device storing the encoded video data via a hardware or software connection 495. According to an embodiment, when the transmitting unit 470 receives the encoded video data provided / sent from the video encoder 405, it may receive and merge other data (e.g., encoded audio data or other auxiliary data) accompanying the encoded video data from a separate source 480.

[0113] According to one embodiment of the present invention, the transmitting unit 470 may also be configured to send additional data along with the encoded video. The additional data may be considered as part of the encoded video data. The additional data may include information for the decoder, which is used to correctly decode the data or to more accurately reconstruct information closer to the pre-encoded image. Examples of the additional data may include all the examples described above with respect to the decoder receiving unit 310.

[0114] As described above, the present invention can be implemented using digital video compression standards widely used and understood by those skilled in the art. The aforementioned digital video compression standards may include at least one known compression standard, such as the standard names MPEG-2, MPEG-4 Video, H.263, H.264 / AVC, H.265 / HEVC, H.266 / VVC, VC-1, AVI, QuickTime, VP-9, VP-10, and Motion JPEG.

[0115] FIG6 is a conceptual diagram illustrating the structure of a video encoder according to another embodiment of the present invention. The structure shown in FIG6 corresponds to well-known standard encodings such as ITU-T H.266 and ISO / IEC 23090-3, as well as the general structure of a video encoder called MPEG-1 Part 3 or Versatile Video Coding (VVC).

[0116] According to FIG6, the video encoder 605 may be configured to receive an encoded bitstream 602 as input and output uncompressed or unencoded raw video data 601. During intra-frame coding, the video data 601 can be directly provided to the luma mapping unit 610a, or it can be provided to the luma mapping unit 610b via the inter-frame prediction unit 620, which includes motion vector extraction. In the case of intra-frame coding, the mapped luma signal can be provided not only to the output combiner 606 independently, but also to the output combiner 606 by selecting either the intra-frame prediction coded signal output by the intra-frame prediction unit 625 (608) or the inter-frame prediction coded signal output by the inter-frame prediction unit 620 from the luma mapping unit 610b.The result of the above-described output combiner can be applied to the chroma scaling unit 615 (the operations of the luminance signal mapping unit 610 and the chroma scaling unit 615 are collectively referred to as the luminance mapping / chroma scaling (LMCS) process). The scaled chroma signal can be provided to the transform unit 630, which in particular can perform an adaptive color transform on the chroma signal. The coefficients obtained from the above transform are applied to the quantization unit 640 and quantized as described on page 12 / 18 of the specification, CN 121533011 A. Thus, lossy compression can be achieved, and the result of the lossy compression can be output as a bitstream 602 using a lossless compression method based on a multi-hypothesis CABAC unit 650.

[0117] On the other hand, in order to generate the coding loop, the result of the aforementioned lossy compression can undergo inverse quantization 645, inverse transform 635, and luma signal amplification 617 before entering the actual decoding process. The result of the aforementioned luma signal amplification can be provided to the internal combiner 607 together with the result of the selection 608 of at least one previously generated intra-frame predictive coding signal or inter-frame predictive coding signal. After undergoing inverse luma mapping 617, the result of the aforementioned internal combiner can be processed by a deblocking filter 660, a sample adaptive offset (SAO) 670, an adaptive loop filter (ALF), etc., for reproducing the image quality improvement process in the decoder. As described above, the result of the reproduction action in the decoder can be applied to the reference image buffer 690 and can be reused for predictive coding by the aforementioned inter-frame prediction unit 620.

[0118] The present invention can also be implemented or used in combination with the next-generation video codec developed by the Joint Video Experts Team (JVET), an international standardization expert group.The aforementioned enhanced compression model may include: an improved intra-frame predictive coding method, an improved inter-frame predictive coding method, an improved transform and transform coefficient coding method, an improved adaptive loop filtering method, a bilateral filtering method, a novel sample adaptive offset (SAO) method for image quality improvement, an extended entropy coding method, and an improved gradual decoding refresh (GDR) technique.

[0119] Features of the Invention

[0120] The present invention relates to a video coding and decoding method and apparatus utilizing an improved intra-frame predictive method, which can be used in the field of video coding and decoding including the above embodiments.

[0121] FIG7 is a conceptual diagram illustrating a template matching prediction method according to an embodiment. The template matching prediction method according to this embodiment can be performed by a video decoder and / or a video encoder disclosed in this specification.

[0122] Referring to FIG7, the current picture may include a previously decoded (or encoded) area within it, namely a reconstructed area 790, and a current block 710 representing the encoding / decoding unit in progress of the current decoding (or encoding). Wherein, the aforementioned current image can be understood, according to the implementation, as referring to at least one of slice, frame, subpicture, and tile, and the aforementioned current block can be understood, according to the implementation, as referring to concepts such as block, sub-block, prediction unit, or coding unit.

[0123] On the other hand, among the samples constituting the aforementioned restored region 790 (i.e., which can represent pixels), the set of samples adjacent to the current block 710 can be defined as template 715.

[0124] The aforementioned template 715 may include samples belonging to the left block, top block, and upper left block of the aforementioned current block 710 and adjacent to the aforementioned current block 710, and can also be understood as referring to the set of L-shaped samples with a predetermined sample thickness 717 that are directly adjacent to the current block 710. The aforementioned sample thickness 717 can be specified as any value of 1 pixel or more. However, in this specification, for ease of technical understanding, it is described as 1 pixel. However, it should be readily understood that the present invention can also be applied in substantially the same way to templates with a sample thickness 717 of 2 pixels or more. Furthermore, the aforementioned sample thickness 717 can be non-fixed, variable, or different depending on the orientation or range of the template. Additionally, the aforementioned template 715 may further include samples contained in the lower left and / or upper right blocks of the aforementioned current block 710.

[0125] According to the template matching prediction method of this embodiment, a block 720 corresponding to the current block 710 can be searched within the restored region 790, and the corresponding block 720 can be used as a prediction block. Specifically, the template matching prediction method according to this embodiment may include: performing a template matching 735 by searching for the occurrence position of samples similar to those constituting the template 715 within the restored region 790; deriving a prediction template 725 based on the template matching 735; and using a block (or, which may represent any set of samples corresponding to it) that is adjacent to the prediction template 725 and corresponds to the current block 710 as the prediction block 720.

[0126] At this time, information representing the direction and distance from the current block 710 to the prediction block 720 can be defined as a block vector. Subsequently, the pixel information of the prediction block 720 can be used in combination with various conventional encoding methods in the encoding process of the current block 710. For example, it can be used for differential coding.

[0127] FIG8 is a conceptual diagram of the storage and use of block vectors in a template matching prediction method according to an embodiment. The template matching prediction method according to this embodiment can be performed by the video decoder and / or video encoder disclosed in this specification.

[0128] In the template matching prediction method of this embodiment, when the above-mentioned prediction block is used, the distance between the current block 810 and the prediction block 820 is defined as block vector 830, which can be used in subsequent encoding and decoding. The above-mentioned block vector 830 is not only used for the template matching prediction method, but can also be stored for purposes such as intra-frame block copy encoding and decoding methods. In addition, the decoding (or encoding) of the next block (e.g., 840) of the current block 810 can also be configured to refer to the above-mentioned block vector 850. According to an embodiment of the present invention, the above-mentioned block vector can be a value determined by the encoder and decoder respectively by calculation in the same way. In this case, it is not a problem even if the above-mentioned block vector is not included in the encoded bitstream.

[0129] On the other hand, when performing template matching, the complexity of the encoder / decoder may increase due to excessive computation in the restoration region. This is because the search range for template matching in the restoration region is too large, or the number of pixels within the template is too large. Furthermore, an efficient method is needed for storing the block vectors used in template matching prediction.

[0130] According to a template matching prediction method of an embodiment of the present invention, when searching for a prediction template within the search range of the restoration region, the prediction template can be searched by referring to (or using) a portion of the samples within the template (i.e., pixels).As described above, selectively using a portion of pixels within the original template to search for a prediction template can be termed downsampling.

[0131] On one hand, the template matching prediction method can directly use the selected sample itself to search for a prediction template.

[0132] On the other hand, the template matching prediction method can correct the selected sample by referring to the value of an unselected sample and use the corrected sample to search for a prediction template.

[0133] FIG9 is an example diagram of template downsampling according to an embodiment of the present invention.

[0134] Referring to FIG9, the template matching prediction method according to an embodiment of the present invention can be configured to use the peripheral samples adjacent to the left and top sides of the current block 910 as template 920. In this case, through downsampling, only a portion of the samples 930 can be selectively used for template matching calculation. For example, according to an embodiment of the present invention, as shown in FIG9(a), only the samples selected by downsampling from the initial template by selecting only about half of the samples can be used for template matching calculation. In Figure 9(a), an example is shown where, when the sample thickness of the template 920 is 1 pixel, approximately half of the pixels belonging to the template 920 are selected at intervals of a certain sampling interval (1 pixel interval in Figure 9(a)). However, this selection method by downsampling can be determined in various ways depending on the implementation method and is not limited thereto. For example, as shown in Figure 9(b), it can also be configured to select approximately 1 / 4 of the pixels belonging to the template 920 at intervals of 3 pixels. Obviously, various downsampling methods not shown can be used. In this way, calculations on all pixels of the template can be avoided, thereby reducing complexity.

[0135] According to another embodiment of the invention, the downsampling described above can be performed differently depending on the size of the template and / or the size of the block. That is, the number of samples required for calculation can be adjusted according to the size of the template or the size of the block. Specification 14 / 18 pages 18 CN 121533011 A

[0136] Figure 10 is an example diagram of template downsampling based on the size of the block according to an embodiment of the invention.

[0137] According to one embodiment of the template matching prediction method, whether to perform downsampling can be adaptively determined based on the size of the block. For example, the template matching prediction method may include the following steps: not performing downsampling when the size of the block is small (Figure 10(a)), and performing downsampling when the size of the block is large (Figure 10(b)).

[0138] A template matching prediction method according to another embodiment may include the following steps: as shown in FIG10(b), for relatively small blocks, a relatively high density downsampling is used, for example, downsampling of one sample approximately every two samples; and as shown in FIG10(c), for relatively large blocks, a relatively low density downsampling is performed, for example, downsampling of one sample approximately every four samples.

[0139] A template matching prediction method according to yet another embodiment may be configured to make the horizontal and / or vertical downsampling methods different according to the aspect ratio of the block. For example, as shown in FIG10(d), for blocks with a longer horizontal length, the downsampling density on the left side of the shorter template may be relatively high, or no downsampling may be performed; while the downsampling density on the upper side of the longer template may be relatively low.

[0140] The downsampling degree and method shown in the above examples are exemplary and are not limited to the present invention. For example, for larger blocks, it can be configured to use approximately one sample out of every four samples or one sample out of more than four samples; for smaller blocks, it can be configured to use approximately one sample out of every two samples or all samples; and various other downsampling ratios can be applied in different forms as required by the present invention.

[0141] According to another embodiment of the present invention, when performing template matching calculations within the search range of the restored region, the above-mentioned calculation method can be set for each compression unit, such as a video sequence, frame, slice, sub-image, tile, macroblock, or coding tree unit (CTU), preferably transmitted from the encoder to the decoder via symbols contained in the bitstream syntax information. If only one calculation method is used, no transmission is required. However, the possibility of applying different calculation methods is not excluded. For example, in one embodiment, it can be configured to use an algorithm that sums the absolute values ​​of the differences between pixels. In another embodiment, it can be configured to use an algorithm based on methods such as Hadamard transform. In yet another embodiment, multiple modes or methods of downsampling for template matching calculations can be introduced. As described above, when implementing multiple template matching calculation methods, the encoder can record which calculation method is applied to a specific sequence, frame, slice, sub-image, or tile as a symbol included in the above grammatical information in the bitstream, and the decoder can read the above symbol, derive and use the above calculation method.

[0142] According to an embodiment of the present invention, when setting the search range for template matching prediction, the search range can be set as follows: the search is performed centered on the search range block vector generated based on the block vector of the surrounding blocks or the block vector candidate list.

[0143] FIG11 is an example diagram of setting the search range of a block vector according to an embodiment of the present invention.

[0144] Referring to FIG11, the peripheral blocks adjacent to the current block 1130 may include at least one left block 1132 and an upper block 1134. Exemplarily, the aforementioned adjacent blocks 1132, 1134 may be blocks having block vectors and / or having candidate lists for block vector operations.

[0145] The block vector search method according to an embodiment of the present invention may include the step of generating a search range block vector 1110 based on the current block 1130 by referring to the block vector 1115 of the left block (e.g., by copying it in the same direction and distance by reflecting the inter-block distance). And it can be configured to calculate a search range 1140 of a certain size along the up, down, left, and right directions, based on the point 1117 pointed to by the search range block vector 1110 (or centered on that point).

[0146] A block vector search method according to another embodiment of the present invention may include: generating a search range block vector 1120 by referring to the block vector 1125 of the upper block; and calculating a search range 1145 of a certain size, based on the point 1127 pointed to by the block vector 1120. In this way, it is possible to avoid searching and calculating all pixels of the frame, thereby reducing complexity, and by referring to adjacent block vectors, the accuracy of prediction can also be guaranteed.

[0147] According to yet another embodiment of the block vector search method of the present invention, when setting the search range for template matching prediction, the search range information can be set for each compression unit, such as a sequence, frame, slice, sub-image, tile, macroblock, or CTU, etc. Preferably, it is configured to be transmitted from the encoder to the decoder through symbols included in the bitstream syntax information. For example, in one aspect, the search range can be defined as including information including horizontal and vertical dimensions, and the size of the search range can be changed and applied for each compression unit. For example, when selecting one of the block vector candidates from the surrounding blocks in order to generate the above-mentioned search range block vector, information about which block vector candidate was used to derive the above-mentioned search range block vector from the above-mentioned candidate list, and / or which block vector from the left and top sides was referenced, can be set for each of the above-mentioned compression units, and configured to be transmitted from the encoder to the decoder.

[0148] According to another embodiment of the block vector search method of the present invention, when it is necessary to refer to samples outside the above-mentioned search range, it can be configured to use the following padding method: extending and using values ​​belonging to the above-mentioned search range. For example, when searching outside compression units such as slices, frames, sub-images, tiles, macroblocks, coding tree units, etc., the above-mentioned padding method can be used.On the other hand, in another embodiment, when it is necessary to search outside of compression units such as slices, frames, sub-images, tiles, macroblocks, and CTUs, it can be configured to be treated as a boundary line and reset.

[0149] In one aspect, the setting of the above-mentioned search range can be determined within the search range of other prediction techniques that may be included in the encoder / decoder, such as intra-frame block copying. For example, it can be configured to set the above-mentioned search range by selecting a direction with a larger or smaller area compared to the search range used for intra-frame block copying.

[0150] According to another embodiment of the present invention, the prediction vector used for template matching can be configured to be used after referring to the block vectors in the surrounding blocks or the block vector candidate list and making individual corrections.

[0151] FIG12 is a conceptual diagram of a block vector-based prediction vector determination method according to an embodiment of the present invention.

[0152] Referring to FIG12, it is assumed that by searching the region of the existing and recovered image, the optimal position 1250 that most matches the current block 1230 (or a set of such samples) is confirmed. The optimal prediction vector 1260 pointing to the above-mentioned optimal position 1250 can also be calculated. In one embodiment of the present invention, in order to determine the above-mentioned optimal position, it can also be configured to: in addition to performing the above-mentioned template matching, it can also perform an additional comparison between the current block sample and the pixel sample in the restoration area, thereby enhancing the optimality of the template. At this time, when representing the above-mentioned optimal block vector 1260, the surrounding blocks can be referenced, in the example of FIG12, the block vector 1215 of the left block 1232, and by reflecting the distance between blocks, it is copied in the same direction and distance, and a reference prediction vector 1210 based on the current block 1230 is generated.

[0153] Next, the information representing the difference 1270 between the above-mentioned reference prediction vector and the above-mentioned optimal prediction vector can be represented as the size of the vector, flag information, or index information, and information composed of at least one of the above-listed information. Preferably, the above information is configured to be transmitted from the encoder to the decoder through symbols contained in the bitstream syntax. For example, when represented by the size of the vector, the information describing the differential vector 1270 can be transmitted as a symbol. For example, when represented by a flag, the aforementioned flag information can be transmitted as a symbol. As another example, when represented by an index, the index information of a table representing the various possibilities of the aforementioned difference vector can be transmitted as a symbol.

[0154] In one embodiment of the present invention, the modification of the aforementioned block vector can be configured to be performed only within a specified range.For example, the range for searching the optimal block vector 1260 to correct the above-mentioned block vector can be configured to be performed only within the limited search range shown in FIG11 (e.g., the search range represented by reference numerals 1140 or 1145). Of course, it is readily understood from the specification on pages 16 / 18 of 20 CN 121533011 A that various methods for limiting the search range can be used in addition to the example of FIG11 above.

[0155] According to an embodiment of the present invention, when storing the block vector used in template matching prediction, it can be configured to store the current block in an arbitrary block size, such as: 4X4, 8X8, 16X16, 32X32, 64X64, etc. During the video encoding process, the video can be divided and compressed into slices, frames, sub-images, tiles, macroblocks, CTUs or other arbitrary compression units, and can be configured to be divided and stored in an arbitrary block size after the above compression is performed. For example, it can be configured to store block vectors every 4×4 blocks during the current frame compression and every 16×16 blocks after compression. In one embodiment, it can be configured to calculate and store a representative block vector when multiple block vectors are stored as a block. In one embodiment, it can be configured to set the resolution or storage unit of the block vectors differently depending on the intra-frame or inter-frame frame and store them. In one embodiment, it can be configured to re-store the block vectors at an arbitrary block vector resolution after compression with the current slice or frame or sub-image or tile or arbitrary unit. In one embodiment, it can be configured to store the block vectors when there is an optimal block vector or a corrected block vector obtained using the current block.

[0156] When the stored block vectors are used in subsequent encoding and decoding processes, or when previously stored block vectors are used for template matching prediction of the current block, scaling can be performed after considering the resolution of the stored block vectors. For example, when the resolution of the stored block vectors is relatively high, a shift operator can be used to reduce the resolution. For example, when the resolution of the stored block vector is relatively low, a shift operator can be used to improve the resolution.

[0157] In one embodiment of the invention, at least one of the block's horizontal or vertical dimensions, area, number of pixels within the block, and / or information about surrounding blocks can be considered to determine whether to apply template matching prediction techniques. In this case, it can be configured to select and use a template corresponding to the left, top, or both the left and top sides of the current block. Furthermore, in one embodiment, it can be configured to determine whether to combine template matching-based prediction techniques with other prediction techniques.

[0158] As an example according to an embodiment, when the block's horizontal dimension is greater than a first threshold and its vertical dimension is greater than a second threshold, it can be configured to restrict the use of template matching.For another example, when the horizontal size of the block is less than the third threshold and the vertical size is less than the fourth threshold, it can be configured to restrict the use of template matching. For yet another example, when the number of pixels in the block is greater than or less than the fifth threshold, it can be configured to restrict the combined use of other techniques besides the template matching prediction technique mentioned above. For yet another example, when the area of ​​the block is greater than or less than the sixth threshold, it can be configured to restrict the combined use of the template matching prediction technique mentioned above with other techniques. For example of the above-mentioned restriction combination, it can mean that the block conditions are taken into account and the combination of geometric segmentation prediction and template matching prediction is restricted.

[0159] According to an embodiment of the present invention, when template matching prediction is used in the chroma (which may refer to Cb / Cr) signal region, it can be configured to directly copy at least one block vector from the block corresponding to the same position based on the luminance (which may refer to Y) signal and use it as the block vector of the chroma signal block. According to another embodiment, it can be configured to re-perform template matching on the chroma block based on the block vector of the corresponding luminance block. According to another embodiment, it can be configured to set the search range of template matching based on the block vector of the corresponding luminance block, and re-execute template matching. According to another embodiment, it can be configured to refer to the samples in the color difference block corresponding to the area indicated by the block vector of the corresponding luminance block.

[0160] As described above, in the case of inferring the prediction block based on the information of the luminance block in the color difference signal region, according to an embodiment of the present invention, it can be configured to: use the sample indicated by at least one block vector in the block located at the same position as a candidate vector, based on the luminance signal block. For example, it can be configured to perform a weighted average of more than one candidate sample to generate another candidate sample.

[0161] The present invention has been described above with reference to the accompanying drawings and embodiments, but as described above, this does not mean that the scope of protection of the invention is limited to the drawings or embodiments disclosed above. Those skilled in the art should understand that various modifications and changes can be made to the present invention without departing from the inventive concept and scope set forth in the claims of the present invention.Instruction manual page 18 / 18, Figure 1 (CN 121533011 A), Figure 2 (CN 121533011 A), Figure 3 (CN 121533011 A), Figure 4 (CN 121533011 A), Figure 5 (CN 121533011 A), Figure 6 (CN 121533011 A), Figure 7 (CN 121533011 A), Figure 8 (CN 121533011 A), Figure 9 (CN 121533011 A), Figure 10 ... Figure 11 of the instruction manual, page 32 of 10 / 12, CN 121533011 A; Figure 12 of the instruction manual, page 34 of 12 / 12, CN 121533011 A.

Claims

1. A template-matching-based decoding method performed by a decoder, characterized in that, include: The step of obtaining information about the template set of samples adjacent to the current block from the restored region of the current image; Within the search area encompassed by the restored area, a template matching step is performed to search for the predicted template corresponding to the template; The step of selecting a prediction block adjacent to the prediction template; as well as The step of performing intra-frame prediction decoding on the current block based on the prediction block. The template matching is performed by using a portion of the samples within the template or by using a portion of the search region.

2. The method according to claim 1, characterized in that, The steps for performing the template matching include: The step of downsampling the template; and The step of searching for and predicting the template location by comparing the samples of the downsampled template with the samples of the search region.

3. The method according to claim 2, characterized in that, The downsampling steps include: Based on the horizontal size of the current block, the step of adjusting at least one of the following for the upper sample of the template: whether or not downsampling is performed, and the scaling factor.

4. The method according to claim 2, characterized in that, The downsampling steps include: Based on the vertical dimensions of the current block, the step of adjusting at least one of the following for the left sample of the template: whether or not downsampling is performed, and the magnification.

5. The method according to any one of claims 3 or 4, characterized in that, The downsampling step Execution is based on downsampled symbol information derived from the bitstream.

6. The method according to claim 1, characterized in that, The search area, It is constrained based on the prediction vector from the current block.

7. The method according to claim 6, characterized in that, The prediction vector, It is generated based on at least one reference prediction vector information of the surrounding blocks of the current block.

8. The method according to claim 7, characterized in that, The search area, With the point pointed to by the prediction vector as the center point, it has a first sample width in the horizontal direction and a second sample width in the vertical direction.

9. The method according to claim 8, characterized in that, At least one of the first sample width and the second sample width is derived from the bit stream.

10. The method according to claim 6, characterized in that, The prediction vector, It is selected by indexing from at least one of the block vector candidates that the current block has in its surrounding blocks.

11. The method according to claim 10, characterized in that, The index is derived from the bitstream.

12. The method according to claim 6, characterized in that, The prediction vector, It is generated based on the difference vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the reference prediction information.

13. The method according to claim 6, characterized in that, Also includes: The step of comparing the current block sample with samples from the search region contained in the restored area of ​​the image to search for the best predicted block location that is closest to the current block sample. The prediction vector, It is generated based on the difference vector information represented by the difference between at least one reference prediction vector information of the surrounding blocks of the current block and the best prediction vector information indicating the position of the best prediction block.

14. The method according to any one of claims 12 or 13, characterized in that, The differential vector information is derived from the bit stream.

15. The method according to claim 14, characterized in that, The difference vector information is specified by an index indicating any one of at least two predefined values.

16. The method according to claim 6, characterized in that, Also includes: The decoder stores the predicted vector for use in subsequent decoding steps.

17. The method according to claim 6, characterized in that, Also includes: The step of generating the prediction vector for at least one lower-level decoding unit; For a higher-level decoding unit that includes the lower-level decoding unit, the step of designating one of the at least one prediction vector as a representative prediction vector; and The step of storing the representative prediction vector through the decoder.

18. The method according to claim 1, characterized in that, Also includes: When performing template matching in the chromatic difference signal region, the step of generating a prediction vector is based on at least one reference prediction vector corresponding to a block at the same position in the luminance signal region with the chromatic difference signal region as a reference.

19. A decoder device, characterized in that, include: A template matching unit is configured to obtain information about templates from the restored region of the current image, which are a set of samples adjacent to the current block, and perform template matching within a search region contained in the restored region to search for a predicted template corresponding to the template; and The prediction decoding unit is used to select a prediction block adjacent to the prediction template and perform intra-frame prediction decoding on the current block based on the prediction block. The template matching is performed by using a portion of the samples within the template or by using a portion of the search region.

20. An encoder device, characterized in that, include: A template matching unit is configured to obtain information about templates from the encoded regions of the current image, which are sets of samples adjacent to the current block, and to perform template matching within a search region encompassed by the encoded regions, searching for a predicted template corresponding to the template; and The predictive coding unit is used to select a prediction block adjacent to the prediction template and perform intra-frame predictive coding on the current block based on the prediction block. The template matching is performed by using a portion of the samples within the template or by using a portion of the search region.